Thread Pool

A thread pool is a pre-created set of worker threads that pull work from a shared queue. The application submits jobs. The pool hands each job to a free worker. From ECE459 L09 (also from Hemal Shah crash course).

Why?

Spawning a thread per task is expensive. If tasks are short, the create + destroy cost dominates the actual work. A pool amortizes that cost across many jobs.

What you get:

• Workers are created once and reused
• Pool size scales to the hardware, so the app doesn’t need to know the core count
• Natural backpressure when all workers are busy

How many threads to use?

Depends on the workload:

• Compute-bound: fewer than the number of virtual CPUs
• I/O-bound: more, since threads spend most of their time blocked

Amdahl’s Law bounds the useful count either way

Implementations

• Java: java.util.concurrent.ThreadPoolExecutor
• C#: System.Threading.ThreadPool
• GLib: GThreadPool
• Rust: the threadpool crate

Example

Example from L09. A pool of 8 workers, a shared VecDeque with 4000 jobs plus a -1 sentinel, and 4 consumer jobs submitted via pool.execute:

use std::collections::VecDeque;
use std::sync::{Arc, Mutex};
use threadpool::ThreadPool;
 
fn main() {
    let pool = ThreadPool::new(8);
    let queue = Arc::new(Mutex::new(VecDeque::new()));
    println!("main thread has id {}", thread_id::get());
 
    for j in 0..4000 {
        queue.lock().unwrap().push_back(j);
    }
    queue.lock().unwrap().push_back(-1);
 
    for _ in 0..4 {
        let queue_in_thread = queue.clone();
        pool.execute(move || {
            loop {
                let mut q = queue_in_thread.lock().unwrap();
                if !q.is_empty() {
                    let val = q.pop_front().unwrap();
                    if val == -1 {
                        q.push_back(-1);
                        println!("Thread {} got the signal to exit.", thread_id::get());
                        return;
                    }
                    println!("Thread {} got: {}!", thread_id::get(), val);
                }
            }
        });
    }
    pool.join();
}

One execute = one job, not one-per-worker

pool.execute(...) queues one job. With 4 consumer loops and a pool of 8, only 4 workers actually run. That’s why the loop submits the consumer closure 4 times, once per worker that should run it.

Output is interleaved since workers race for the queue lock:

main thread has id 4455538112
Thread 123145474433024 got: 0!
Thread 123145474433024 got: 1!
Thread 123145474433024 got: 2!
...
Thread 123145478651904 got: 3999!
Thread 123145476542464 got the signal to exit.
Thread 123145484980224 got the signal to exit.
Thread 123145474433024 got the signal to exit.
Thread 123145478651904 got the signal to exit.

The -1 sentinel gets pushed back after each consumer sees it, so every worker eventually exits cleanly.

Related

• Amdahl’s Law