Brief Explination of Thread Pool.Why do we use Thread Pool

      Thread pool is where a number of threads are created to perform a number of tasks, which are usually organized in a queue. The results from the tasks being executed might also be placed in a queue, or the tasks might return no result (for example, if the task is for animation). Typically, there are many more tasks than threads. As soon as a thread completes its task, it will request the next task from the queue until all tasks have been completed. The thread can then terminate, or sleep until there are new tasks available.

 

      The number of threads used is a parameter that can be tuned to provide the best performance. Additionally, the number of threads can be dynamic based on the number of waiting tasks. For example, a web server can add threads if numerous web page requests come in and can remove threads when those requests taper down. The cost of having a larger thread pool is increased resource usage. The algorithm used to determine when to create or destroy threads will have an impact on the overall performance:

• create too many threads, and resources are wasted and time also wasted creating any unused threads
• destroy too many threads and more time will be spent later creating them again
• creating threads too slowly might result in poor client performance (long wait times)
• destroying threads too slowly may starve other processes of resources

     The algorithm chosen will depend on the problem and the expected usage patterns.
If the number of tasks is very large, then creating a thread for each one may be impractical.
Another advantage of using a thread pool over creating a new thread for each task is thread creation and destruction overhead is negated, which may result in better performance and better system stability. Creating and destroying a thread and its associated resources is an expensive process in terms of time. An excessive number of threads will also waste memory, and context-switching between the runnable threads also damages performance.

      For example, a socket connection to another machine—which might take thousands (or even millions) of cycles to drop and re-establish—can be avoided by associating it with a thread which lives over the course of more than one transaction.

      When implementing this pattern, the programmer should ensure thread-safety of the queue. In Java, you can synchronize the relevant method using the synchronized keyword. This will bind the block modified with synchronized into one atomic structure, therefore forcing any threads using the associated resource to wait until there are no threads using the resource. As a drawback to this method, synchronization is rather expensive. You can also create an object that holds a list of all the jobs in a queue, which could be a singleton.

       Typically, a thread pool executes on a single computer. However, thread pools are conceptually related to server farms in which a master process, which might be a thread pool itself, distributes tasks to worker processes on different computers, in order to increase the overall throughput. Embarrassingly parallel problems are highly amenable to this approach.
  
Use of Thread pool:

       Thread Pools are useful when you need to limit the number of threads running in your application at the same time. There is a performance overhead associated with starting a new thread, and each thread is also allocated some memory for its stack etc.
Instead of starting a new thread for every task to execute concurrently, the task can be passed to a thread pool. As soon as the pool has any idle threads the task is assigned to one of them and executed. Internally the tasks are inserted into a Blocking Queue which the threads in the pool are dequeuing from. When a new task is inserted into the queue one of the idle threads will dequeue it successfully and execute it. The rest of the idle threads in the pool will be blocked waiting to dequeue tasks. 
 
     Thread pools are often used in multi threaded servers. Each connection arriving at the server via the network is wrapped as a task and passed on to a thread pool. The threads in the thread pool will process the requests on the connections concurrently. A later trail will get into detail about implementing multithreaded servers in Java. 
 
      Java 5 comes with built in thread pools in the java.util.concurrent package, so you don't have to implement your own thread pool. You can read more about it in my text on the java.util.concurrent.ExecutorService. Still it can be useful to know a bit about the implementation of a thread pool anyways.

Here is a simple thread pool implementation:

public class ThreadPool {

  private BlockingQueue taskQueue = null;
  private List threads = new ArrayList();
  private boolean isStopped = false;

  public ThreadPool(int noOfThreads, int maxNoOfTasks){
    taskQueue = new BlockingQueue(maxNoOfTasks);

    for(int i=0; i

public class PoolThread extends Thread {

  private BlockingQueue taskQueue = null;
  private boolean       isStopped = false;

  public PoolThread(BlockingQueue queue){
    taskQueue = queue;
  }

  public void run(){
    while(!isStopped()){
      try{
        Runnable runnable = (Runnable) taskQueue.dequeue();
        runnable.run();
      } catch(Exception e){
        //log or otherwise report exception,
        //but keep pool thread alive.
      }
    }
  }

  public synchronized void stop(){
    isStopped = true;
    this.interrupt(); //break pool thread out of dequeue() call.
  }

  public synchronized void isStopped(){
    return isStopped;
  }
}

       The thread pool implementation consists of two parts. A ThreadPool class which is the public interface to the thread pool, and a PoolThread class which implements the threads that execute the tasks.

       To execute a task the method ThreadPool.execute(Runnable r) is called with a Runnable implementation as parameter. The Runnable is enqueued in the blocking queue internally, waiting to be dequeued.

       The Runnable will be dequeued by an idle PoolThread and executed. You can see this in the PoolThread.run() method. After execution the PoolThread loops and tries to dequeue a task again, until stopped.
      
       To stop the ThreadPool the method ThreadPool.stop() is called. The stop called is noted internally in the isStopped member. Then each thread in the pool is stopped by calling PoolThread.stop(). Notice how the execute() method will throw an IllegalStateException if execute() is called after stop() has been called.
        
        The threads will stop after finishing any task they are currently executing. Notice the this.interrupt() call in PoolThread.stop(). This makes sure that a thread blocked in a wait() call inside the taskQueue.dequeue() call breaks out of the wait() call, and leaves the dequeue() method call with an InterruptedException thrown. This exception is caught in the PoolThread.run() method, reported, and then the isStopped variable is checked. Since isStopped is now true, the PoolThread.run() will exit and the thread dies.