5.2.1 Connector Implementation

When a message enters the system, the associated connector spawns a new thread to handle the request. At that point, the message is sent to the configured handler chain, which typically includes a handler to translate the transport payload into the objects expected by the Agent internally, and the servlet handler, which is the entry point into the servlet container that runs the service. Each connector contains a unique instance of the handler chain and all handlers configured in the chain.

5.2.2 Optimizing Connector Thread Pools

Connector thread pools are configured in the connector definitions in rwagent.xml. Here is an example from the default HTTP connector definition. The values shown are also the default values for the other defined connectors.

<rwsf:connector name="HTTP/1.1" class="rwsf_transport_http35.createHttpConnectorImp" handlerChain="http"> <rwsf:property name="acceptor-threads" value="1"/> <rwsf:property name="thread-pool-min" value="15"/> <rwsf:property name="thread-pool-max" value="50"/> ... </rwsf:connector>

The acceptor-threads property is used to configure the number of threads for taking requests off the wire and passing them to the message handling threads. This operation is simple and fast, so generally one thread is sufficient to keep up with incoming requests. The intent is to avoid connection time out errors even if the Agent is severely backed up in the handling of service requests. If your requests may arrive so fast that they will overwhelm the default single acceptor thread, simply add another thread or two.

---

Be aware that multiple acceptor threads results in a small possibility that requests may not be queued up in the order received. This is because Agent low-level threading uses the operating system's thread scheduler. The scheduler could switch from a thread that has taken an incoming request off the socket but has not yet enqueued it, to another acceptor thread that takes another request off the socket and enqueues it before control returns to the first acceptor thread. These two incoming requests would then be enqueued in reverse order.

---

When the Agent starts up, it creates a message handling thread pool with thread-pool-min number of threads. If the load increases to the point where there are more than thread-pool-min service requests in this layer, the Agent creates additional threads up to a maximum of thread-pool-max. Once the number of threads increases, it stays at that level until the Agent is shut down and restarted.

The obvious question is: what are the optimum minimum and maximum values? Here a couple of considerations:

• What is the anticipated load on the connector? A high load suggests a higher number of threads in the pool.

• How processor-intensive is the service processing? The whole point of thread pools is to keep the system processors busy, whatever the number of processors. If service execution relies mainly on processor cycles, fewer threads are needed because service execution will be more likely to finish quickly, freeing its thread to accept a new service request. If, on the other hand, service processing involves a lot of blocking time, when an operation is waiting on something like I-O, it increases the chance that all of the services in all of the threads will be blocking and the processors will sit idle. In this case, you want to enlarge the thread pool to decrease the likelihood of this happening.

By considering the above factors, you can make an educated guess on how large to make your thread pool, both initially and under load. Ultimately, though, the only way to determine the optimum values is empirically. Set some values, put the system under load, and gather some metrics. Then change the values and gather those numbers. Eventually you should be able to arrive at a set of values that yields the best performance.