Many customers report scalability problems with their application servers - when the traffic goes up, the response time gets much worse, and when the traffic reaches a critical point, their application server grinds to a halt.

The scalability problem is inevitable with high traffic levels, but ZXTM has several features which can be used to manage this problem and greatly increase the capacity of your application servers.

Request and response buffering

Your application servers scale poorly when they have to manage large numbers of simultaneous connections. They spend more and more time switching between connections, and less and less time doing real work. This problem is exacerbated when your clients are connecting over slow, unreliable networks like the Internet because connections take much longer to start and complete. Features like HTTP keepalives only make this situation worse, as the application servers must manage large numbers of idle keepalive connections as well!

ZXTM acts as a complete HTTP proxy, managing slow connections from the clients independently from fast connections to your local application server:

• ZXTM can reduce the number of TCP connections it uses to communicate with the back-end servers. It does this by multiplexing many HTTP requests from different clients down a single HTTP connection
• ZXTM carefully restricts the number of idle keepalive connections with its servers: this reduces the number of connecitons that each server must manage
• ZXTM's request and response buffering is used to ensure that servers can work as quickly as possible, reading requests and writing the entire response without any network delay.

So, ZXTM solves problems that are caused by clients connecting over slow networks, using long-lived keepalive connections, etc - these are the primary problems that cause the 'avalanche' problem that afflicts the large majority of application server clusters.

Proactively managing connections when things go slow

ZXTM uses a unique technology called 'TrafficScript'. This is a programming language which allows you to write rules which control how ZXTM handles each network connection. TrafficScript is very powerful, and you can do a lot of useful things with it.

Here are some examples of what can be done, but they just illustrate the possibilities. These examples can be changes and extended in many ways using the capabilities of TrafficScript to manage your traffic appropriately.

How does ZXTM monitor response time?

ZXTM uses 'Service Level Monitoring' (SLM) classes to monitor response time. You can configure an SLM class with your desired response time (eg, 200ms), and then ZXTM will tell you if your servers are taking longer than that to respond.

For example, suppose you want to test if your servers take longer than 200ms to respond for .jsp requests:

Create an SLM class called 'Servlet Pages' with a response time of 200ms; and then use the following trafficscript rule:

$path = http.getPath();  # returns "/path/to/servlet.jsp"
if( string.endsWith( $path, ".jsp" ) ) {
   connection.setServiceLevelClass( "Servlet Pages" );
}

Now, ZXTM will monitor all requests for .jsp files; you can use the UI to graph the response time in real-time, and ZXTM can be configured to log a message or email you if too many connections don't get served within the 200ms threshold.

How can ZXTM's behaviour be customized?

Suppose you want the following logic:

• If my jsp pages are a little slow (less than 20% of requests take too long), I want to apply a random backoff to delay each connection
• If my JSP pages are very slow (more than 20% of requests take too long), I want to do the following:
  • If the user does not have a 'JSPSESSIONID' cookie, send him to a 'too busy - try later' page. This will stop new users from establishing a session.
  • If the user does have a 'JSPSESSIONID' cookie and he comes from my internal IP address range, I want to wait for up to 5 seconds to test if the servers run faster again. If not, I want to send an error message.

Here is an example of a trafficscript rule that will implement this logic (the rule hasn't been tested, so apologies for any errors!):

# we're only concerned about .jsp requests
# don't run this rule for other requests

$path = http.getPath();
if( ! string.endsWith( $path, ".jsp" ) ) {
   break;
}

# What percentage of .jsp pages are served fast enough?
$tolerance = slm.confirming( "Servlet Pages" );

if( $tolerance == 100 ) {
   # everything is OK.  Let this connection through, and
   # remember to time it with the SLM class
   connection.setServiceLevelClass( "Servlet Pages" );
   break;
}

if( $tolerance >= 80 ) {
   # things are a little slow.  pause the connection for
   # up to 2000 milliseconds, then let it through
   connection.sleep( math.random( 2000 ) );
   connection.setServiceLevelClass( "Servlet Pages" );
   break;
}

# Otherwise, things are very slow ($tolerance < 80).

# If the user does not have a JSPSESSIONID cookie, he has
# not started a transaction yet.  Don't let him through;
# send him to '/too-busy.html' instead.

if( ! http.getCookie( "JSPSESSIONID" ) ) {
   http.sendResponse( "302 Redirect", "text/plain",
      "We are too busy", "Location: /too-busy.html" );
}

# If the user is internal (comes from 10.*.*.*), then
# delay him for up to 5 seconds.  If conditions improve,
# let him through; otherwise, send him away.

$clientIP = request.getRemoteIP();
if( string.ipmaskMatch( $clientIP, "10.0.0.0/8 ) ) {
   $count = 5;
   while( $count > 0 ) {
      connection.sleep( 1000 );
      if( slm.conforming( "Servlet Pages" ) >= 80 ) {
         # things are better - break out of the 'while' loop
         break;
      }
      $count = $count - 1;
   }
   
   # If we timed out, send a redirect
   if( $count == 0 ) {
      http.sendResponse( "302 Redirect", "text/plain",
         "We are too busy", "Location: /too-busy.html" );
   }
}

# let the connection through, and remember to time it.
connection.setServiceLevelClass( "Servlet Pages" );

Owen Garrett [Zeus Dev Team] 01 July 2005