How can you respond immediately to sudden increases in load, even at night, and still sleep soundly in your bed? Automated Dynamic Provisioning of course!

The popularity of Virtual Infrastructures seems to be increasing on a daily basis, and besides the obvious savings in power, cooling, and hardware, a major benefit of the Virtual data center is increased manageability, and dynamic provisioning. Regardless of whether you think of ZXTM as an Application Delivery Controller, or an Application Traffic Manager, its position in the network means that it is perfectly placed to help you take control of your virtual environment.

ZXTM has always had the ability to accelerate web applications, and optimize throughput and resource utilization. Sophisticated monitoring, rate shaping, content caching, and the various process and network offloading features of ZXTM have all helped manage, and better utilize the available resources. Now with ZXTM 5.0 and Java Extensions, we can go a step further and actually provision new resources on the fly, directly from ZXTM.

A Detailed Example of Dynamic Provisioning

The ability to control VMware Virtual Infrastructures can be used in combination with a ZXTM Service Level Monitoring class (SLM). This gives us the ability to dynamically deploy a new node, when the SLM class detects that the service is under load. A simple TrafficScript™ rule can check the conformance of a service, and in the event that the percentage of conforming requests drops below a predetermined thresh hold, it can initiate provisioning. The TrafficScript rule will call a Java Extension and direct it to power on a new node, once the power change has completed, the extension could then connect back to ZXTM and add that new node into the service pool. Recently I have been working on a Java Extension to do just that.

The VMPoolControl Java Extension

The VMPoolControl Java Extension can be used to manage the power state of virtual machines running in a VMware Virtual Infrastructure. It uses a configuration file at start up to locate the VMware VI Server, ZXTM SOAP control port, and get a list of pools that can be managed by the Java Extension. The Java Extension takes one initialization parameter, "ZEUSHOME", which should be the full path to the location of the ZXTM install, on a ZXTM appliance this would be /opt/zeus/.

The Java Extension uses the ZEUSHOME parameter to locate the extensions configuration file. The reason for doing this, is so that the configuration file can be uploaded to the ZXTM via the UI "Extra Files" catalog page.

The Configuration file is plain text and should look something like this:

viurl = https://10.100.1.42/sdk
viuser = admin
vipass = admin
vidatacenter = Zeus Systems Engineering
zxtmurl = https://admin:admin@10.100.50.10:9090/soap
seconds = 10
draintime = 15

webservice = vmWeb1,10.100.50.21:80,on vmWeb2,10.100.50.22:80,on vmWeb3,10.100.50.23:80,off

The viurl, viuser, and vipass should hold the URL and credentials for accessing the VI server. The vidatacenter should match the data center which contains the virtual machines for your pools. The zxtmurl, should be the URI of ZXTMs management server. Next are two optional timing settings, the seconds value is the number of seconds you want the Java Extension to delay after completing a task, and the draintime, is the amount of time you want a node to drain before being suspended. The defaults are 10 and 15 respectively. The seconds delay is to give ZXTM time to re-evaluate the load and SLM conformance, once the new server has come on line or been suspended.

All other lines in the file are treated as pools, the format of a pool line should start with the name of the pool followed by the equals "=" sign. Then there is a space separated list of node information. The node information is comma separated, and contains: the node name (should match the vmname in ESX), the node IP:Port (should match the node in ZXTM), and the power state.

As you can see, I have only one pool, called "webservice", which contains three nodes. vmWeb1 and vmWeb2 are powered on, while vmWeb3 is powered off, or suspended. I would recommend nodes are left suspended, so that they come into service quickly when required. This configuration matches the VMware view of the data center too.

To make use of the VMPoolControl Java Extension, you will need to upload a few pre-requisite Java libraries first. You need the vim, vim25, and apputils libs from the VMware VI SDK. You will also need the com.zeus.soap.zxtm_1_0.jar for the ZXTM Control API, and any pre-requisites those libraries may require, probably Apache commons logging, and a few others; read the ZXTM-API.jar article for more details.

Once all dependencies are met, you can run the Java Extension from a TrafficScript rule using the java.run() function:

java.run("com.zeus.vmware.VMPoolControl", "poweron", "webservice");

The first argument is the fully qualified name of the Java Extension, in the case of VMPoolControl, that's com.zeus.vmware.VMPoolControl, the second argument should be either "poweron" or "suspend" to perform the appropriate power action on the VM, and the final argument is a pool name.

The VMPoolControl extension will pick which node to power on or off, and this decision is based on the order that the node appears inside the configuration file. Nodes appearing towards the end of the list will be suspended sooner, and powered on later, while nodes towards the beginning, will be powered on sooner and suspended later. For this reason, you should list them in order of running preference.

That's all you need to know about the Java Extension. The next step is to create a TrafficScript rule, which will decide when and how to make use of VMPoolControl.

Using VMPoolControl with TrafficScript

To make intelligent use of the Java Extension, we're going to assign a SLM class to the virtual server, and in the event that the SLM conformance falls below 80%, we will run the Java Extension, and ask it to power on a node for us.

$slm = slm.conforming();
if ( $slm < 80 ) {
   java.run("com.zeus.vmware.VMPoolControl", "poweron", "webservice");
}

That's pretty much all there is to it. Go home and sleep soundly, for if your server gets busy overnight, then ZXTM and VMware will take care of it for you.

Not convinced? Take a look at this little video of the VMPoolControl Extension in action.

Further Information & Downloads

Provisioning actions via SOAP control messages are comparatively slow. For that reason, the VMPoolControl Java Extension is a threaded class, consisting of a worker thread, which does most of the work, and the service thread, which is called by ZXTM. The Service thread simply checks to see if a task is running and starts one if there is not, otherwise it does nothing. The worker thread is responsible for connecting to VMWare and initiating the power change task, it then waits for the task to complete, and then it connects to ZXTM in order to add/remove the node from the pool.

You may download the source code and compiled versions of the VMPoolControl Java Extension below:

The source code for VMPoolControl The VMPoolControl Java Extension Binary

The VMPoolControl Java Extension also has the ability to suspend virtual machines. When a suspend action is requested, the extension first drains the node, then it removes the node from the ZXTM pool, and finally suspends the virtual machine. I have not made use of that feature here, but you could modify the TrafficScript to suspend nodes when a service is mostly idle. I'll leave that as an exercise for the reader.

For further information, read the source code! ;-)

Mark Boddington [Zeus Systems Engineering] 29 August 2008