Geographic location detection using TrafficScript

Sometimes you may want to determine the country of origin of each remote client so that you can act on this information using TrafficScript™. This article describes how to use Maxmind's free GeoLite Country database within ZXTM to determine a client's geographic location from their source IP address.

For example, you may wish to present two different versions of your home pages, with national news stories and advertisements for local visitors and international ones for other visitors. If you are hosting content that is restricted under export regulations, you may wish to redirect users from restricted locations.

This article is quite lengthy. It describes:

• How to obtain a source IP-to-country database and create a version suitable for use in ZXTM;
• How to import the database into ZXTM and search it with a TrafficScript rule;
• A simple example that tests the TrafficScript code;
• Another example showing how to ban users from particular locations.

The Source Database

In this article, we'll use Maxmind's free GeoLite Country database. If it's not sufficiently accurate, you can use their commercial GeoIP Country database instead.

The database contains ranges of IP addresses and their corresponding countries. For the purposes of reading the database within TrafficScript, it's easiest to convert the database into a compact array of IP ranges that can be searched quickly using a binary search:

+------+-----------+------+-----------+------+-----------+ ... +------+-----------+ 
| IP 1 | Country 1 | IP 2 | Country 2 | IP 3 | Country 3 | ... | IP n | Country n | 
+------+-----------+------+-----------+------+-----------+ ... +------+-----------+ 

This array will be constructed as a string; each IP address as a 4-byte integer, and each country as the two-letter country code (so the string is 6*n bytes long). An IP address in the range IPi <= IP < IPi+1 is located in Countryi. Where there are gaps in the database, these are identified with the two-letter country code '??'.

The following Perl script reads the CSV version of the GeoLite Country database and outputs the array:

• Download ip2array.pl

Save this file in the same folder as the GeoIpCountryWhois.csv CSV database file, and run it as follows:

• Linux/Unix
  

  $ chmod +x ./ip2array.pl
  $ ./ip2array.pl GeoIpCountryWhois.csv geoip.dat
  

• Windows

  Install ActivePerl (the free standard version is sufficient), then run the perl script:

  C:\Folder\Name> ip2array.pl GeoIpCountryWhois.csv geoip.dat
  

This operation will write the array to a file named 'geoip.dat'. Using the September 2007 version of the database, this file is 579,432 bytes long. It contains entries for 232 ISO 3166 country codes, along with codes for proxies and satellite providers.

Copy this file to your ZEUSHOME/zxtm/conf/extra folder on your ZXTM machine using scp or an equivalent. For example, to copy it to a ZXTM appliance at 192.168.1.1 from a Windows system, you can use the free pscp utility:

C:\Folder\Name> pscp geoip.dat root@192.168.1.1:/opt/zeus/zxtm/conf/extra

The TrafficScript code

Now that you've copied the geoip.dat file to your zxtm/conf/extra folder, you can read it using the resource.get() function in TrafficScript. This efficient function reads the file once from disk and caches it until it changes, so even though the file is 0.5Mb, there is no performance hit.

The following code looks up the client's remote address and converts it to 4-byte integer form. It then binary-searches the array and determines the two-character country code:

$ipaddr = request.getRemoteIP();

# Integer representation of $ipaddr >> 1
string.regexmatch( $ipaddr, "(\\d+)\\.(\\d+)\\.(\\d+)\\.(\\d+)" );
$ip = ((($1*256+$2)*256+$3)*128+$4/2);

$arr = resource.get( "geoip.dat" );

# initialize indices
$i = 0; $j = string.len( $arr )/6-1;

# $arr[$i] <= $ip < $arr[$j]
# iteratively halve the distance between $i and $j until they are adjacent

while( $j-$i > 1 ) {
   # midpoint between $i and $j
   $k = ($i+$j)/2;

   # compare $ip with $arr[$k]
   if( string.bytesToInt( string.subString( $arr, $k*6, $k*6+3 ) ) > $ip ) {
      $j = $k;
   } else {
      $i = $k;
   }
}

# Now, $arr[$i] <= $ip < $arr[$j] and $j == $i+1
# Look up the 2-character country code (returns '??' if unknown)
$ccode = string.subString( $arr, $i*6+4, $i*6+5 );

One point to note: all integers in TrafficScript are stored as 32-bit signed integers. To avoid integer overflows and keep the code simple, all IP addresses are converted to 31-bit integers (shifting-right to drop the bottom-most bit); the array already contains the 31-bit values.

Testing the code

It's quite easy to test this conversion. Create a request rule using the following TrafficScript code and assign it to an HTTP virtual server (you can create a new one with no back-end nodes if you like, or use an existing one).

if( !string.startsWith( http.getPath(), "/geo" ) ) break;

$ipaddr = http.getFormParam( "ip" );

# Integer representation of $ipaddr >> 1
string.regexmatch( $ipaddr, "(\\d+)\\.(\\d+)\\.(\\d+)\\.(\\d+)" );
$ip = ((($1*256+$2)*256+$3)*128+$4/2);

$arr = resource.get( "geoip.dat" );

# initialize indices
$i = 0; $j = string.len( $arr )/6-1;

# $arr[$i] <= $ip < $arr[$j]
# iteratively halve the distance between $i and $j until they are adjacent

while( $j-$i > 1 ) {
   # midpoint between $i and $j
   $k = ($i+$j)/2;

   # compare $ip with $arr[$k]
   if( string.bytesToInt( string.subString( $arr, $k*6, $k*6+3 ) ) > $ip ) {
      $j = $k;
   } else {
      $i = $k;
   }
}

# Now, $arr[$i] <= $ip < $arr[$j] and $j == $i+1
# Look up the 2-character country code (returns '??' if unknown)
$ccode = string.subString( $arr, $i*6+4, $i*6+5 );

http.sendResponse( 200, "text/plain", 
   "IP = ".$ipaddr."\nCountry = ".$ccode."\n",
   "" );

You can call this script by requesting the URL "/geo" with a querystring containing a parameter named 'ip' which contains an IP address. For example, type the following into your browser location bar:

• http://zxtm.server:port/geo?ip=131.111.131.1

Another example

Suppose that access to part of your website is restricted by export controls. You can limit access and redirect users within the restricted geographies using TrafficScript as follows:

# Don't worry about pages that don't start "/restricted"
if( !string.startsWith( http.getPath(), "/restricted" ) ) break;

# calculate origin country (from code above):
$ipaddr = request.getRemoteIP();
.....
$ccode = string.subString( $arr, $i*6+4, $i*6+5 );

# Restrict access to Cuba, Iran, N.Korea, Sudan, Syria, 
# proxies and other unknown locations
$banned = "CU,IR,KP,SD,SY,A1,A2,??,";

if( string.contains( $banned, $ccode."," ) ) {
   http.redirect( "/accessdenied.html" );
}

Obviously, this method is only as accurate as the source database and it's just advisory; it does not satisfy any legal obligations you may have under export regulations.

More...

This technique is useful if you want to handle site visitors from various locations in different ways. You can use the http.setResponseCookie() function in TrafficScript to set a client-side 'location' cookie that the destination webserver can read.

If you want to perform global request distribution, do take a look at Zeus' ZXTM Global Load Balancer product for a capable and complete solution.

Owen Garrett [Zeus Dev Team] 18 September 2007 8 comments