Stingray Traffic Manager Forward Proxy and Tunneling SSL (06/21/2008)
In September 2007, Riverbed released Stingray Traffic Manager version 4.2. One of the new features in the 4.2 release was 'Forward Proxy Mode', which allowed Stingray Traffic Manager to be used as a forward proxy.
A little while after the release, Ben from Stingray's Development Team kindly provided a community article giving step by step instructions on how to configure Stingray Traffic Manager as a Web Cache Proxy Server. Ben's post concentrated on proxying HTTP requests, however proxy servers can sometimes be configured to proxy HTTPS requests too.
This article will provide a TrafficScript™ example that can be used to proxy those HTTPS requests
The CONNECT Method
So why doesn't the original TrafficScript also work with HTTPS? Well, when a browser attempts to connect to a resource over HTTP, it simply sends the request to the proxy and waits for a response; it's all HTTP.
However, when a browser wants to retrieve a secure page over HTTPS, the browser sends a CONNECT request instead. If it used HTTP, then the connection between the browser and the proxy would be in clear text, which is obviously bad, plus the proxy server would have to contain a full SSL stack to handle the connection. Not a problem for Stingray, but not all proxy servers have our SSL stack :-)
The internet gods decided that an all around simpler solution would be to just ask the proxy to open a TCP socket and then let the browser and the remote server handle all the SSL communication themselves. In a nutshell: the HTTPS proxy uses a special CONNECT method to locate the server, and then streams the SSL data directly between the endpoints. It's not HTTP and the HTTP virtual server used for other proxy requests won't work here.
So what if you need to proxy both HTTP and HTTPS traffic? You need two Virtual Servers.
One should be configured as described in Ben's article, and this will handle the HTTP traffic. The second Virtual Server should be configured using the "Generic Client First" protocol and this will handle HTTPS. Once you have the virtual server configured, you'll need to add a TrafficScript request rule.....
# Check for SSL connect requests. If it's not a CONNECT then break
In the code above we first check to see if this request is actually a proxy CONNECT request, if it isn't then we break out of the rule. If we are a CONNECT request then we need to extract the hostname and the port provided in the request. I have decided to only allow requests going to port 443, and send back a "403 Forbidden" response code if they're going anywhere else. The RFC for SSL Tunneling says this:
...the proxy cannot verify that the protocol being spoken is really SSL, and so the proxy configuration should explicitly limit allowed connections to well-known SSL ports (such as 443 for HTTPS, 563 for SNEWS, as assigned by the Internet Assigned Numbers Authority).
Of course Stingray Traffic Manager can allow clients to connect to any port if you want to allow it, but please bear those words in mind should you decide to remove the port restriction.
The next job for our TrafficScript is to resolve the hostname to an IP address (this part shamelessly stolen from Ben). If we can't resolve the host we return a "404 Unknown Host" error.
The last two lines are quite interesting, you'll note we are using pool.select() instead of pool.use(). This is because the Stingray Traffic Manager needs to respond directly to the client's CONNECT request. If we used pool.use() TrafficScript processing would stop and the CONNECT line would be sent to the remote HTTPS server, which is not what we want. Instead we select the pool for use with the data stream and send back a "200 Connection Established" response to the client. The SSL handshake will follow and we will stream all of the data to and from the remote server unhindered.
Using only one Virtual Server
You could do all this using only one virtual server of course. HTTP is a client first protocol, and will run just fine over the virtual server we're using for HTTPS.
However, you will have to re-write Ben's code using the low level request.get() and request.set() functions because the http classes will not be available. Once you've done that and merged it with my code, you'll discover that you don't have access to Content-Caching, Compression, or any other HTTP-specific features, which, I'm sure you'll agree, is less than ideal.
There is an alternative. Remember my code exits if the request does not use the "CONNECT" method, so the HTTPS proxy can be configured to use a Ben's HTTP proxy Virtual Server as it's default pool. Once the TrafficScript identifies that the current request is not for HTTPS we loop the traffic back into the HTTP proxy.
Using this configuration, you still have two virtual servers, but you only need to publish one proxy service entry point. Best of all, you can still utilise Stingray’s' HTTP optimization and caching abilities on the loop back virtual server.