LOCAL HOST REMOTE HOST +--> (ISP1) ---+ / \ (SOCKS client) -> (ts_client) ---> (ISP2) ------ > (ts_server) -> (SOCKS server) \ / +--> (ISP3) ---+
At its essence Tunnel Splitter is a ruby and EventMachine based client/server pair which allows you to split a single TCP connection into multiple parts and send them over multiple TCP connections. The client does the splitting, the server does the reconstruction. At first glance this doesn’t seem to be anything special, but it actually allows you to do something very cool: use the aggregate the bandwidth of multiple ISPs for a single TCP connection.
There are already plenty of load balancing routers out there. Although they do load balance connections between multiple ISPs, any single connection will only be going out over a single ISP. Not to their discredit, it’s the best you can do with only a router. Tunnel splitter requires a little more. It requires that you have access to a server sitting on a connection twice as fast (up+down) as the sum of all your ISP connections. This is easily available nowadays. I currently run my tunnel splitter server on an $8/month VPS.
If it’s not already obvious, tunnel splitter can save you a lot of money if you need a fast connection. Getting a single 50Mbps connection to your house will cost a lot more than getting 6 15Mbps connections (or whatever speed your ISP provides). The increase in price as you need more and more bandwidth from a single ISP connection is almost never linear.
Tunnel splitter also provides transparent redundancy if ISPs go down. It uses a custom protocol on top of TCP to make sure all packets get delivered. If 10 packets have already been sent out through ISP1 before it is determined that it went down, they will be resent out through another link, so your tunneled TCP connection will not drop.
By itself tunnel splitter only tunnels one local port to one remote host:port. This isn’t that useful by itself. But when you put a SOCKS server at that remote host:port now you can tunnel any application over it which supports SOCKS proxying. Add a little routing magic (which I will discuss later) and you can proxy _anything_, even if it has no SOCKS support.
Let’s assume you are connected to two ISPs, one on eth0 and one on eth1. And let’s say your server on a fast connection is running with the address tsplitterserver.mayhem.org. And let’s say you have a SOCKS server running on the server listening on localhost port 1080. On the server you would simply run:
ruby ts_server.rb 0.0.0.0:6000 localhost:1080
This tells it to listen for tunnel splitter client connections on port 6000 and forward the tunneled connections to the SOCKS server on port 1080.
Client side we would run this:
ruby ts_client.rb localhost:1080 tsplitterserver.mayhem.org:6000:eth0~10 tsplitterserver.mayhem.org:6000:eth1~10 # note that the localhost:1080 on the client and the server are unrelated and do not actually need to match
This tells the tunnel splitter client to create 10 tunnels to the server over the eth0 interface and 10 tunnels to the server over the eth1 interface, and then wait for incoming connections on port 1080. When the tunnel splitter client receives connections on port 1080 it will split them up and load balance them across all 20 connections. The reason we create 10 tunnels over each ISP here is to make sure we are maxing out each ISPs bandwidth. Some ISPs impose per connection caps so you can’t get your full bandwidth without using multiple connections. If you’re sure that your ISP doesn’t do this then you are fine with just a single connection and can leave off the
Now if you point any SOCKS capable app to localhost:1080 it will be using the aggregate bandwidth of both your ISPs, we’re done! Not quite. If you’re using Linux there is another issue we need to address first.
If you’re using OSX or another BSD (I’m assuming they’re the same, but I’ve only tested on OSX) then you can skip this next section.
The way tunnel splitter makes connections over a specific ISP is to bind the source address to the address of that ISP. The problem is that Linux by default does not route packets based on source address. So with our current setup all the packets will go out over the same ISP and the other one will be totally unused. The simplest way to fix this is to install the
shorewall firewall. You don’t need to use it as a firewall but it provides the least involved solution (that I know of) to get Linux to route based on source addresses. The instructions can be found in
There _is_ a way to set up the needed routing without shorewall but I haven’t looked into it too much yet and it seems to be a little more complicated.
It works out of the box (at least on OSX).
Applications which don’t support SOCKS proxying
For these applications you can use transocks_em. It is a server which receives all connections from specific apps (via some routing magic) and then forwards them to a SOCKS proxy of your choice. I’ll make a post about it soon, so stay tuned.
Tunnel splitter can also be useful in single ISP situations. As I mentioned above some ISPs impose per TCP connection bandwidth limits. If you are just downloading a file you can often overcome these with download acceleration software which makes multiple HTTP requests. This doesn’t work however for things like streaming video (youtube, hulu, etc). Tunnel splitter solves this problem because it allows you to have a single TCP connection use your ISPs maximum total bandwidth.
The fact that connections will never be dropped by tunnel splitter can be useful as well. You can sleep or hibernate your computer, or all your internet connections can go down for any amount of time, and when you restore things your remote connection will still be active. This works because of a few things. First, whatever program you are proxying is actually connected to localhost (to the tunnel splitter client), so your ISPs going down doesn’t affect that conneciton. Second, the remote connection from the tunnel splitter server to the final destination is of course not affected by your computer’s local status. Thirdly, it is only the tunnel splitter client and server which will detect that the tunnel connections have gone down. But as a side effect of its redundancy protocol, it will keep retrying forever to connect the tunnel connections. Once they do reconnect, any pending data on either side will be sent across as if nothing ever happened. As far as the client you are proxying and the remote server you are proxying to are concerned they have both had active TCP connections the entire time. Of course if you are proxying a protocol which does it own protocol level connection health checking or pinging (IRC) it will detect that there are no responses and most likely drop the connection. I have experimented a little with adding support for both the tunnel splitter client and server to be protocol aware and respond to these pings so that the even a protocol with connection health checks could stay up indefinitely. Not sure if this is something people will really care much about though.
Tunnel splitter is still very beta although I have been using it locally for all my web traffic and movie streaming for close to a year now and it’s been quite a while since I’ve run into any bugs.
All comments, criticisms, and suggestions welcome. Also feel free to contact me at firstname.lastname@example.org
Check out the README for more info.