Second Line, Second ISP?

If you host your Web sites in-house, it's a certain bet that at some stage your leased line will start to fill up and need to be upgraded. All lines will 'max out' at times but, when Web site performance starts to suffer and visitors or clients start to get annoyed, some remedial action is usually required. There are a number of options available. You could increase the capacity of your existing line; in most cases this will be a simple procedure, just as if your ISP and telco have 'turned up the volume'. This is the fastest and also the cheapest option.

However, in addition to adding extra bandwidth, you could use this upgrade as a way to improve your resilience against failures. You can do this by taking your extra bandwidth via a second leased line, which might come from your existing ISP or be bought in from elsewhere. Whichever option you take there are some important rules to follow:

• Your existing line is almost certainly delivered to your building by BT, Cable & Wireless or one of the smaller cable companies. Ensure that a different telco than the one which supplies your existing line delivers the new one. One thing to beware of here is that C&W can have fairly long lead times for its new line installations.
• Of equal importance is making sure that the new line enters your building in a different place to the existing one. That way, if a truck demolishes a corner of the building, or someone digs up the pavement outside, you'll hopefully only lose service on one of the lines. A true story: a friend of Paul's was having a second line installed but neglected his advice on this matter. As the contractors were outside digging up the pavement to install the second line, you've guessed it, they cut right through the first one! He had complete loss of service for over a week.
• If you buy the new line from your existing ISP, make sure it supplies one from a different remote POP. If your existing line terminates in Telehouse in London's Docklands, for instance, the new one might be supplied from the ISP's Network Operations Centre (assuming this isn't based at Telehouse) or perhaps via a local POP. These rules will protect you against failure of the carrier, physical risks, failure at your office, failure of a remote POP and, if you take the line from a different provider, failure of the ISP itself.

Given the choice of a new line from your existing ISP or one from an alternative supplier, is there an obvious choice? At first sight you'd appear to have more protection by shopping elsewhere. However, when Paul was faced with this choice recently, he ended up taking the second line from his existing provider INS (www.insnet.net). Why? Well, when one of your lines fails, you're going to have to make everything run through the other line. Each line will have an associated address block (probably a class C), so you could manually reconfigure all of your machines to use an address in the appropriate block and to use the other router as the default gateway. However, if you have a large number of machines or virtual servers, this could take some time and has the makings of an administrative nightmare. It also requires someone on site at the time to make the changes, and it will all need to be reversed when the other line comes back on stream. This isn't exactly ideal.

However, your ISP probably has links to many other providers, and they seem to manage to reroute things if a route fails. How do they do it? Well, they'll have something called an autonomous system number, or ASN, and be running something like BGP (Border Gateway Protocol). We'll spare you the boring details, but suffice to say that these allow organisations to join in with the big boys of the Internet by automatically routing traffic via different routes as required. The trouble is, the big boys like to keep their club fairly exclusive - you almost certainly won't have enough of an address requirement to get a block of addresses big enough not to be treated as 'an annoyance' by many of the bigger ISPs, especially in the US. When this happens, they'll simply ignore your routing instructions.

If you take your new line from your existing ISP, however, it can take care of line failures for you. Your on-site routers (assuming they are Cisco, and why would anyone use anything else?) can be configured to run Hot Standby Router Protocol, or HSRP. This allows one router to monitor the function of the other and, if it detects a problem, to take over all of its functions. Some Cisco routers can be configured to run Multiple-HSRP. Using this removes the concept of primary and backup routers, and each will act as backup for the other, so you're totally protected.

Internally, your ISP will be running another horrible acronym called EIGRP (Enhanced Internet Gateway Routing Protocol), which allows it to assign different weightings to your two lines for both address blocks. Normally, the traffic will take the 'cheapest' route, but it will try the alternative if that isn't available. This exactly describes the setup INS has provided for Paul's company CST Group. Obviously this still doesn't protect against total ISP failure, but one of the reasons CST chose INS in the first place is that its network has been designed from day one to ensure that it won't suddenly go off air and, unlike many of its competitors, it never has.

Testing the new configuration was great fun. Switch off a router and everything is working again within a few seconds. Pull the plug on a leased line and, again, the little LEDs on the other router just start to flash more quickly. It's almost like magic, but not as magical as the reassuring feeling you get knowing that you won't get called out of bed at 3am because your monitoring service has lost sight of your servers.