Last week I sat down with Dan Backman and several members of the engineering team at Plexxi to put together the BGP design and configuration for one of our customers. It had been a while since I had my fingers in a BGP configuration, but it is indeed like riding a bike (which for a Dutchman is a foundational skill).
BGP has its origins in the late 1980s. The National Science Foundation sponsored NSFnet at the time consisted of early cisco routers combined with plain servers (we called them PCs back then) running "gated", the earliest available software stack implementing routing protocols. As an evolution of an earlier exterior routing protocol creatively named Exterior Gateway Protocol (EGP), BGP (RFC 1105), introduced a stateful TCP based connection between routers. It more officially created routing domains designated with autonomous system numbers, and probably its most prominent feature, the AS Path, a list of autonomous systems a route had been advertised through, allowing coarse grained loop detection, and the ability to actually see where a route initially originated. It was the first of a now large list of descriptive metrics associated with routes. A huge step forward in inter domain routing and developed for scale, BGP changed the inter domain routing similar to how OSPF changed it for intra domain routing.
In early 1991, BGP version 3 was introduced (RFC 1267) and added the ability to carry additional metrics associated with routes, called path attributes, not to be confused with the original AS Path, which became one of these attributes. BGP-3 was very widely used in the Internet at the time, and BGP got its last major overhaul in 1994 in RFC 1654 to what we now know as BGP-4. BGP-4 introduced classless addressing and amazingly the fundamentals of the protocol have not changed in the past 20 years. Many new attributes have been added, IPv6 and multicast address support has been added, but all within the foundation of the original BGP-4 specification.
What has become clear though is that BGP is not really a routing protocol, it is really a network prefix exchange mechanism, where each prefix is described with a set of metrics, some numeric, some not. The real power behind all of this is that BGP is light for the amount of prefixes it handles and as such is being used beyond connecting traditional autonomous systems.
BGP has found many uses inside the enterprise network, inside the enterprise data center. There is a very old best practice to keep IGPs as small as possible to keep link state databases small and therefore reconvergence after failure faster. Server virtualization has increased the amount of subnets used at the edge of the datacenter significantly, where a single server may well be part of 10s or more subnets. All of those subnets need to be communicated throughout the datacenter, and while the traditional way would have these all part of the IGP, using BGP to distribute all these subnets provides a far more scalable, controllable and most importantly stable way to move routes around.
Running two routing protocols on the same network is somewhat unusual at first, but there is a clear separation in their roles. The IGP (OSPF, ISIS, static, …) determines the connectivity between the routers. It calculates how to get from any router to any other router, consider it the mechanism to control the inside of the routed infrastructure. BGP is then used to communicate what subnets are attached to the outside of each of these routers. There are bound to be many subnets there and BGP is good at distributing large amounts of routes. The router will use information from both to figure out how to get to one of these subnets at the edge of the networks. It uses BGP to find out what router to go to, then uses OSPF/ISIS to figure out how to get to that router inside of the routed network. Its almost similar to gates at an airport. The TV screens (BGP) tells you what gate your flight leaves from, the signs inside the airport (OSPF/ISIS) tell you how to get to that gate. In general, BGP tells you what exit to use, OSPF/ISIS tells you how to get to that exit.
The beauty of this separation is that you can actually change the way you determine how to get to an exit. In previous posts I have discussed that a shortest path may not always be what you want. Or need. An external controller with a network wide view may calculate many different paths to get to an exit, in a way that OSPF or ISIS cannot, taking into consideration what applications are on these subnets and what they need from the network. And with the increasing amount of subnets attached to an exit, BGP has proven itself to be a very flexible, robust and convenient way to advertise these routes to whoever needs to know. Other routers in the same domain, upstream routers in the path to the outside world, or even the controller itself. Some of our SDN friends have found a very active use of BGP in communicating reachability information.
With 20 years under the belt and counting, BGP is one of those protocols that is so fundamentally sound in its architecture and design that it finds new uses every few years. And even in the world of SDN, BGP will have its role. What exactly that role is beyond the current use we will see, but I will happily bet that it will be many many years before it is retired. And as a BGP kind of guy, that makes be happy.
The post I am a BGP fanboy appeared first on Plexxi.