One of the things I like about OpenBSD is that it has so many ways to communicate with other systems. Today I explore building an MPLS network using OpenBSD. When I was trying to make this setup work for myself, I couldn't find any good documents on how to set things up. So after piecing things together from different scraps that were 10 years old, I figured I write this article up to help someone else.
MPLS or (Multi Protocol Label Switching) is a protocol used by the largest providers to segment their WANs and provide "managed" services to customers. MPLS works by encapsulating packets into an MPLS frame and labeling that frame. (Usually multiple labels are on the frame as we'll see.) That frame is moved from host to host with the label being either pushed, popped or swapped on the frame. I will not go into all the intricacies of MPLS here, but it's import to understand that the outer label will change every hop the packet takes.
First some terminology
P router - Provider router that is in the center of the provider network. Only P and PE routers attach to the P routers
PE router - Provider Edge router attaches to both P routers and CE routers. Encapsulation/Decapsulation of the customer frames happens on the PE routers. Multiple customers are generally connected to PE routers
CE router - Customer Edge router attaches to PE routers. This is the customers WAN router. It has no knowledge of the underlying MPLS network.
In order to make MPLS work there are a layering of multiple routing protocols. Each layer and each protocol has a purpose. I'll explain each layer as we get to it.
Here is the diagram we'll be recreating. Since this is being built in a hypervisor, the management interfaces on the routers are only there to help connect from the hypervisor to the Guest OS. They do not play any roll in the MPLS network and I've left them off of the diagram.
The first layer is the underlay. This is where the P and PE routers operate.
The P router will connect both the PE routers. On the VIO1 and VIO2 interfaces, we add the additional tag of mpls to indicate to the OS that this interface will participate in MPLS.
Next we make sure that the P router will forward packets.
Next we will need to enable a routing protocol. This is critical to allow the loopback interfaces of each P / PE router to communicate with each other. This communication is used by ldpd(8) to exchange labels. While static routes will work (which you should never do), we'll use a routing protocol. Fortunately, OpenBSD provides several to choose from: ripd(8), ospfd(8), eigrpd(8). And because eigrpd(8) doesn't get much love, we'll use that here. The setup is very simple. We first tell eigrpd it's router-id, to update the fib with any routes it learns and then we tell it what interfaces to listen on.
Lastly we'll configure ldpd(8). This routing daemon will exchange label information with the PE routers (and P's if we had any more) and update the lib or label information base which is equivalent to the fib for IP protocols. Configuring ldpd(8) is very similar to eigrpd(8). We set the router-id, tell it to update the fib and tell it what interfaces to communicate on. The only extra step here is that we tell it what neighbors it will communicate with.
The PE routers are going to be very similar to the P router when configuration the underlay.
Similar configuration to PE1.
Both PE1 and PE2
Now enable the config and daemons on PE1 and PE2
First thing is to ping over the locally connected interfaces.
Next is to ping from loopback to loopback.
Now let's check our underlay routing protocols.
Pay attention to the routes marked with the *D EX flags. These are routes learned from other eigrp neighbors.
Now let's check out ldpd. Notice that the labels have been automatically assigned and that the IP of the peers shows up in the list.
PE's and CE's
Now that the provider network is setup we'll turn our attention to the PE and CE communication. On the PE the CE connections and routing will take place in their own routing domain/routing table. For this example we'll use rdomain 3. Similar to the start we'll first configure the interfaces.
Here is a new interface. This interface will provide a forwarding path from the rdomain it's in to the MPLS layer in rdomain 0.
The astute among you will notice that the IP address on mpe3 is the same as on lo3. This is fine. While I cannot speak authoritatively, the address is just needed to 'enable' the interface to participate in IP router.
Finally we enable bgpd(8). We will need to enable 2 instances of bgpd(8) the first instance will run in rdomain 0 and will be responsible for leaning the routes in rdomain 3 and importing and tagging them. The second bgpd(8) will run in rdomain 3 and communicate with the CE.
PE2 configuration will be similar to PE1
PE1 and PE2 Enablement
Now we enable the interfaces and start the bgpd(8) servers
For bgpd(8) in rdomain 3 we will need to "create" a new rc script. Then we can set the arguments and start it.
The CE configuration is quite simple. We will just configure the interfaces and setup routing. To simulate host networks behind the CE an lo10 interface will be created.
CE1 and CE2 Enablement
Now lets make sure the configurations are working. To start with on PE1 and PE2 the bgpd servers in rdomain 0 should have formed peers.
Notice that the rib shows that 184.108.40.206/32 has an rd of 0:0 and a nexthop of 0.0.0.0 meaning it's local and 220.127.116.11/32 has a nexthop of 192.0.2.2 which eill take the packet over MPLS.
If we ping from CE1 to CE2 and do a tcpdump(8) on P router vio1 we can see the MPLS tags in the packet.
If there is a problem check the /var/log/daemon log file for hints as to what might be failing. If all else fails, start from the beginning. Check the underlay, then move to the overlay
If you stuck with this article to the end, thank you. Let me know if I've make any mistakes or if something can be improved.