Wednesday, October 9, 2013

Using BIRD to route over OpenVPN tunnels.

OpenVPN tunnels, good. BIRD routing daemon, great. OSPF on OpenVPN tunnels, headache. The combination of OpenVPN and BIRD routing daemon for OSPF is nothing new for me. I've been using it pretty happily for over a year. However, it always seemed as though something was a bit precarious. As I have started to scale the implementation, I've realized that I needed a rethink of my strategy.

The original design used a /31 subnet on each tunnel, with each endpoint using an address. Logically, it made perfect sense. The catch is that OSPF does not automatically advertise the IP address of the tunnels (a quirk of OpenVPN, BIRD, or both). Traffic would flow through the routers, but the remote tunnel address would be unreachable. I solved this by adding a stubnet x.x.x.x/31 directive to the OSPF  configuration at one of the endpoints. At the time this worked fine.

As I brought a third site online, with multiple VPN tunnels between each site, the original design quickly broke down. The /31s were no longer consistently routable. I remedied this by changing the /31 stubnet to a /32, with each router advertising its own IP addresses. Life was briefly good, until I realized that restarting the OpenVPN tunnels would fail. OpenVPN would log the error, "router-a openvpn_1199[18976]: FreeBSD ifconfig failed: external program exited with error status: 1" The following message was also logged, "ifconfig: ioctl (SIOCAIFADDR): File exists" From the depths of my memory, I recalled that this error occurs when you attempt to configure an interface with an address that already exists in the routing table. Since BIRD was already advertising the /32 host IP, trying to address the tunnel would fail.

Some creative restarting of OpenVPN and BIRD resolved the problem, but is an unacceptable solution for production. I turned to the BIRD mailing list with my situation, and very quickly received a response from one of the BIRD developers. He suggested a number of ideas, the most promising of which is to dedicate a subnet to each router, as a pool from which to draw addresses for local tunnel endpoints on that specific host. The subnet is then configured as a stubnet by BIRD. The ends of a given tunnel may not be remotely close to each other, but this is fine because a tunnel is a point-to-point link. Tunnels can be restarted at will, without contention from existing routes in the table.

The diagram below shows an example topology. Notice that the endpoint of each tunnel is in a completely different network. For example, on the tunnel between router-a and router-b, router-a uses an IP of, and router-b uses It doesn't matter, it works just fine. With the given stubnet configured in the bird.conf for each router, all tunnel endpoints are reachable. The network and broadcast addresses for each router's stubnet are also useable, as shown in the example.