I’ve been trying to setup a BFD neighbor for a link connecting two important sites on a Nexus 7010. That link is only using iBGP for routing. This seems like a really easy thing to, unless you run into bad documentation with few key missing facts.
I was reading the Nexus 7000 Cisco Configuration Guide for Enabling BFD for BGP at http://www.cisco.com/en/US/docs/switches/datacenter/sw/6_x/nx-os/interfaces/configuration/guide/if_bfd.html. The document specifies that all you have to do to enable BFD for BGP is :
1. enable the bfd feature,
2. enable bfd on an interface
3. enable bfd under the BGP neighbor.
See below for the configuration as specified by Cisco.
feature bfd interface Ethernet1/10 bfd interval 100 min_rx 100 multiplier 5 router bgp 65100 neighbor 172.16.2.1 remote-as 65100 bfd
The problem with this feature is that BFD won’t see each other as neighbors. You won’t see any debug messages or keepalives or any other bfd packets. When I was troubleshooting it, I noticed that by specifying a source and destination IP address for BFD neighbors (under the interface) brought up the adj. The problem with that was that BGP didn’t recognize that IP address and during testing, BFD didn’t bring down the BGP neighbor.
Doing more research and with the help of TAC, we found a little bullet point in the Nexus 3000 documentation.
- BFD for BGP supports single-hop eBGP peers and single-hop iBGP peers with update-source (source http://www.cisco.com/en/US/docs/switches/datacenter/nexus3000/sw/unicast/503_u2_2/l3_bfd_for_bgp.pdf).
Basically, for the BFD feature to work on Nexus 7000, you have to specify the update-source for the iBGP session. Very simple and easy, but if not documented then it’s a little bit difficult to know. I would never think of going to the Nexus 3000 documentation for a Nexus 7000 configuration. See below for full configuration.
feature bfd interface Ethernet1/10 bfd interval 100 min_rx 100 multiplier 5 router bgp 65100 !<..> neighbor 172.16.2.1 remote-as 65100 bfd update-source eth1/10
Quick verification commands:
show bfd neighbors OurAddr NeighAddr LD/RD RH/RS Holdown(mult) State Int Vrf 172.16.2.1 172.16.2.2 1090519042/1090519042 Up 9193(5) Up Eth1/10 default
show ip bgp summary | i BFD BFD live-detection is configured and enabled, state is Up
Now that everything is nicely setup, how do you test it? You can’t just disconnect or shutdown the interface, bgp fast-external-fallover will trigger the BGP session to go down upon interface status change. One way to test is by filtering only BFD and BGP messages. For that I created a new ACL TEST-BDF-BGP that block ports 3784 & 3785 (BFD and BFD-Control) and TCP 179 for BGP. As long as the BGP session will go down before the default BGP hold timer, BFD will trigger it and our test succeeds.
conf t ip access-list TEST-BFD-BGP deny tcp any any eq 3784 deny tcp any any eq 3785 deny udp any any eq 3784 deny udp any any eq 3785 deny tcp any any eq 179 permit ip any any int eth1/10 ip access-group TEST-BFD-BGP in
Hope that clarifies things for someone else that is setting up iBGP with BFD on a Nexus 7000.