Skip to content

VXLAN Config – Adding BGP EVPN

In the last blog post I setup a basic VXLAN config just allowing 2 hosts on separate leaf switches to communicate within their common subnet. In that configuration all of the hosts were learned by the VTEPs from BUM traffic flooded using multicast. Using BGP EVPN, hosts are learned by their own leaf switch and then advertised to the BGP speaking spine switches which reflect them back down to all other leafs. As soon as a leaf switch learns of a host (most likely from that host sending a GARP or some ARP request) it will advertise that host’s information to everyone else. Even when using BGP EVPN arp requests are still sent to all VTEPs using multicast even though the VTEPs have probably already learned about the remote host via BGP. That is where arp suppression comes into play. Arp suppression allows the VTEP to respond to arp requests if the VTEP already knows the remote hosts mac address. There is a bit of upfront work to get this setup though because some switches require you carve up the TCAM to make room for ether-arp.

LEAF-1(config)# hardware access-list tcam reg arp-ether 256 double-wide
ERROR: Aggregate TCAM region configuration exceeded the available Ingress TCAM slices. Please re-configure.
!
LEAF-1# sh hardware access-list tcam region
..... Find one that you can live without. In my case, I set RARP to 512 (wouldn't work at 1024 either, so i droped it to 512)
LEAF-2(config)# hardware access-list tcam region racl 512
Warning: Please save config and reload the system for the configuration to take effect
.... 
.... Reload
.... 
LEAF-1# hardware access-list tcam reg arp-ether 256 double-wide
.... 
.... Reload
.... 

Diagram I’ll be working with:

SPINE Configuration

# SPINES-1 and 2
feature vn-segment-vlan-based
feature nv overlay
feature bgp
nv overlay evpn

Next we will configure the EVPN address family for BGP. I’ll be using iBGP and since we don’t have a full mesh, i’m going to configure the spines as route reflectors.

router bgp 100
  neighbor 192.168.255.3
    remote-as 100
    address-family l2vpn evpn
      send-community extended
      route-reflector-client
  neighbor 192.168.255.4
    remote-as 100
    address-family l2vpn evpn
      send-community extended
      route-reflector-client

That’s it for the spines. Now onto the leafs

LEAF Configuration

We already have most of the features enabled, so I’ll just be adding in the ones needed for BGP EVPN and enabling SVI routing

# LEAF-1 and 2
feature interface-vlan
feature bgp
nv overlay evpn

Now we need a new vlan/VNI and VRF to use for VXLAN Routing. It also requires an SVI for the vlan without an IP address

vlan 1000
 vn-segment 1000
!
## DO NOT forget the "evpn" portion or else nothing will work
vrf context HQ
 vni 1000
 rd auto
 add ipv4 unicast
  route-target both auto evpn
!
int vlan 1000
 vrf member HQ
 no shut
 ip forward

I’ll now add the VNI to the NVE and tell the NVE to advertise hosts via BGP.

int nve1
 host-reachability protocol bgp
 member vni 1000 associate-vrf
 member vni 100
  mcast-group 239.1.1.1
  suppress-arp

BGP can now be configured to peer with the route reflectors for address-family l2vpn evpn

router bgp 100
 neighbor 192.168.255.1 remote-as 100
  update-source loopback0
  address-family l2vpn evpn
   send-community extended
 neighbor 192.168.255.2 remote-as 100
  update-source loopback0
  address-family l2vpn evpn
   send-community extended

The VNIs currently in use (not the one associated with the VRF) can now be advertised into BGP. As you build more you simply add them to this evpn config

evpn
 vni 100 l2
 rd auto
 route-target both auto

Last thing i’m going to do is enable the anycast gateway feature. All of the VTEPs will share the same IP and mac address for all L3 VXLAN Vlans. You should probably make sure you don’t configure a mac address that’s already in use somewhere, so finding some reserved or globally unused OUI space is recommended.

# Configure the anycast mac address which is a global config
fabric forwarding anycast-gateway-mac 0007.fa00.0001
!
int vlan 100
 vrf member HQ
 ip add 192.168.10.1/30
 fabric forwarding mode anycast-gateway
 no shut

Let’s add another VNI, move HOST-2 into the vlan and verify routing works

vlan 200
 vn-segment 200
int vlan 200
 vrf member HQ
 ip add 192.168.20.1/24
 fabric forwarding mode anycast-gateway
 no shut
int nve1
 member vni 200
  mcast-group 239.1.1.1
  suppress-arp
evpn
 vni 200 l2
 rd auto
 route-target both auto
LEAF-1# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 192.168.255.3, local AS number 100
BGP table version is 39, L2VPN EVPN config peers 2, capable peers 2
1 network entries and 1 paths using 216 bytes of memory
BGP attribute entries [1/156], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.255.1   4   100      61      64       39    0    0 00:50:08 0
192.168.255.2   4   100      61      66       39    0    0 00:50:06 0
!
!
LEAF-2# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 192.168.255.4, local AS number 100
BGP table version is 47, L2VPN EVPN config peers 2, capable peers 2
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.255.1   4   100      64      63       47    0    0 00:49:36 0
192.168.255.2   4   100      64      61       47    0    0 00:49:41 0

You can already see that no advertisements have been received by either side.

I generated traffic on the hosts by pinging each other

LEAF-1# sh bgp l2vpn evpn sum
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 192.168.255.3, local AS number 100
BGP table version is 84, L2VPN EVPN config peers 2, capable peers 2
9 network entries and 11 paths using 1816 bytes of memory
BGP attribute entries [8/1248], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [2/8]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
192.168.255.1   4   100      51      62       84    0    0 00:27:45 2
192.168.255.2   4   100      49      60       84    0    0 00:27:50 2
!
!
LEAF-1# sh bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 84, local router ID is 192.168.255.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 192.168.254.3:32867    (L2VNI 100)
*>l[2]:[0]:[0]:[48]:[aabb.cc00.0300]:[0]:[0.0.0.0]/216
                      192.168.254.3                     100      32768 i
*>i[2]:[0]:[0]:[48]:[aabb.cc00.0400]:[0]:[0.0.0.0]/216
                      192.168.254.4                     100          0 i
*>l[2]:[0]:[0]:[48]:[aabb.cc00.0300]:[32]:[192.168.10.10]/272
                      192.168.254.3                     100      32768 i
*>i[2]:[0]:[0]:[48]:[aabb.cc00.0400]:[32]:[192.168.10.30]/272
                      192.168.254.4                     100          0 i

Route Distinguisher: 192.168.254.3:32967    (L2VNI 200)
*>l[2]:[0]:[0]:[48]:[aabb.cc00.0200]:[0]:[0.0.0.0]/216
                      192.168.254.3                     100      32768 i
*>l[2]:[0]:[0]:[48]:[aabb.cc00.0200]:[32]:[192.168.20.20]/272
                      192.168.254.3                     100      32768 i

Route Distinguisher: 192.168.254.4:32867
* i[2]:[0]:[0]:[48]:[aabb.cc00.0400]:[0]:[0.0.0.0]/216
                      192.168.254.4                     100          0 i
*>i                   192.168.254.4                     100          0 i
* i[2]:[0]:[0]:[48]:[aabb.cc00.0400]:[32]:[192.168.10.30]/272
                      192.168.254.4                     100          0 i
*>i                   192.168.254.4                     100          0 i

Route Distinguisher: 192.168.255.3:4    (L3VNI 1000)
*>i[2]:[0]:[0]:[48]:[aabb.cc00.0400]:[32]:[192.168.10.30]/272
                      192.168.254.4                     100          0 i

I sent a set of pings from HOST-3 to HOST-2 (different VTEPs and Subnets) and grabbed the packets off the wire to look. They are using the VXLAN routing VNI 1000 when being routed to another VTEP

Published inCisco

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *