Support Support Downloads Knowledge Base Apex Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[QFX] EVPN VXLAN configuration knobs and caveats

1

2

Article ID: KB32854 KB Last Updated: 10 May 2021Version: 2.0
Summary:

There are many different configurations knobs that can be used; depending on design.  This article serves as a quick reference for EVPN/VXLAN configuration knobs and caveats.

Solution:

‘Preferred’ knob for IRB IP

To support ping on VGA IP 'virtual-gateway-accept-data' (VGA) needs to be configured. If VGA IP is lower than IRB IP, ARPing would fail. IRB IP address is needed to process ARP requests and the default lower IP is preferred. So, if VGA has a lower IP than IRB, configure the “preferred” knob for IRB IP.
 
   set interface irb unit <unit id> family inet address <address> preferred
  set interface irb unit <unit id> family inet6 address <address> preferred

For more details, refer to the documentation on virtual-gateway-address.


chained-composite-next-hop

CNH (chained-composite-next-hop) is a must for EVPN pure type 5 with VXLAN encapsulation. Without this, PFE would not program the tunnel NH.

You have to explicitly set it on QFX5110.  

set routing-options forwarding-table chained-composite-next-hop ingress evpn

 With QFX10K, it is applied as part of the default configuration.

jnpr@QFX> show configuration routing-options forwarding-table | display inheritance defaults

See PR1303246 - require routing-options forwarding-table chained-composite-next-hop for EVPN type-5 routes


Router-id, overlay BGP local-address and IP address of vtep-source-interface loopback interface must be the same.

Loopback interface IPV4 address which is being used as vtep-source-interface, Router-ID & overlay BGP local-address must be same.

If Router-ID address is different than loopback ipv4 address which is being as local-address also in BGP will cause EVPN route type-3 and type-2 to use router-id as protocol-next-hop and would cause forwarding issue.

The IPV4 address configured for vtep-source-interface in an EVPN instance needs to match bgp local-address of the iBGP group involved in EVPN family signaling. If the two IPV4 addresses do not match, VXLAN tunnels to PEs participating in the EVPN instance will not be setup properly and lead to forwarding loss.

See PR1036561 - EVPN VXLAN: vxlan tunnels not created to Provider Edge Routers if source-vtep-interface ipv4 address doesn't match bgp local-address of ibgp session signaling EVPN family.


Multipath (routing options)

If there are multiple EVPN Type 5 gateway in remote AS, then incoming traffic might get blackholed. In this case, to ensure proper forwarding, there are two options:
  1. Make sure every device advertises at least one unique prefix. That way the prefix will always be the best path and remote endpoint will be installed in the forwarding table.
  2. If PEs are advertising some common prefix, then multipath (routing-option) is required to make sure that each remote endpoint is installed in the forwarding table. 

As a best practice it is safe to use multipath (Routing Options) 

See PR1305068 - At least one EVPN type 5 route from a remote endpoint must be installed in the local routing table to correctly forward inbound traffic received from that endpoint


virtual-gateway-address (VGA) on non-VXLAN IRB

In EVPN-VLXAN deployment, virtual-gateway-address (VGA) is used on L3 gateway to enable the default gateway function. This has to be used on VXLAN IRBs. If VGA is configured on non-VXLAN (standard VLAN) IRB, then gateway functionality can be broken on the entire device (including previous working VLANs & VXLANs) and forwarding would be affected.  

PR1360646 - On QFX10k, virtual-gateway-address should be only configured on a irb interface associated with a vxlan VLAN.


virtual-gateway-v4-mac

This knob is used to have customized VMAC. When this knob is not configured with VGA, then VRRP based MAC is used.

There is one more scenario where this knob is useful. When frame leaves the spine or ping is done to VIP, reply packet has the source MAC as IRB MAC. Generally, there is no issue with this, but to make sure switch replies with VMAC, use virtual-gateway-v4-mac.  After configuring this knob when ping is done to IRB, then relay packet would also start using configure VMAC as SMAC rather IRB MAC.

For more details, refer to the documentation on virtual-gateway-v4-mac


Proxy ARP/NDP and ARP/NDP suppression

This feature is the default enabled from version 17.3R1 and higher on QFX10K. This feature reduces the flooding of ARP and NDP messages in the EVPN network, resulting in a more efficient use of core bandwidth. However, if there is any issue due to this feature, it can be disabled using 'no-arp-suppression'

For more details, refer to the documentation on EVPN Proxy ARP and ARP Suppression, and NDP and NDP Suppression


Routing traffic between a VXLAN and a VLAN on QFX5110

(QFX5110 switches) By default, routing traffic between a VXLAN and a Layer 3 logical interface—for example, an interface configured with the set interfaces interface-name unit logical-unit-number family inet address ip-address/prefix-length command—is disabled. If this routing functionality is required in your EVPN-VXLAN network, you can perform additional configuration to make it work. For more information, see Understanding How to Configure VXLANs and Layer 3 Logical Interfaces to Interoperate.

(QFX5100, QFX5110, QFX5120, QFX5200, QFX5210, EX4300-48MP, and EX4600 switches) A physical interface cannot be a member of a VLAN and a VXLAN. That is an interface that performs VXLAN encapsulation and de-encapsulation cannot also be a member of a VLAN. For example, if a VLAN that is mapped to a VXLAN is a member of trunk port xe-0/0/0, any other VLAN that is a member of xe-0/0/0 must also be assigned to a VXLAN.

Note: Starting in Junos releases 18.1R3 and 18.4R1 for QFX5100, QFX5110, QFX5200, and QFX5210 switches and release 19.4R1 for QFX5120 switches, this limitation no longer applies because you can configure flexible Ethernet services on the physical interface. For more information, see Understanding Flexible Ethernet Services Support With EVPN-VXLAN

 


no-gateway-community

This needs to be configured if virtual-gateway-address is configured. When anycast IP is used as the gateway, then MAC needs to synch between spines. MAC is synched by 'default gateway extended community' and this is enabled by default. However, when virtual-gateway-address is used, a VRRP based MAC “00:00:5e:00:01:01” is used on both the spines, so MAC synch is not needed. There is no need for 'default gateway extended community'. To stop the CLI knob, 'no-gateway-community' is used.  If you do not use this knob, Leafs can program IRB MAC in PFE and can cause forwarding issues.

If anycast MAC (Statically Defined IRB Interface MAC Address on both spines) is used, then 'no-gateway-community' must be used.

set protocols evpn default-gateway no-gateway-community

If you do not want to advertise IRB MAC, then use 'do-not-advertise'

set protocols evpn default-gateway do-not-advertise

For more details, see the documentation on L3 Gateway & VMTO Comparison for EVPN with MPLS & VXLAN.


proxy-macip-advertisement

In case of Non-Collapsed, this configuration is required on Spine (L3GW) to advertise MAC+IP route.

For more details, see the documentation on proxy-macip-advertisement.

Modification History:
2021-05-10: Updated 'Routing traffic between a VXLAN and a VLAN on QFX5110' section
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search