Support Support Downloads Knowledge Base Service Request Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[EX/QFX] Maximum number of ECMP groups supported by EX and QFX switches

0

0

Article ID: KB34303 KB Last Updated: 06 Jun 2019Version: 1.0
Summary:

This article discusses the table size for ECMP (Equal-Cost MultiPath) and how its related features affect the number of ECMP objects on EX and QFX product line.

Symptoms:

Except for EX4300, whose ECMP table size is 4K, the ECMP table size of EX and QFX product is 16K. An ECMP table is shared by all protocols (e.g. IPv4, IPv6, MPLS, VXLAN, etc). An ECMP object is created on ECMP tables for each ECMP set of active routes. Depending on the JUNOS version and protocols, Static or Dynamic ECMP entries are allocated for ECMP objects. The maximum number of ECMP objects is affected by how ECMP entries are allocated. Other than that, MBB (Make Before Break) and fragmentation also affect the maximum number of ECMP objects.

Solution:

There are several features and factors that affect the maximum number of ECMP object.

  1. Static ECMP Entries

    Before 14.1X53-D30 and 15.1R3, ECMP entries are allocated statically based on the value defined by command set chassis maximum-ecmp, whose default value is 16. If the default value is used, an ECMP object always allocates 16 entries in ECMP tables regardless the number of active routes in the ECMP set.

  2. Dynamic ECMP Entries

    Obviously, static ECMP entries is not an efficient way to create ECMP objects. In 14.1X53-D30 and 15.1R3, a feature called Dynamic ECMP Entries for IPv4, IPv6, and MPLS is added. Dynamic ECMP entries only allocates the number of entries needed in an ECMP object. For example, if there are 5 active routes in an ECMP set, only 5 entries are allocated for the ECMP object in the ECMP table. The feature significantly improves the maximum number of ECMP objects.

    Dynamic ECMP entries for VXLAN is added in the following software versions:

    14.1X53-D48, 17.1R3, 17.2R3, 17.4R2-S4, 17.4R3, 18.1R3-S1, 18.1R4, 18.2R2, 18.3R2, and 18.4R1.
  3. MBB (Make Before Break)

    Assume that a customer uses 16 routes for all ECMP objects. Theoretically, 1000 ECMP objects can be used. However, it is not wise to implement 1000 ECMP objects because of MBB. MBB is a feature always enable in JUNOS.

    Assume that there is a change in the routing table (e.g. routing protocol flap). The number of active routes in an ECMP object is reduced from 16 to 14. MBB creates a new ECMP object with 14 entries. Then, the old ECMP object is then deleted. If 1000 ECMP objects are created before the change in the routing table, MBB is unable to create an ECMP object, which result in a forwarding problem on the switch.

  4. Fragmentation

    An ECMP object requires all entries allocated in a contiguous block. After a large number of ECMP object changes, the ECMP table may be fragmented; so that, the largest contiguous block in the ECMP table may be smaller than the number of available ECMP entries. Fragmentation may cause a failure to create ECMP objects.

Below is the instruction on checking the number of ECMP entries in an ECMP table

  1. Enter start shell in CLI

    root@switch:RE> start shell
    root@switch:RE:0%
  2. Create a command.txt in /var/tmp

    root@switch:RE:0% echo 'set dcb bc "l3 multipath show"' > /var/tmp/command.txt
    root@switch:RE:0%
  3. Save the ECMP table on a text file

    root@switch:RE:0% cprod -A fpc0 /var/tmp/command.txt > /var/tmp/output.txt
    root@switch:RE:0%
  4. Count the number of ECMP entries

    root@switch:RE:0% more /var/tmp/output.txt | grep Interfaces | egrep -o '[0-9]{1,}' | wc -l
           6
    root@switch:RE:0% 
    

In this example, there are 6 ECMP entries in the ECMP table.

In summary, the ECMP table size is 16K for most product. It is not recommended to use up all 16K entries in ECMP table because of MBB and fragmentation. Depending on the frequency of changes in ECMP objects and the number of active routes in ECMP object, the utilization of ECMP table can reach 90%.

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Security Alerts and Vulnerabilities

Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search