[EX/QFX] Per-packet load-balancing in routing-instance on EX/QFX switches

By default, ECMP (Equal Cost Multiple Path) is used to load balance traffic on EX/QFX switches, when there are multiple equal-cost paths available to the same destination. EX/QFX switches support per-packet (flow) load balancing in the global routing-instance (inet.0), as well as in the user-defined routing-instance (Virtual-Router).

However, when a per-packet load balancing policy is applied to a global routing-instance (inet.0), it is effective for all routing instances (global and user-defined routing instances). EX/QFX switches do not support the configuration of separate forwarding policies for user-defined routing instances.

Note: Per-packet load balancing is actually per-flow load balancing. EX/QFX switch hardware does not support per-packet balancing.

Examples for configuring and verifying per-packet load balancing

Topology

Configuration on EXSwitch1

set interfaces ge-0/0/2 unit 0 family ethernet-switching vlan members testLB
set interfaces vlan unit 2 family inet address 192.168.2.18/24
set interfaces ge-0/0/10 unit 0 family inet address 192.168.10.18/24
set interfaces ge-0/0/11 unit 0 family inet address 192.168.11.18/24

set routing-instances Router1 instance-type virtual-router
set routing-instances Router1 interface ge-0/0/10.0
set routing-instances Router1 interface ge-0/0/11.0
set routing-instances Router1 interface vlan.2
set routing-instances Router1 protocols ospf area 0.0.0.0 interface ge-0/0/10.0
set routing-instances Router1 protocols ospf area 0.0.0.0 interface ge-0/0/11.0
set routing-instances Router1 protocols ospf area 0.0.0.0 interface vlan.2

The relative interfaces of other switches are configured and OSPF is used to exchange routes. A loopback interface is configured with IP 4.4.4.1/24 on EXSwitch4.

EXSwitch4

set interfaces lo0 unit 0 family inet address 4.4.4.1/24

Verification

Global routing table before applying policy:

root@EXSwitch1> show route

Router1.inet.0: 11 destinations, 11 routes (11 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

4.4.4.0/24         *[OSPF/10] 00:26:10, metric 2
                    > to 192.168.10.19 via ge-0/0/10.0
                      to 192.168.11.17 via ge-0/0/11.0

We can see the global routing table having both the available next-hops.

Forwarding table (before applying policy):

root@EXSwitch1> show route forwarding-table family inet destination 4.4.4.1

Routing table: Router1.inet
Internet:
Destination      Type  RtRef Next hop             Type   Index   NhRef  Netif
4.4.4.1/32       user  0     192.168.10.19        ucst   1305    6      ge-0/0/10.0

Forwarding table has only one next-hop, which is installed based on ECMP.

If flow below 10 is received on EXSwitch1 with the following source/destination IP addresses and port before applying the per-packet (flow) load-balancing forwarding policy:

Source IP/Port             Destination IP/Port
192.168.2.151:4979   4.4.4.1:80
192.168.2.151:4979   4.4.4.1:81
192.168.2.151:4979   4.4.4.1:82
192.168.2.151:4979   4.4.4.1:83
192.168.2.151:4979   4.4.4.1:84
192.168.2.151:4979   4.4.4.1:85
192.168.2.151:4979   4.4.4.1:86
192.168.2.151:4979   4.4.4.1:87
192.168.2.151:4979   4.4.4.1:88
192.168.2.151:4979   4.4.4.1:89

If we check the pps rate on both the available interfaces:

root@EXSwitch1> show interfaces ge-0/0/10 | match rate   
  Input rate     : 5120 bps (10 pps)
  Output rate    : 5120 bps (10 pps)

root@EXSwitch1> show interfaces ge-0/0/11 | match rate   
  Input rate     : 0 bps (0 pps)
  Output rate    : 0 bps (0 pps)

We can see that all the flows are going out only from ge-0/0/10.

Now let's apply the per-packet (flow) load-balancing forwarding policy for per-packet (flow) load balancing.

set policy-options policy-statement Policy-1 then load-balance per-packet
set routing-options forwarding-table export Policy-1

As we can see, the policy is applied globally but it has impact on the Router1 instance as well.

Global routing table (after applying policy)

root@EXSwitch1> show route
4.4.4.0/24         *[OSPF/10] 05:03:40, metric 2
                    > to 192.168.10.19 via ge-0/0/10.0
                      to 192.168.11.17 via ge-0/0/11.0

Forwarding table (after applying policy)

root@EXSwitch1> show route forwarding-table family inet destination 4.4.4.1
Routing table: Router1.inet
Internet:
Destination        Type RtRef Next hop           Type Index NhRef Netif
4.4.4.1/32         user     0                    ulst 131070     3
                              192.168.10.19      ucst  1305     4 ge-0/0/10.0
                              192.168.11.17      ucst  1306     4 ge-0/0/11.0

We can see after applying the per-packet (flow) policy that the load-balancing forwarding policy has both the available routes installed in the forwarding table as well.

If we send the same traffic flow and check the traffic rate on both next-hop interfaces:

root@EXSwitch1> show interfaces ge-0/0/10 | match rate   
  Input rate     : 5120 bps (10 pps)
  Output rate    : 2560 bps (5 pps)

root@EXSwitch1> show interfaces ge-0/0/11 | match rate   
  Input rate     : 0 bps (0 pps)
  Output rate    : 2560 bps (5 pps)

Here you can see that 50% flows are going through one next-hop and another 50% flows are going through another next-hop.

Knowledge Base