This article explains why QFX5100 cannot take a path to the longest route of the management port if it is configured as a default route, and what can be done to work around the issue.
When you configure a static route for the management port, it works as expected. However, if you configure a default route with the next-hop as network port, the static route for ME cannot be used.
Example:
Using the following topology and configuration, execute a ping from 1.1.1.1 to 2.2.2.1.
Topology:
EX2200(ge-0/0/0. IP: 1.1.1.1)---(ge-0/0/0 IP: 1.1.1.2)QFX5100(em0.0 IP: 2.2.2.2)---(ge-0/0/0 IP: 2.2.2.1)EX3200
Configure the following without a default route on QFX5100:
set interfaces ge-0/0/0 unit 0 family inet address 1.1.1.1/24
set routing-options static route 2.2.2.0/24 next-hop 1.1.1.2
set interfaces ge-0/0/0 unit 0 family inet address 1.1.1.2/24
set interfaces em0 unit 0 family inet address 2.2.2.2/24
set interfaces ge-0/0/0 unit 0 family inet address 2.2.2.1/24
set routing-options static route 1.1.1.0/24 next-hop 2.2.2.2
Results of the ping from EX2200 to EX3200:
root@EX2200# run ping 2.2.2.1 rapid count 10
PING 2.2.2.1 (2.2.2.1): 56 data bytes
!!!!!!!!!!
Add a default route on QFX5100.
set routing-options static route 0.0.0.0/0 next-hop 1.1.1.1
Results of the ping from EX2200 to EX3200:
root@EX2200# run ping 2.2.2.1
PING 2.2.2.1 (2.2.2.1): 56 data bytes
36 bytes from 1.1.1.1: Time to live exceeded
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 ee47 0 0000 01 01 c55d 1.1.1.1 2.2.2.1
36 bytes from 1.1.1.2: Redirect Host(New addr: 1.1.1.1)
Vr HL TOS Len ID Flg off TTL Pro cks Src Dst
4 5 00 0054 ee47 0 0000 40 01 865d 1.1.1.1 2.2.2.1
As you can see, the ping failed and direct route is not usable.
This is expected behavior. The management port connects to the Routing Engine (RE) directly, so the route of em0 will not be injected into the Packet Forwarding Engine (PFE).
There is no problem on the route/forwarding table for the 2.2.2.0 route, but the next-hop is ge-0/0/0 at the PFE level.
****************************
root@QFX5100> show route
inet.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 00:10:27
> to 1.1.1.1 via ge-0/0/0.0
1.1.1.0/24 *[Direct/0] 00:21:57
> via ge-0/0/0.0
1.1.1.2/32 *[Local/0] 00:21:57
Local via ge-0/0/0.0
2.2.2.0/24 *[Direct/0] 00:20:38
> via em0.0
2.2.2.2/32 *[Local/0] 00:20:38
Local via em0.0
[master:0]
root@QFX5100> show route forwarding-table
Routing table: default.inet
Internet:
Enabled protocols: Bridging,
Destination Type RtRef Next hop Type Index NhRef Netif
default user 0 3c:61:4:68:cd:83 ucst 1742 3 ge-0/0/0.0
default perm 0 rjct 51 1
0.0.0.0/32 perm 0 dscd 49 1
1.1.1.0/24 intf 0 rslv 1746 1 ge-0/0/0.0
1.1.1.0/32 dest 0 1.1.1.0 recv 1744 1 ge-0/0/0.0
1.1.1.1/32 dest 0 3c:61:4:68:cd:83 ucst 1742 3 ge-0/0/0.0
1.1.1.2/32 intf 0 1.1.1.2 locl 1745 2
1.1.1.2/32 dest 0 1.1.1.2 locl 1745 2
1.1.1.255/32 dest 0 1.1.1.255 bcst 1743 1 ge-0/0/0.0
2.2.2.0/24 intf 0 rslv 335 1 em0.0
2.2.2.0/32 dest 0 2.2.2.0 recv 333 1 em0.0
2.2.2.1/32 dest 0 40:b4:f0:7c:71:43 ucst 336 1 em0.0
2.2.2.2/32 intf 0 2.2.2.2 locl 334 2
2.2.2.2/32 dest 0 2.2.2.2 locl 334 2
2.2.2.255/32 dest 0 2.2.2.255 bcst 332 1 em0.0
224.0.0.0/4 perm 0 mdsc 50 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 46 1
255.255.255.255/32 perm 0 bcst 47 1
TFXPC0( vty)# show route ip prefix 2.2.2.0
IPv4 Route Table 0, default.0, 0x80000:
Destination NH IP Addr Type NH ID Interface
--------------------------------- --------------- -------- ----- ---------
default 1.1.1.1 Unicast 1742 RT-ifl 0 ge-0/0/0.0 ifl 559
****************************
When packets are received on the network port, then the QFX5100 device will check the next-hop at the PFE level. If there is no matching next-hop information on the PFE, the packet will be forwarded to the RE, which will check the route table to forward it further. Due to this, once configured as a default route, the packet destination to 2.2.2.0 will match the default route and be forwarded.
The key point is letting the packets received on the network port with destination 2.2.2.0 be processed at the PFE level. We can add a static route with the "receive" option to achieve this.
Add configuration on QFX5100:
set routing-options static route 2.2.2.0/23 receive
Because there is a 2.2.2.0/24 direct route on the chassis, we have to add a shorter route for it.
Results:
root@EX2200# run ping 2.2.2.1 rapid count 10
PING 2.2.2.1 (2.2.2.1): 56 data bytes
!!!!!!!!!!
--- 2.2.2.1 ping statistics ---
10 packets transmitted, 10 packets received, 0% packet loss
round-trip min/avg/max/stddev = 8.134/11.449/14.100/1.441 ms
root@QFX5100> show route
inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 00:50:55
> to 1.1.1.1 via ge-0/0/0.0
1.1.1.0/24 *[Direct/0] 01:02:25
> via ge-0/0/0.0
1.1.1.2/32 *[Local/0] 01:02:25
Local via ge-0/0/0.0
2.2.2.0/23 *[Static/5] 00:21:07
Receive
2.2.2.0/24 *[Direct/0] 01:01:06
> via em0.0
2.2.2.2/32 *[Local/0] 01:01:06
Local via em0.0
root@QFX5100> show route forwarding-table
Routing table: default.inet
Internet:
Enabled protocols: Bridging,
Destination Type RtRef Next hop Type Index NhRef Netif
default user 0 3c:61:4:68:cd:83 ucst 1742 3 ge-0/0/0.0
default perm 0 rjct 51 1
0.0.0.0/32 perm 0 dscd 49 1
1.1.1.0/24 intf 0 rslv 1746 1 ge-0/0/0.0
1.1.1.0/32 dest 0 1.1.1.0 recv 1744 1 ge-0/0/0.0
1.1.1.1/32 dest 0 3c:61:4:68:cd:83 ucst 1742 3 ge-0/0/0.0
1.1.1.2/32 intf 0 1.1.1.2 locl 1745 2
1.1.1.2/32 dest 0 1.1.1.2 locl 1745 2
1.1.1.255/32 dest 0 1.1.1.255 bcst 1743 1 ge-0/0/0.0
2.2.2.0/23 user 0 recv 48 1
2.2.2.0/24 intf 0 rslv 335 1 em0.0
2.2.2.0/32 dest 0 2.2.2.0 recv 333 1 em0.0
2.2.2.1/32 dest 0 40:b4:f0:7c:71:43 ucst 336 1 em0.0
2.2.2.2/32 intf 0 2.2.2.2 locl 334 2
2.2.2.2/32 dest 0 2.2.2.2 locl 334 2
2.2.2.255/32 dest 0 2.2.2.255 bcst 332 1 em0.0
224.0.0.0/4 perm 0 mdsc 50 1
224.0.0.1/32 perm 0 224.0.0.1 mcst 46 1
255.255.255.255/32 perm 0 bcst 47 1
TFXPC0( vty)# show route ip prefix 2.2.2.0
IPv4 Route Table 0, default.0, 0x80000:
Destination NH IP Addr Type NH ID Interface
--------------------------------- --------------- -------- ----- ---------
2.2.2/23 Recv 48 RT-ifl 0
TFXPC0( vty)# show route ip prefix 2.2.2.1
IPv4 Route Table 0, default.0, 0x80000:
Destination NH IP Addr Type NH ID Interface
--------------------------------- --------------- -------- ----- ---------
2.2.2/23 Recv 48 RT-ifl 0
Note that all packets matching this route will be forwarded to the RE, and you can monitor these packets. But there is a risk that it will cause high CPU.
root@QFX5100> monitor traffic interface em0
verbose output suppressed, use <detail> or <extensive> for full protocol decode
Address resolution is ON. Use <no-resolve> to avoid any reverse lookup delay.
Address resolution timeout is 4s.
Listening on em0, capture size 96 bytes
Reverse lookup for 2.2.2.1 failed (check DNS reachability).
Other reverse lookup failures will not be reported.
Use <no-resolve> to avoid reverse lookups on IP addresses.
07:59:20.776355 Out IP truncated-ip - 24 bytes missing! 1.1.1.1 > 2.2.2.1: ICMP echo request, id 49489, seq 0, length 64
07:59:20.777402 In IP truncated-ip - 24 bytes missing! 2.2.2.1 > 1.1.1.1: ICMP echo reply, id 49489, seq 0, length 64
2021-02-24: Article reviewed for accuracy, no changes required; article accurate and valid