Multicast upstream interface showing wrong interface after a routing change. The issue is seen when the group-ranges have /32 prefix instead of a subnet. for example: set routing-instances JTAC-INSTANCE protocols pim rp static address 10.0.103.9 group-ranges 239.100.10.4/32. Upstream traffic will not be switched to the new IIF on a path changeover and that will cause traffic to be impacted.
Here is an example of the issue:
Before triggering the issue, the uplink interface is via ae2.1004. There is also a backup path which is via irb.1099.
root@spine1> show multicast route instance JTAC-INSTANCE group 239.100.10.4 source-prefix 10.100.100.6 extensive
Instance: JTAC-INSTANCE Family: INET
Group: 239.100.10.4
Source: 10.100.100.6/32
Upstream interface: ae2.1004
Downstream interface list:
xe-0/0/31:1.4000
Number of outgoing interfaces: 1
Session description: Organisational Local Scope
Statistics: 574 kBps, 585 pps, 43353 packets
Next-hop ID: 2097151
Upstream protocol: PIM
Route state: Active
Forwarding state: Forwarding
Cache lifetime/timeout: 360 seconds
Wrong incoming interface notifications: 21
Uptime: 00:00:10
Trigger the routing change by disabling the uplink interface on the peer device
{master:0}
root@spine1> show multicast route instance JTAC-INSTANCE group 239.100.10.4 source-prefix 10.100.100.6 extensive
Instance: JTAC-INSTANCE Family: INET
Group: 239.100.10.4
Source: 10.100.100.6/32
Upstream interface: ae2.1004 <<<<<<<<<<<<< incorrect upstream interface
Downstream interface list:
xe-0/0/31:1.4000
Number of outgoing interfaces: 1
Session description: Organisational Local Scope
Statistics: 574 kBps, 585 pps, 43353 packets
Next-hop ID: 2097151
Upstream protocol: PIM
Route state: Active
Forwarding state: Forwarding
Cache lifetime/timeout: 360 seconds
Wrong incoming interface notifications: 21
Uptime: 00:05:52
{master:0}
root@spine1> show route 10.100.100.6
inet.0: 32 destinations, 34 routes (32 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 2w1d 02:39:27
> to 10.85.173.1 via em0.0
[OSPF/150] 1w6d 19:55:56, metric 0, tag 0
> to 172.31.0.229 via irb.996
JTAC-INSTANCE.inet.0: 36 destinations, 47 routes (36 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.100.100.0/24 *[OSPF/150] 5d 07:08:48, metric 0, tag 0
> to 10.5.2.165 via irb.1099
The correct output should be as below is below
{master:0}
root@spine1> show pim join instance JTAC-INSTANCE extensive 239.100.10.4 source 10.100.100.6
Instance: PIM.JTAC-INSTANCE Family: INET
R = Rendezvous Point Tree, S = Sparse, W = Wildcard
Group: 239.100.10.4
Source: 10.100.100.6
Flags: sparse,spt
Upstream interface: irb.1099 <<<<<<<<<<<<<Correct upstream interface
Upstream neighbor: 10.5.2.165
Upstream state: Join to Source, No Prune to RP
Keepalive timeout: 345
Uptime: 00:05:48
Downstream neighbors:
Interface: Pseudo-GMP
xe-0/0/31:1.4000
Number of downstream interfaces: 1
Number of downstream neighbors: 0
This is because the /32 prefix is interpreted differently in the routing pipeline and it discards the data traffic received on the new IIF at the PFE stage instead of trapping to the CPU. So, IIF_MISMATCH mechanism can not be detected here with this config, resulting is traffic loss until the PIM join is timed out.
{master:0}
root@spine1> show multicast statistics instance JTAC-INSTANCE interface irb.1099
Instance: JTAC-INSTANCE Family: INET
Interface: irb.1099
Routing protocol: PIM Mismatch error: 0
Mismatch: 0 Mismatch no route: 0
Kernel resolve: 0 Routing notify: 0
Resolve no route: 0 Resolve error: 0
Resolve filtered: 0 Notify filtered: 0
In kbytes: 0 In packets: 0
Out kbytes: 0 Out packets: 0
Instead of /32, the customer configuration can be slightly tweaked to "set routing-instances JTAC-INSTANCE protocols pim rp static address 10.0.103.9 group-ranges 239.100.10.0/24" /24 subnet or any other non /32 prefix which can cover the required group range(s). Here is an example:
1) Remove the 239.100.10.4/32 and add 239.100.10.0/24 from spine1 pim rp configuration
{master:0}
root@spine1> configure
Entering configuration mode
{master:0}[edit]
root@spine1# delete routing-instances JTAC-INSTANCE protocols pim rp static address 10.0.103.9 group-ranges 239.100.10.4/32
{master:0}[edit]
root@spine1# set routing-instances JTAC-INSTANCE protocols pim rp static address 10.0.103.9 group-ranges 239.100.10.0/24
2) Trigger the routing change by disabling the uplink interface on the peer device
3) Check the multicast route and pim join status for 239.100.10.4
{master:0}
root@spine1> show pim join instance JTAC-INSTANCE extensive 239.100.10.4 source 10.100.100.6
Instance: PIM.JTAC-INSTANCE Family: INET
R = Rendezvous Point Tree, S = Sparse, W = Wildcard
Group: 239.100.10.4
Source: 10.100.100.6
Flags: sparse,spt
Upstream interface: irb.1099 <<<<<<<<<<<<<Correct upstream interface
Upstream neighbor: 10.5.2.165
Upstream state: Join to Source, No Prune to RP
Keepalive timeout: 322
Uptime: 00:36:45
Downstream neighbors:
Interface: Pseudo-GMP
xe-0/0/31:1.4000
Number of downstream interfaces: 1
Number of downstream neighbors: 0
{master:0}
root@spine1> show multicast route instance JTAC-INSTANCE group 239.100.10.4 source-prefix 10.100.100.6 extensive
Instance: JTAC-INSTANCE Family: INET
Group: 239.100.10.4
Source: 10.100.100.6/32
Upstream interface: irb.1099 <<<<<<<<<<<<<Correct upstream interface
Downstream interface list:
xe-0/0/31:1.4000
Number of outgoing interfaces: 1
Session description: Organisational Local Scope
Statistics: 982 kBps, 1000 pps, 54729 packets
Next-hop ID: 2097151
Upstream protocol: PIM
Route state: Active
Forwarding state: Forwarding
Cache lifetime/timeout: 360 seconds
Wrong incoming interface notifications: 1
Uptime: 00:36:50
{master:0}
root@spine1> show route 10.100.100.6
inet.0: 32 destinations, 34 routes (32 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 2w1d 02:29:52
> to 10.85.173.1 via em0.0
[OSPF/150] 1w6d 19:46:21, metric 0, tag 0
> to 172.31.0.229 via irb.996
content.inet.0: 36 destinations, 47 routes (36 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.100.100.0/24 *[OSPF/150] 5d 06:59:13, metric 0, tag 0
> to 10.5.2.165 via irb.1099
In the output, we see the IIF_MISMATCH incremented by 1 (which is cases by the routing change) PIM relies on PFE to send an IIF_MISMATCH on the new RPF and on this NEW mismatch Multicast route will be switched to the new RPF.
{master:0}
eng@spine1> show multicast statistics instance JTAC-INSTANCE interface irb.1099
Instance: JTAC-INSTANCE Family: INET
Interface: irb.1099
Routing protocol: PIM Mismatch error: 0
Mismatch: 1 Mismatch no route: 0
Kernel resolve: 0 Routing notify: 0
Resolve no route: 0 Resolve error: 0
Resolve filtered: 0 Notify filtered: 0
In kbytes: 0 In packets: 0
Out kbytes: 0 Out packets: 0