This article describes how to dump various tables on nfp-vrouter SmartNIC-based Contrail compute nodes. These tables can be helpful while troubleshooting issues with a customer's nfp-vrouter environment.
A SmartNIC-based vRouter has an offloaded structure so an offloaded flow, l2, MPLS, mirror, NH table, and route table are maintained by the nfp_vrouter kernel module. The following specifies the list of files that contain a formatted dump of tables that are read directly from the NFP as well as the method to display the corresponding vRouter kernel table.
|
NFP table via debugfs
|
Corresponding kernel vRouter table
|
|
/sys/kernel/debug/nfp_vrouter/bond
|
cat /proc/net/bonding/bond0
|
|
/sys/kernel/debug/nfp_vrouter/flowtable
|
flow -l
|
|
/sys/kernel/debug/nfp_vrouter/l2
|
rt --family bridge --dump [vrf]
|
|
/sys/kernel/debug/nfp_vrouter/mirror
|
mirror --dump
|
|
/sys/kernel/debug/nfp_vrouter/mpls
|
mpls --dump
|
|
/sys/kernel/debug/nfp_vrouter/nexthop
|
nh --list
|
|
/sys/kernel/debug/nfp_vrouter/vif
|
vif --list
|
|
/sys/kernel/debug/nfp_vrouter/vxlan
|
vxlan --dump
|
|
/sys/kernel/debug/nfp_vrouter/routes/nfplpm_state
|
rt --dump [vrf]
|
Some examples of the above table
root@compute:~# docker exec -it agilio_vrouter bash
Note the offloaded flow entries as shown here:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/flowtable
INDEX HASH KEY(SIP|DIP|FAMILY|PROTO|SPORT|DPORT|NH) ACTION
1704648 0x00D9E703 (11.95.56.172|2.95.56.172|2|6|22|32794|23)NAT
1960824 0x00C3AA21 (11.95.56.172|11.37.90.172|2|1|18834|0|23)FWD
2486184 0x001E24A7 (7.101.168.192|4.0.254.169|2|6|32794|22|5)NAT
3514216 0x00F67577 (11.37.90.172|11.95.56.172|2|1|18834|0|23)FWD
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/bond
Mode=0
Xmit_policy=0
active_slaves=0
0 0 0 0 0 0 0 0
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/l2
Index Hash KEY(VRF|MAC) LABEL NH_index
9288 0x0076D313 (2|00:15:4d:12:21:fd) 0 3
16144 0x00306B45 (0|00:00:5e:00:01:00) 0 3
27328 0x00DCDB03 (2|02:34:37:89:a9:c6) 29 34
53928 0x00EC0405 (1|00:00:5e:00:01:00) 0 3
55216 0x0091BB03 (0|9a:6f:50:0c:fa:6f) 16777215 1
57104 0x00736DED (0|00:25:90:c4:98:b2) 16777215 1
57248 0x006A7839 (2|02:00:00:00:00:02) 16777215 12
65344 0x00ADE413 (1|ff:ff:ff:ff:ff:ff) 2 25
65384 0x0068AFB5 (2|02:28:80:65:3e:ca) 34 20
68248 0x000E8AB9 (2|02:14:b9:2b:2f:bf) 29 20
70272 0x001262D1 (1|00:15:4d:12:21:fd) 19 8
93136 0x0072A3BB (1|00:15:4d:12:21:f3) 19 20
95208 0x00853A0B (2|02:00:00:00:00:01) 16777215 12
99944 0x00AECCFB (0|00:15:4d:12:21:f3) 16777215 1
114000 0x00B1A2FF (0|00:25:90:c4:98:aa) 16777215 1
117128 0x00C955D1 (2|ff:ff:ff:ff:ff:ff) 6 36
124000 0x0088B5C7 (2|00:00:5e:00:01:00) 0 3
130872 0x00CE0D91 (0|00:15:4d:12:21:fd) 0 3
(Note that we do not see any entries in the example because no mirroring is configured.)
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/mirror
Index RID NH_ID Flags VNI
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/mpls
Label NH_index
16 5
17 6
18 7
19 8
20 9
21 21
22 25
23 26
24 36
25 28
26 29
27 30
28 31
29 34
4609 25
4621 36
Observe the offloaded NH entries in the following example:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/nexthop
Index TYPE FAMILY FLAGS VRF OIF ECMP(nh|label)
0 DISCARD 2 0x00001 0 0
1 DISCARD 2 0x200001 0 0
3 L2 RCV 2 0x200001 0 0
4 ENCAP 2 0x200001 0 0
5 ENCAP 2 0x200003 0 1
6 ENCAP 2 0x200001 0 1
7 ENCAP 2 0x200021 0 1
8 ENCAP 7 0x200001 0 1
9 ENCAP 7 0x200003 0 1
10 RCV 2 0x200101 1 1
11 RCV 2 0x200003 1 1
12 ENCAP 2 0x200001 -1 2
13 ENCAP 2 0x200003 -1 2
14 RESOLVE 2 0x200001 0 0
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/vif
VIF VRFID TYPE FLAGS NH MTU VLAN VF PORT/QID BOND
0 0 P 0x2002C0 4 1514 65535 0 0 -1
1 0 H 0x200140 5 0 65535 0 0 0
2 65535 A 0x200040 0 0 65535 0 0 0
3 2 V 0x2001C1 28 9000 65535 58 0 0
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/vxlan
VNID NH_index
2 21
6 26
Observe the offloaded entries per vrf for both v4 and v6 families here:
(agilio_vrouter):/# cat /host/sys/kernel/debug/nfp_vrouter/routes/nfplpm_state
<clipped>
vrfid_len: 8
v4_s0_nbits: 16
V4 FW active buf: 1
V4 last trie generate: Successful
V4 last trie commit: Successful
<clipped>
v4 routes count: 21
v4 routes max pfxlen (ip + vrfid): 40
1,0.0.0.0/0 --> [NextHop 1, label 0]
0,0.0.0.0/0 --> [NextHop 15, label 0]
0,224.0.0.0/8 --> [NextHop 10, label 0]
0,192.168.40.0/24 --> [NextHop 14, label 0]
2,59.0.0.0/24 --> [NextHop 1, label 0]
1,192.168.40.51/32 --> [NextHop 6, label 0]
0,192.168.40.254/32 --> [NextHop 15, label 0]
0,192.168.40.51/32 --> [NextHop 10, label 0]
0,192.168.40.48/32 --> [NextHop 16, label 0]
0,192.168.40.49/32 --> [NextHop 17, label 0]
1,192.168.40.52/32 --> [NextHop 20, label 17]
0,192.168.40.52/32 --> [NextHop 19, label 0]
1,169.254.169.254/32 --> [NextHop 11, label 0]
0,192.168.40.9/32 --> [NextHop 18, label 0]
2,59.0.0.254/32 --> [NextHop 13, label 0]
2,8.8.8.8/32 --> [NextHop 13, label 0]
2,169.254.169.254/32 --> [NextHop 11, label 0]
2,59.0.0.1/32 --> [NextHop 28, label 0]
0,169.254.0.3/32 --> [NextHop 28, label 0]
2,59.0.0.2/32 --> [NextHop 20, label 25]
2,59.0.0.3/32 --> [NextHop 20, label 30]
V6 FW active buf: 1
V6 last trie generate: Successful
V6 last trie commit: Successful
<clipped>
v6 active per-vrfid route sets: 3
route set for VRFID 2, routes count 3:
2,ff02:0000:0000:0000:0000:0000:0000:0002/128 --> [NextHop 12, label 0]
2,ff02:0000:0000:0000:0000:0000:0000:0001/128 --> [NextHop 12, label 0]
2,fe80:0000:0000:0000:0000:0000:5e00:0100/128 --> [NextHop 12, label 0]
route set for VRFID 1, routes count 3:
1,ff02:0000:0000:0000:0000:0000:0000:0002/128 --> [NextHop 12, label 0]
1,ff02:0000:0000:0000:0000:0000:0000:0001/128 --> [NextHop 12, label 0]
1,fe80:0000:0000:0000:0000:0000:5e00:0100/128 --> [NextHop 12, label 0]
route set for VRFID 0, routes count 1:
0,ff00:0000:0000:0000:0000:0000:0000:0000/8 --> [NextHop 10, label 0]
(agilio_vrouter):/#