Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[MX] Configuring and Troubleshooting PBB-EVPN

0

0

Article ID: KB36080 KB Last Updated: 09 Sep 2020Version: 2.0
Summary:

This article provides useful commands and operational guidelines to troubleshoot PBB-EVPN environments.

Symptoms:
Topology and relevant config:

Single homed topology where:

  • CE1 and CE2 are in same subnet.
  • CE3 and CE4 are in same subnet.
  • CE3/4 are logical-systems.
  • pe1 and pe2 provide the layer2 hand-off and are BGP neighbors that share EVPN routes
  • Single-hop LSP between pe1 and pe2
  • xe-0/3/0.329 on CE2 and xe-0/3/1.329 on pe2 is added to illustrate a point discussed later
  • Rest of the topology is self explanatory.
  CE1/CE3 xe-0/0/0 mac: 3c:8a:b0:88:3a:c8          CE2/CE4 xe-0/3/0 mac: ac:4b:c8:45:80:f6
             +-------------------+                +----------------------------------------+
             |                   |                |                                        |
             |  CE1 and LS:CE3   |                |           CE2 and LS:CE4               |
             |                   |                |                                        |
             |                   |                |                                        |
             |                   |                |                                        |
             +-------------------++               +----------------------------------------+
    LS:CE3 xe-0/0/0.300| xe-0/0/0.0     LS:CE4 xe-0/3/0.300| xe-0/3/0.0       |xe-0/3/0.329
           10.30.30.2  | 10.20.20.2/24         10.30.30.1  | 10.20.20.1/24    |10.40.40.1/24
                       |                                   |                  |
                       |                                   |                  |
                       |                                   |                  |
                       |                                   |                  |
                vlan300| vlan111                    vlan300| vlan560          | vlan331
                ELAN300| ELAN981                    ELAN300| ELAN981          |ELAN981
           xe-0/0/1.300| xe-0/0/1.334          xe-0/3/1.300| xe-0/3/1.334     |xe-0/3/1.329
+---------------------------+                       +----------------------------+
|                           |                       |                            |
|           pe1             |xe-1/2/0       xe-0/2/0|               pe2          |     
|                           +-----------------------+                            | 
|      jtac-mx80            |10.4.1.1/30   10.4.1.2 |        jtac-mx960          |
|                           |                       |                            |
+---------------------------+                       +----------------------------+
     lo0: 10.100.100.101                             lo0: 10.100.100.100
     pip0 Mac: 9c:cc:83:09:41:b0                      pip0 Mac: 44:f4:77:0d:0f:b8
PE2:
root@mx960# show routing-instances pbbaccess | no-more
instance-type virtual-switch;
interface pip0.0;
bridge-domains {
    ELAN981 {
        domain-type bridge;
        vlan-id 981;
        enable-mac-move-action;
        interface xe-0/3/1.329;
        interface xe-0/3/1.334; 
        bridge-options {
            mac-table-size {
                1024;
                packet-action drop;
            }
        }
    }
    ELAN300 {
        domain-type bridge;
        vlan-id 300;
        enable-mac-move-action;
        interface xe-0/3/1.300;
        bridge-options {
            mac-table-size {
                1024;
                packet-action drop;
            }
        }
    }
}
pbb-options {
    peer-instance pbbcore;
}
service-groups {
    sgaccess {
        service-type elan;
        pbb-service-options {
            isid 981 vlan-id-list 981;
            isid 300 vlan-id-list 300;
        }
    }
}

root@mx960# show routing-instances pbbcore | no-more     
instance-type virtual-switch;
interface cbp0.0;
route-distinguisher 10.100.100.100:100;
vrf-target target:65001:100;
protocols {
    evpn {
        control-word;
        pbb-evpn-core;
        extended-isid-list all;
    }
}
bridge-domains {
    corebridge1 {
        vlan-id 100;
        isid-list [ 981 300 ];
        vlan-id-scope-local;
    }
}

root@mx960# show protocols bgp    
group EVPN {
    type internal;
    local-address 10.100.100.100;
    family inet {
        unicast;
    }
    family evpn {
        signaling;
    }
    neighbor 10.100.100.101;
}   

root@mx960# show interfaces xe-0/3/1 | no-more
flexible-vlan-tagging;
mtu 9122;
hold-time up 10000 down 120;
encapsulation flexible-ethernet-services;
gigether-options {
    no-flow-control;
    no-auto-negotiation;
}
unit 33 {
    encapsulation vlan-bridge;
    vlan-id 2;
}
unit 37 {
    encapsulation vlan-bridge;
    vlan-id 59;
}
unit 288 {
    encapsulation vlan-bridge;
    vlan-id 398;
}
unit 300 {
    encapsulation vlan-bridge;
    vlan-id 300;
}
unit 329 {
    encapsulation vlan-bridge;
    vlan-id 331;
}
unit 334 {
    encapsulation vlan-bridge;
    vlan-id 560;
}
unit 400 {
    encapsulation vlan-bridge;
    vlan-id 400;
}

labroot@jtac-mx960> show configuration interfaces cbp0 
unit 0 {
    family bridge {
        interface-mode trunk;
        bridge-domain-type bvlan;
        isid-list all;
    }
}
labroot@jtac-mx960> show configuration interfaces pip0    
unit 0 {
    family bridge {
        interface-mode trunk;
        bridge-domain-type svlan;
        isid-list all-service-groups;
    }
}

PE1: config on pe1 is similar to pe2 hence omitted for brevity.

Solution:
  1. Verify BGP session is up and exchanging evpn address-family routes:

    root@mx960> show bgp summary
    Groups: 1 Peers: 1 Down peers: 0
    Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
    inet.0               
                           0          0          0          0          0          0
    bgp.evpn.0           
                          14         14          0          0          0          0
    Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
    10.100.100.101       65001      23114      23116       0       0 1w0d 6:33:39 Establ
      inet.0: 0/0/0/0
      bgp.evpn.0: 3/3/3/0
      pbbcore.evpn.0: 3/3/3/0
      __default_evpn__.evpn.0: 0/0/0/0
    Pip0 interface MAC should be advertised as B-MAC (Bridge-MAC) in type 2 BGP route:
    root@mx960> show interfaces pip0 | match hard
      Current address: 44:f4:77:0d:0f:b8, Hardware address: 44:f4:77:0d:0f:b8
  2. Verify :

    Type 2 route advertised and received that has the info about the label to be used for unicast traffic. This is per B-MAC.
    Type 3 route advertised and received that has the info about the label to be used for broadcast traffic. This is per ISID (Service Id).
    These labels are going to be used as VPN labels based on the traffic type.
    root@mx960> show route advertising-protocol bgp 10.100.100.101 table pbbcore.evpn.0 detail
    
    pbbcore.evpn.0: 20 destinations, 20 routes (20 active, 0 holddown, 0 hidden)
    * 2:10.100.100.100:100::0::44:f4:77:0d:0f:b8/304 MAC/IP (1 entry, 1 announced)
    BGP group EVPN type Internal
         Route Distinguisher: 10.100.100.100:100
         Route Label: 16
         ESI: 00:00:00:00:00:00:00:00:00:00
         Nexthop: Self
         Localpref: 100
         AS path: [65001] I
         Communities: target:65001:100
    
    * 3:10.100.100.100:100::300::10.100.100.100/248 IM (1 entry, 1 announced)
    BGP group EVPN type Internal
         Route Distinguisher: 10.100.100.100:100
         Route Label: 17
         PMSI: Flags 0x0: Label 17: Type INGRESS-REPLICATION 10.100.100.100
         Nexthop: Self
         Localpref: 100
         AS path: [65001] I
         Communities: target:65001:100
         PMSI: Flags 0x0: Label 17: Type INGRESS-REPLICATION 10.100.100.100
    
    * 3:10.100.100.100:100::981::10.100.100.100/248 IM (1 entry, 1 announced)
    BGP group EVPN type Internal
         Route Distinguisher: 10.100.100.100:100
         Route Label: 17
         PMSI: Flags 0x0: Label 17: Type INGRESS-REPLICATION 10.100.100.100
         Nexthop: Self
         Localpref: 100
         AS path: [65001] I
         Communities: target:65001:100
         PMSI: Flags 0x0: Label 17: Type INGRESS-REPLICATION 10.100.100.100
        
    root@mx960# run show route receive-protocol bgp 10.100.100.101 table bgp.evpn.0 detail
    
    bgp.evpn.0: 14 destinations, 14 routes (14 active, 0 holddown, 0 hidden)
    * 2:10.100.100.101:100::0::9c:cc:83:09:41:b0/304 MAC/IP (1 entry, 0 announced)
         Import Accepted
         Route Distinguisher: 10.100.100.101:100
         Route Label: 299776
         ESI: 00:00:00:00:00:00:00:00:00:00
         Nexthop: 10.100.100.101
         Localpref: 100
         AS path: I
         Communities: target:65001:100
    
    * 3:10.100.100.101:100::300::10.100.100.101/248 IM (1 entry, 0 announced)
         Import Accepted
         Route Distinguisher: 10.100.100.101:100
         Nexthop: 10.100.100.101
         Localpref: 100
         AS path: I
         Communities: target:65001:100
         PMSI: Flags 0x0: Label 299792: Type INGRESS-REPLICATION 10.100.100.101
    
    * 3:10.100.100.101:100::981::10.100.100.101/248 IM (1 entry, 0 announced)
         Import Accepted
         Route Distinguisher: 10.100.100.101:100
         Nexthop: 10.100.100.101          
         Localpref: 100
         AS path: I
         Communities: target:65001:100
         PMSI: Flags 0x0: Label 299792: Type INGRESS-REPLICATION 10.100.100.101
  3. Verify EVPN session and database. Database would be having only the remote and local B-MAC info as C-MAC (CE1/3 MAC)  is learnt in data plane:

    root@mx960> show evpn database
    Instance: pbbcore
    VLAN  DomainId  MAC address        Active source                  Timestamp        IP address
          0         44:f4:77:0d:0f:b8  Local                          Jun 26 15:40:57
          0         9c:cc:83:09:41:b0  10.100.100.101                Jun 29 12:58:59
             
    root@mx960> show evpn instance extensive    
    Instance: __default_evpn__
      Route Distinguisher: 10.100.100.100:0
      Number of bridge domains: 0
      Number of neighbors: 0
    
    Instance: pbbcore
      Route Distinguisher: 10.100.100.100:100
      Per-instance MAC route label: 16
      Per-instance multicast route label: 17
      PBB EVPN Core enabled
      Control word enabled
      Duplicate MAC detection threshold: 5
      Duplicate MAC detection window: 180
      MAC database status                     Local  Remote
        MAC advertisements:                       1       1
        MAC+IP advertisements:                    0       0
        Default gateway MAC advertisements:       0       0
      Number of local interfaces: 1 (1 up)
        Interface name  ESI                            Mode             Status     AC-Role
        cbp0.0          00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root
      Number of IRB interfaces: 0 (0 up)
      Number of protect interfaces: 0
      Number of bridge domains: 5
      Number of Bundle bridge domains: 1
        Bundle Bridge domain Num ISIDs    Mode             MAC sync
        corebridge1          5            Extended         Enabled
      Number of neighbors: 1
        Address               MAC    MAC+IP        AD        IM        ES Leaf-label
        10.100.100.101         1         0         0         5         0
      Number of ethernet segments: 0 

Other useful commands:

root@mx960> show l2-learning provider-instance

PBN Routing Instance: pbbaccess
Flags (P2P -ELINE service,         MP -ELAN service,
       M1  -Many svlans to 1 isid, O1 -One svlan to 1 isid)

PBN               S-VLAN   ISID       Source               PBBN             B-VLAN  Flags
Bridging                              B-MAC                Bridging                  
Domain                                                     Domain
ELAN981             981    981        44:f4:77:0d:0f:b8    corebridge1      100     M1,MP
ELAN300             300    300        44:f4:77:0d:0f:b8    corebridge1      100     M1,MP

root@mx960> show l2-learning backbone-instance

Backbone Routing Instance : pbbcore, PBBN-ID: 0
Backbone Bridging domain : corebridge1, VLAN-ID : 100

Flags (P2P -ELINE service,         MP -ELAN service)
       M1  -Many svlans to 1 isid, O1 -One svlan to 1 isid)

ISID      PBN               Provider          S-VLAN  Flags    Backbone
          Routing           Bridging                           Destination
          Instance          Domain                             MAC
981       pbbaccess         ELAN981          981     M1,MP    01:1e:83:00:03:d5 <-- 003d5---ISID 981 in hex
300       pbbaccess         ELAN300          300     M1,MP    01:1e:83:00:01:2c <-- 0012c---ISID 300 in hex   

Check the bridge-mac table:

Before any CE traffic
root@mx960> show bridge mac-table

MAC flags       (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
    O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)

Routing instance : pbbcore
 Bridging domain : corebridge1, VLAN : 100
   MAC                 MAC      Logical          NH     MAC         active
   address             flags    interface        Index  property    source      
   01:1e:83:00:01:2c   DC                        1048579            0.0.0.0     <-- 0012c---ISID 300 in hex               
   01:1e:83:00:03:d5   DC                        1048580            0.0.0.0     <-- 003d5---ISID 981 in hex                  
   9c:cc:83:09:41:b0   DC                        1048581            10.100.100.101         
Ping initiated to 10.20.20.2 (CE1) and 10.30.30.2(CE3) from 10.20.20.1 (CE2) and 10.30.30.1 (CE4) respectively . MAC table on pe2 shows dynamically learned C-MAC address of CE1 and CE3 along with their corresponding B-MAC mapping in their respective ISID mapped vlan:
root@mx960> show bridge mac-table                 

MAC flags (S -static MAC, D -dynamic MAC,
           SE -Statistics enabled, NM -Non configured MAC, P -Pinned MAC)

Routing instance : pbbaccess
 Bridging domain : ELAN981, ISID : 981, VLAN : 981
   MAC                 MAC      Logical                 Remote
   address             flags    interface               BEB address
   3c:8a:b0:88:3a:c8   D        rbeb.32768              9c:cc:83:09:41:b0  <-- remote C-MAC of CE1 learned and its corresponding B-MAC mapping
   ac:4b:c8:45:80:f6   D        xe-0/3/1.334         
   

MAC flags (S -static MAC, D -dynamic MAC,
           SE -Statistics enabled, NM -Non configured MAC, P -Pinned MAC)

Routing instance : pbbaccess
 Bridging domain : SIN2XE01.ELAN300, ISID : 300, VLAN : 300
   MAC                 MAC      Logical                 Remote
   address             flags    interface               BEB address
   3c:8a:b0:88:3a:c8   D        rbeb.32768              9c:cc:83:09:41:b0  <-- remote C-MAC of CE3 learned and its corresponding B-MAC mapping
   ac:4b:c8:45:80:f6   D        xe-0/3/1.300         

   
MAC flags       (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC
    O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC)

Routing instance : pbbcore
 Bridging domain : corebridge1, VLAN : 100
   MAC                 MAC      Logical          NH     MAC         active
   address             flags    interface        Index  property    source      
   01:1e:83:00:01:2c   DC                        1048579            0.0.0.0                
   01:1e:83:00:01:90   DC                        1048582            0.0.0.0                  
   01:1e:83:00:03:d5   DC                        1048580            0.0.0.0                   
   9c:cc:83:09:41:b0   DC   
 
Packet Capture Analysis:

A packet capture (ttrace) of the ping was taken on the egress FPC of pe2  to see the packet format of the PBB encapsulated packet and following format was found:

Ethernet header: for Physical Interface: Destination Interface & Source Interface MAC.
MPLS Header : Device is one hop away so only vpn label is present.
PW Ethernet Control Word: Sequence No: 4 Bytes in length, set to 0
Ethernet header: for Bridge:  Destination Bridge & Source Bridge MAC.
IEEE 802.1ah header: has ISID Value along with CE Destination & Source MAC.
802.1q: ISID mapped Vlan.

Original IP packet from CE.
First image is that of a ICMP request and the second image is of the ICMP reply:

ICMP  request:


ICMP reply: