Support Support Downloads Knowledge Base Service Request Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[MX] Lookup chip (LU/XL) congestion due to forwarding loop caused by misconfiguration

0

0

Article ID: KB35314 KB Last Updated: 07 Dec 2019Version: 1.0
Summary:

If there is a mis-configuration which causes the packet to bounce from one instance to another in an endless loop, it may congest the lookup chip and impact the router performance.

This usually happens when a static route is configured with "next-table" knob so the packet goes from one instance to another and somehow the packet is directed back to the same instance due to another route with next-table knob or filter based forwarding pointing the packet back to the same instance.

This article provides an example of such loop and tries to determine the packet which is getting looped. Based on that, corrective action can be taken.  

Symptoms:

The following errors may appear sporadically or continuously based on the number of these packets that are coming to the router:

Nov 12 04:41:57  jtac-lab-re0 : %PFE-5: fpc2 LUCHIP(0) Congestion Detected, Active Zones f:f:f:f:f:f:f:f:f:f:f:f:f:f:f:f
Nov 12 04:42:06  jtac-lab-re0 : %PFE-5: fpc2 LUCHIP(0) Congestion Detected, Active Zones f:f:f:f:f:b:f:f:f:f:f:f:7:f:f:f
Nov 12 04:42:06  jtac-lab-re0 : %PFE-5: fpc2 LUCHIP(0) Congestion Detected, Active Zones f:f:f:f:f:b:f:f:f:f:f:f:7:f:f:f

Nov 12 05:22:45  jtac-lab-re0 : %PFE-3: fpc2 LUCHIP(0): Secondary PPE 11 zone 17 timeout.
Nov 12 05:22:45  jtac-lab-re0 : %PFE-3: fpc2 PPE Thread Timeout Trap:  Count 36327, PC 48,     0x0048:  KT4Way_ARS00_trap
Nov 12 05:43:43  jtac-lab-re0 : %PFE-3: fpc2 LUCHIP(0): Secondary PPE 15 zone 29 timeout.
Nov 12 05:43:43  jtac-lab-re0 : %PFE-3: fpc2 PPE Thread Timeout Trap:  Count 45686, PC 6c0,     0x06c0:  KTLeaf_or_Span
Solution:
  1. First, figure out the impacted traffic by checking if there is any ppe_trap file generated around the time congestion/ppe error appeared.

    file list /var/tmp/ detail | match ppe

    If there is any ppe trap file, get hex dump of the packet from this file:

    file show /var/tmp/ppe_trap_fpc5_LU_1_00.1 | find dispatch

          Dispatch size is 92
          0006c0308008544b
          8c93afc1544b8c93
          bfc1884700013332
          4500003c00004000
          32064ff44043f737
          4044c109780f97d0
          986a971be082ee67
          a0127120c71e0000
          020405b40402080a
          3eda99553a1af64d
          01030307

    The highlighted portion of the dispatch is the actual packet.

  2. Copy/paste the hex dump into any hex to pcap conversion tools available online to see the packet details.

    Packet:

      Ethernet II
     
        Destination: JuniperN_93:bf:c1 (54:4b:8c:93:af:c1)
        Source: JuniperN_93:df:c1 (54:4b:8c:93:bf:c1)
        Type: MPLS label switched packet (0x8847)
     
       MultiProtocol Label Switching Header
     
        0000 0000 0000 0001 0011 .... .... .... = MPLS Label: 19
        .... .... .... .... .... 001. .... .... = MPLS Experimental Bits: 1
        .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1
        .... .... .... .... .... .... 0011 0010 = MPLS TTL: 50
     
       Internet Protocol Version 4
     
        0100 .... = Version: 4
        .... 0101 = Header Length: 20 bytes (5)
        Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
        Total Length: 60
        Identification: 0x0000 (0)
        Flags: 0x4000, Don't fragment
        Time to live: 50
        Protocol: TCP (6)
        Header checksum: 0x4ff4
        Header checksum status: Unverified
        Source: 64.67.247.55
        Destination: 64.68.193.9

    Based on the MAC address of the interface in the above packet, it can be determined in which instance that packet was received and what is the destination address.

  3. Check the destination route in the relevant instance:

    show route 64.68.193.9 table instance-1.inet.0
        
    instance-1.inet.0: 579 destinations, 2021 routes (576 active, 0 holddown, 4 hidden)
    + = Active Route, - = Last Active, * = Both
        
    64.68.192.0/22     *[Static/5] 30w6d 15:26:25, tag 1421197
                        to table instance-2.inet.0

    There appears to be static route which points it to instance-2 to next-table knob

    set routing-instances instance-1 routing-options static route 64.68.192.0/22 next-table instance-2.inet.0

    On instance-2, there is a filter based forwarding (FBF) configuration which is sending the packet back to instance-1 thus causing the loop

    set firewall family inet filter JTAC-FBF term term-1 from source-prefix-list JTAC-FBF-PREFIX-LIST
    set firewall family inet filter JTAC-FBF term term-1 then routing-instance instance-1
    set firewall family inet filter JTAC-FBF term default then accept
    
    set routing-instances instance-2 forwarding-options family inet filter input JTAC-FBF
        
    set policy-options prefix-list JTAC-FBF-PREFIX-LIST 64.68.192.0/22

In order resolve the issue, we can rectify the configuration which causes the forwarding loop.

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Security Alerts and Vulnerabilities

Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search