Support Support Downloads Knowledge Base Service Request Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[MX] Aloha HQoS MPC fails to allocate loopback streams and IFDs go missing after committing 'ingress-and-egress' on PICs

0

0

Article ID: KB34572 KB Last Updated: 19 Nov 2019Version: 2.0
Summary:

On Aloha HQoS MPC cards (MPC2E NG HQoS and MPC3E NG HQoS), IFDs are seen to be missing in the show chassis hardware output after committing the ingress-and-egress knob on the PICs.

This article explains why the IFDs go missing and the Aloha HQoS MPC fails to allocate loopback streams, and provides a recommendation on the ideal configuration to resolve this issue.

Note: MPC reboots automatically upon commit to add or delete the ingress-and-egress knob.

 

Symptoms:

This behavior can be observed as follows:

  1. 40 SFPs are installed into all the Xcvr ports.

User@mx240> show chassis hardware

Hardware inventory:

Item             Version  Part number  Serial number     Description
Chassis                                JN1267EE7AFC      MX240
Midplane         REV 39   750-047865   ACRK4054          Enhanced MX240 Backplane
FPM Board        REV 04   760-059207   CAJW7791          Front Panel Display
PEM 0            Rev 01   740-063046   QCS1746U01V       PS 1.4-2.52kW; 90-264V AC in
PEM 1            Rev 01   740-063046   QCS1744U1FV       PS 1.4-2.52kW; 90-264V AC in
Routing Engine 0 REV 14   740-031116   9016122650        RE-S-1800x4
CB 0             REV 05   750-062577   CAJL4747          Enhanced MX SCB
FPC 1            REV 15   750-054903   CAFY2182          MPC2E NG HQoS
  CPU            REV 12   711-045719   CAFS2278          RMPC PMB
  MIC 0          REV 26   750-028392   CAAE6165          3D 20x 1GE(LAN) SFP
    PIC 0                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 02   740-011613   PHE5YBK           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152428WSF       SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428QUP       SFP-SX
      Xcvr 3     REV 01   740-031851   AM1237SYVUA       SFP-SX
      Xcvr 4     REV 01   740-031851   AM152428WCV       SFP-SX
      Xcvr 5     REV 01   740-031851   AM152428QUY       SFP-SX
      Xcvr 6     REV 01   740-031851   AM152428WRF       SFP-SX
      Xcvr 7     REV 01   740-031851   PP10PWM           SFP-SX
      Xcvr 8     REV 01   740-007326   P5S0P6M           SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428WS9       SFP-SX
    PIC 1                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 01   740-011782   PC30N6E           SFP-SX
      Xcvr 1     REV 01   740-031851   PQ34R0C           SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428QUL       SFP-SX
      Xcvr 3     REV 01   740-031851   PND7YMA           SFP-SX
      Xcvr 4     REV 01   740-031851   PM75SMY           SFP-SX
      Xcvr 5     REV 02   740-011613   PNR0Q79           SFP-SX
      Xcvr 6     REV 01   740-031851   PM75VUM           SFP-SX
      Xcvr 7     REV 02   740-011613   PKC0WHW           SFP-SX
      Xcvr 8     REV 02   740-011613   AM152629CDD       SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428QUQ       SFP-SX
  MIC 1          REV 27   750-028392   CABA7666          3D 20x 1GE(LAN) SFP
    PIC 2                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 02   740-011613   PGL2LGR           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152428QTG       SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428REH       SFP-SX 
      Xcvr 3     REV 01   740-031851   PMS60YK           SFP-SX 
      Xcvr 4     REV 01   740-031851   PP10NDU           SFP-SX
      Xcvr 5     REV 02   740-011613   PNQ5S3Q           SFP-SX
      Xcvr 6     REV 01   740-031851   PQ34TDW           SFP-SX
      Xcvr 7     REV 01   740-011613   PBN09TW           SFP-SX
      Xcvr 8     REV 02   740-011613   PJH259D           SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428RET       SFP-SX
    PIC 3                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 01   740-011782   P9B0PUY           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152629EXJ       SFP-SX
      Xcvr 2     REV 02   740-011613   PNR0PNP           SFP-SX
      Xcvr 3     REV 02   740-011613   PU800P8           SFP-SX
      Xcvr 4     REV 02   740-011613   PHH68MV           SFP-SX
      Xcvr 5     REV 01   740-011782   P8J2HLX           SFP-SX
      Xcvr 6     REV 02   740-011613   PJH25R7           SFP-SX
      Xcvr 7     REV 01   740-031851   PP10R73           SFP-SX
      Xcvr 8     REV 01   740-031851   AM1045SU7TE       SFP-SX
      Xcvr 9     REV 01   740-011782   PAN53TE           SFP-SX
Fan Tray 0       REV 02   711-059360   CAJJ5284          Enhanced Fan Tray
  1. The ingress-and-egress knob is enabled on PIC 0, 1, 2, and 3

User@mx240# show |compare
[edit chassis]
+   fpc 1 {
+       pic 0 {
+           traffic-manager {
+               mode ingress-and-egress;
+           }
+           max-queues-per-interface 4;
+       }
+       pic 1 {
+           traffic-manager {
+               mode ingress-and-egress;
+           }
+           max-queues-per-interface 4;
+       }
+       pic 2 {
+           traffic-manager {
+               mode ingress-and-egress;
+           }
+           max-queues-per-interface 4;
+       }
+       pic 3 {
+           traffic-manager {
+               mode ingress-and-egress;
+           }
+           max-queues-per-interface 4;
+       }
+   }


[edit]
User@mx240# commit
commit complete


User@mx240> show configuration chassis
fpc 1 {
    pic 0 {
        traffic-manager {
            mode ingress-and-egress;
        }
        max-queues-per-interface 4;
    }
    pic 1 {
        traffic-manager {
            mode ingress-and-egress;
        }
        max-queues-per-interface 4;
    }
    pic 2 {
        traffic-manager {
            mode ingress-and-egress;
        }
        max-queues-per-interface 4;
    }
    pic 3 {
        traffic-manager {
            mode ingress-and-egress;
        }
        max-queues-per-interface 4;
    }
}
  1. Only 10 SFPs under PIC 0 show up but the others do not.

FPC 1            REV 15   750-054903   CAFY2182          MPC2E NG HQoS
  CPU            REV 12   711-045719   CAFS2278          RMPC PMB
  MIC 0          REV 26   750-028392   CAAE6165          3D 20x 1GE(LAN) SFP
    PIC 0                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 02   740-011613   PHE5YBK           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152428WSF       SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428QUP       SFP-SX 
      Xcvr 3     REV 01   740-031851   AM1237SYVUA       SFP-SX
      Xcvr 4     REV 01   740-031851   AM152428WCV       SFP-SX
      Xcvr 5     REV 01   740-031851   AM152428QUY       SFP-SX
      Xcvr 6     REV 01   740-031851   AM152428WRF       SFP-SX
      Xcvr 7     REV 01   740-031851   PP10PWM           SFP-SX 
      Xcvr 8     REV 01   740-007326   P5S0P6M           SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428WS9       SFP-SX
    PIC 1                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
  MIC 1          REV 27   750-028392   CABA7666          3D 20x 1GE(LAN) SFP
    PIC 2                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
    PIC 3                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
Fan Tray 0       REV 02   711-059360   CAJJ5284          Enhanced Fan Tray
  1. The following error messages indicate that resources could not be allocated for the last several ge-1/1/* ports.

*** messages ***
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 1 (Unknown) on IF command 3 (IFD add)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 226 (IFD HW aconfig)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 15 (IFD aconfig)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 13 (IFD mconfig)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 54 (Ether set uint8)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 54 (Ether set uint8)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 54 (Ether set uint8)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 151 (Ether set uint32)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 163 (Unknown)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 275 (Unknown)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 1 (Unknown) on IF command 3 (IFD add)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 226 (IFD HW aconfig)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 15 (IFD aconfig)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 13 (IFD mconfig)
Jun  6 18:00:55.765  mx240 kernel: if_pfe: Error 8 (No IFD) on IF command 55 (Ether set boolean)


############ ge-1/1/4 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ######
Jun  6 18:01:07.643  mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/4 phy-port 14 offset 14 
Jun  6 18:01:07.743  mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 
Jun  6 18:01:07.915  mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:14 
Jun  6 18:01:08.075  mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/4 - ingress queueing traffic stream attach failed. 
Jun  6 18:01:08.205  mx240 fpc1 cos-halp: setup ifd:240, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes 
Jun  6 18:01:09.742  mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/4 EGRESS 
Jun  6 18:01:11.536  mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/4. Ingress queueing 1 


############ ge-1/1/5 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ######
Jun  6 18:01:11.663  mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/5 phy-port 15 offset 15 
Jun  6 18:01:11.763  mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 
Jun  6 18:01:11.935  mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:15 
Jun  6 18:01:12.095  mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/5 - ingress queueing traffic stream attach failed. 
Jun  6 18:01:12.224  mx240 fpc1 cos-halp: setup ifd:241, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes 
Jun  6 18:01:13.762  mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/5 EGRESS 
Jun  6 18:01:15.555  mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/5. Ingress queueing 1 


############ ge-1/1/6 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ######
Jun  6 18:01:20.521  mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/6 phy-port 16 offset 16 
Jun  6 18:01:20.622  mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 
Jun  6 18:01:20.793  mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:16 
Jun  6 18:01:20.954  mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/6 - ingress queueing traffic stream attach failed. 
Jun  6 18:01:21.083  mx240 fpc1 cos-halp: setup ifd:242, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes 
Jun  6 18:01:22.620  mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/6 EGRESS 
Jun  6 18:01:24.414  mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/6. Ingress queueing 1 


############ ge-1/1/7 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ######
Jun  6 18:01:29.405  mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/7 phy-port 17 offset 17 
Jun  6 18:01:29.480  mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 
Jun  6 18:01:29.652  mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:17 
Jun  6 18:01:29.813  mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/7 - ingress queueing traffic stream attach failed. 
Jun  6 18:01:29.942  mx240 fpc1 cos-halp: setup ifd:243, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes 
Jun  6 18:01:31.479  mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/7 EGRESS
Jun  6 18:01:33.273  mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/7. Ingress queueing 1 


############ ge-1/1/8 ingress queueing traffic stream attach failed with loopback stream number (16) limitation ######
Jun  6 18:01:38.239  mx240 fpc1 pfe_alloc_iq_stream_offset: ifd ge-1/1/8 phy-port 18 offset 18 
Jun  6 18:01:38.339  mx240 fpc1 XMCHIP(0): xmchip_stream_loopback_alloc: Couldn't allocate loopback stream index - loopback_stream_bitstr 0x1a053d28, num_streams 16 
Jun  6 18:01:38.511  mx240 fpc1 jnh_stream_attach_lpbk_stream(2528): Failed to allocate loopback stream for pfe:0app:Ingress Queueing 1G stream_offset:18 
Jun  6 18:01:38.671  mx240 fpc1 pfe_jam_trinity_set_cchip_stream: ge-1/1/8 - ingress queueing traffic stream attach failed. 
Jun  6 18:01:38.801  mx240 fpc1 cos-halp: setup ifd:244, pfe-id 0, direction:egress, queue capable:yes, ingress queue capable:yes 
Jun  6 18:01:40.338  mx240 fpc1 PFE: pfe_ifd_chip_setup() cos_halp_ifd_setup failed for ge-1/1/8 EGRESS 
Jun  6 18:01:42.132  mx240 fpc1 PFE: pfe_ifd_chip_setup() xm/mqchip_ifd_setup failed for ifd ge-1/1/8. Ingress queueing 1 

 

Cause:

Only 10 ports are available under PIC 0 after committing the ingress-and-egress knob configuration on all the 4 PICs in the FPC. This is due to a hardware limitation on Aloha HQoS cards (MPC2E NG HQoS and MPC3E NG HQoS) for both the 80G and 130G models.

Ingress queuing can be enabled for a maximum of 10 ports per MIC slot, and 20 ports are enabled per Aloha line card with 10 ports per MIC slot. The XM chip has a resource called "loopback streams," which is shared among tunnel interfaces, inline services, ingress queuing, and so on. For ingress queuing, only 10 ports worth of loopback streams are reserved per MIC.

 

Solution:

When the configuration is set with PIC 0 and PIC 2 in FPC 1, all the ports are present in the output of the show chassis hardware command and available.

The above issue occurs when all the four PICs with 10 ports each have the configuration. If only 10 per MIC are configured, for example PIC 0 and PIC 2 or PIC 1 and PIC 3, the issue will not be seen.

User@mx240> show configuration chassis
fpc 1 {
    pic 0 {
        traffic-manager {
            mode ingress-and-egress;
        }
        max-queues-per-interface 4;
    }
    pic 2 {
        traffic-manager {
            mode ingress-and-egress;
        }
        max-queues-per-interface 4;
    }
}
network-services enhanced-ip;

FPC 1            REV 15   750-054903   CAFY2182          MPC2E NG HQoS
  CPU            REV 12   711-045719   CAFS2278          RMPC PMB
  MIC 0          REV 26   750-028392   CAAE6165          3D 20x 1GE(LAN) SFP
    PIC 0                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 02   740-011613   PHE5YBK           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152428WSF       SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428QUP       SFP-SX
      Xcvr 3     REV 01   740-031851   AM1237SYVUA       SFP-SX
      Xcvr 4     REV 01   740-031851   AM152428WCV       SFP-SX
      Xcvr 5     REV 01   740-031851   AM152428QUY       SFP-SX
      Xcvr 6     REV 01   740-031851   AM152428WRF       SFP-SX
      Xcvr 7     REV 01   740-031851   PP10PWM           SFP-SX
      Xcvr 8     REV 01   740-007326   P5S0P6M           SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428WS9       SFP-SX
    PIC 1                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 01   740-011782   PC30N6E           SFP-SX
      Xcvr 1     REV 01   740-031851   PQ34R0C           SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428QUL       SFP-SX
      Xcvr 3     REV 01   740-031851   PND7YMA           SFP-SX
      Xcvr 4     REV 01   740-031851   PM75SMY           SFP-SX
      Xcvr 5     REV 02   740-011613   PNR0Q79           SFP-SX
      Xcvr 6     REV 01   740-031851   PM75VUM           SFP-SX
      Xcvr 7     REV 02   740-011613   PKC0WHW           SFP-SX
      Xcvr 8     REV 02   740-011613   AM152629CDD       SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428QUQ       SFP-SX
  MIC 1          REV 27   750-028392   CABA7666          3D 20x 1GE(LAN) SFP
    PIC 2                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 02   740-011613   PGL2LGR           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152428QTG       SFP-SX
      Xcvr 2     REV 01   740-031851   AM152428REH       SFP-SX 
      Xcvr 3     REV 01   740-031851   PMS60YK           SFP-SX 
      Xcvr 4     REV 01   740-031851   PP10NDU           SFP-SX
      Xcvr 5     REV 02   740-011613   PNQ5S3Q           SFP-SX
      Xcvr 6     REV 01   740-031851   PQ34TDW           SFP-SX
      Xcvr 7     REV 01   740-011613   PBN09TW           SFP-SX
      Xcvr 8     REV 02   740-011613   PJH259D           SFP-SX
      Xcvr 9     REV 01   740-031851   AM152428RET       SFP-SX
    PIC 3                 BUILTIN      BUILTIN           10x 1GE(LAN) SFP
      Xcvr 0     REV 01   740-011782   P9B0PUY           SFP-SX
      Xcvr 1     REV 01   740-031851   AM152629EXJ       SFP-SX
      Xcvr 2     REV 02   740-011613   PNR0PNP           SFP-SX
      Xcvr 3     REV 02   740-011613   PU800P8           SFP-SX
      Xcvr 4     REV 02   740-011613   PHH68MV           SFP-SX
      Xcvr 5     REV 01   740-011782   P8J2HLX           SFP-SX
      Xcvr 6     REV 02   740-011613   PJH25R7           SFP-SX
      Xcvr 7     REV 01   740-031851   PP10R73           SFP-SX
      Xcvr 8     REV 01   740-031851   AM1045SU7TE       SFP-SX
      Xcvr 9     REV 01   740-011782   PAN53TE           SFP-SX
Fan Tray 0       REV 02   711-059360   CAJJ5284          Enhanced Fan Tray

 

Modification History:

2019-11-19: Technical publication link added to Related Links for reference

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Security Alerts and Vulnerabilities

Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search