Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[PTX] Type-3 FPC/10X100GE PIC uses FEC91 on port 2 for LR4 type QSFP optics

0

0

Article ID: KB34777 KB Last Updated: 27 Jul 2019Version: 1.0
Summary:

On PTX3k/5k with Type-3 FPC(FPC3-SFF-PTX-1X) and 10x100GE PIC, we have the following specifications on the Forward Error Correction (FEC) mode:

  • For LR4 type QSFP optics, the default FEC mode is 'FEC91' (RS-FEC) on port 2, and 'NONE' on the other ports (0,1,3-9).
  • For SR4 type QSFP optics, the default FEC mode is 'FEC91' on all the ports.

Although there is a config knob to configure the FEC mode via CLI, there is no need to change the FEC mode at all.

Symptoms:

On 10X100GE PIC, there is a signal integrity issue on port 2 with LR4 optics, so the hardware design is different from the other ports. To overcome the signal integrity issue on port 2 in 100G mode, FEC91 is enabled between PFE ASIC and Retimer chip. FEC91 is always enabled on port 2. This will protect the data payload between PFE ASIC and Retimer chip with FEC when optic type is LR4. 

Do not use FEC mode NONE on port 2. No CLI configuration is needed. If port 2 is connected to a 10x100G port, do not configure FEC mode on the peer port, because FEC is terminated on the Retimer. The data that comes out of port 2 has no FEC in the data stream.

[1] LR4 type optics

FPC 0            REV 40   750-057064   ACPJ9270          FPC3-SFF-PTX-1X
  CPU                     BUILTIN      BUILTIN           SMPC PMB
  PIC 0          REV 05   750-070203   ACPN1419          10x100/10x40/40x10GE
    Xcvr 2       REV 01   740-058732   1GCQA21002X       QSFP-100GBASE-LR4
FPC 8            REV 46   750-057064   BCAE8009          FPC3-SFF-PTX-1X
  CPU                     BUILTIN      BUILTIN           SMPC PMB
  PIC 0          REV 06   750-070203   BCAD9837          10x100/10x40/40x10GE
    Xcvr 0       REV 01   740-058732   1DJQA2190FN
      QSFP-100GBASE-LR4
    Xcvr 1       REV 01   740-058732   1BTQA218075       QSFP-100GBASE-LR4
    Xcvr 2       REV 01   740-058732   1GCQA1160EU       QSFP-100GBASE-LR4

Test 1-1

The port 2 to port 2 connection uses FEC91 by default in between PFE ASIC and PIC Retimer.

          PTX3000 et-0/0/2 -------- Up -------- et-8/0/2 PTX3000
QSFP-100GBASE-LR4                                  QSFP-100GBASE-LR4


PFE ASIC       PIC Retimer       LR4                  LR4         PIC Retimer       PFE ASIC
  FEC            FEC          optics                  optics      FEC               FEC
Enabled -- FEC91 --> Terminated -- >--- no FEC data --->--- Enabled -- FEC91 --> Terminated
Terminated <-- FEC91 --- Enabled--- <-- no FEC data ---< --- Terminated <-- FEC91 --- Enabled

User@PTX3000> show configuration interfaces

User@PTX3000> show interfaces et-0/0/2 media
Physical interface: et-0/0/2, Enabled, Physical link is Up
  Interface index: 144, SNMP ifIndex: 587
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                          4842
    Errored blocks                      4986
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors             10981643
    FEC Uncorrected Errors             160411
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0

User@PTX3000> show interfaces et-8/0/2 media
Physical interface: et-8/0/2, Enabled, Physical link is Up
  Interface index: 133, SNMP ifIndex: 554
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                            13
    Errored blocks                        12
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors            208198928
    FEC Uncorrected Errors            1038527
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0

Test 1-2 

Port 2 uses FEC91 in between PFE ASIC and Retimer, and port 1 uses NONE by default.

Data that comes out of port 2 has no FEC in the data stream, so the peer port (non port 2) must be NONE.

          PTX3000 et-0/0/2 -------- Up -------- et-8/0/1 PTX3000
QSFP-100GBASE-LR4                                       QSFP-100GBASE-LR4

PFE ASIC       PIC Retimer     LR4                     LR4        PIC Retimer      PFE ASIC
  FEC91          FEC91      optics                     optics     FEC NONE         FEC NONE
Enabled --- FEC91 --> Terminated --- >--- no FEC data ---> --------no FEC data -----------
Terminated <-- FEC91 --- Enabled --- <--- no FEC data ---< --------no FEC data -----------

User@PTX3000> show configuration interfaces                                 

User@PTX3000> show interfaces et-0/0/2 media
Physical interface: et-0/0/2, Enabled, Physical link is Up
  Interface index: 144, SNMP ifIndex: 587
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                          4842
    Errored blocks                      4986
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors             10981643
    FEC Uncorrected Errors             160411
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0


User@PTX3000> show interfaces et-8/0/1 media
Physical interface: et-8/0/1, Enabled, Physical link is Up
  Interface index: 143, SNMP ifIndex: 553
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                             0
    Errored blocks                         0
  Ethernet FEC Mode  :                   NONE
  Ethernet FEC statistics              Errors
    FEC Corrected Errors                    0
    FEC Uncorrected Errors                  0
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0

Test 1-3 

Port 2 uses FEC91 in between PFE ASIC and Retimer.

Data that comes out of port 2 has no FEC in the data stream because FEC is terminated on the Retimer on port 2. If on the peer port connected to port 2 is FEC enabled, the link will be DOWN due to FEC mode mismatch, because the peer expects to receive FEC data.

User@PTX3000> show configuration interfaces
et-8/0/1 {
    gigether-options {
        fec fec91;
    }
}


User@PTX3000> show interfaces et-0/0/2 media
Physical interface: et-0/0/2, Enabled, Physical link is Down
  Interface index: 144, SNMP ifIndex: 587
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running Down

  Active alarms  : LINK
  Active defects : LINK, LOCAL-FAULT
  PCS statistics                      Seconds
    Bit errors                            20
    Errored blocks                        90
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors             48596652
    FEC Uncorrected Errors             487695
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0


User@PTX3000> show interfaces et-8/0/1 media
Physical interface: et-8/0/1, Enabled, Physical link is Down
  Interface index: 143, SNMP ifIndex: 553
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running Down

  Active alarms  : LINK
  Active defects : LINK, LOCAL-FAULT
  PCS statistics                      Seconds
    Bit errors                             0
    Errored blocks                         0
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors                    0
    FEC Uncorrected Errors                  0
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0

Test 1-4 

The non port 2 to non port 2 connection do not use FEC.

User@PTX3000> show configuration interfaces

User@PTX3000> show interfaces et-8/0/0 extensive
Physical interface: et-8/0/0, Enabled, Physical link is Up
  Interface index: 129, SNMP ifIndex: 504, Generation: 143
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled,
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                             2
    Errored blocks                         9
  Ethernet FEC Mode  :                   NONE
  Ethernet FEC statistics              Errors
    FEC Corrected Errors                    0
    FEC Uncorrected Errors                  0
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0

 

User@PTX3000> show interfaces et-8/0/1 extensive
Physical interface: et-8/0/1, Enabled, Physical link is Up
  Interface index: 143, SNMP ifIndex: 553
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled,
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                             0
    Errored blocks                         0
  Ethernet FEC Mode  :                   NONE
  Ethernet FEC statistics              Errors
    FEC Corrected Errors                    0
    FEC Uncorrected Errors                  0
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0


[2]  SR4 type optics

Test 2-1

For SR4 opotics, FEC is enabled by default on all ports, and we turn off the FEC on the Retimer. This will maintain the end to end FEC (far end port need to have FEC enabled). If FEC is configured to NONE on an end, the link will be Down due to FEC mode mismatch.

User@PTX3000> show interfaces et-0/0/2 media
Physical interface: et-0/0/2, Enabled, Physical link is Up
  Interface index: 134, SNMP ifIndex: 587
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled,
  Flow control: Enabled
  Device flags   : Present Running


  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                             0
    Errored blocks                         0
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors                 2752
    FEC Uncorrected Errors                  0
    FEC Corrected Errors Rate              45
    FEC Uncorrected Errors Rate             0

User@PTX3000> show interfaces et-0/0/6 media
Physical interface: et-0/0/6, Enabled, Physical link is Up
  Interface index: 138, SNMP ifIndex: 591
  Link-level type: Ethernet, MTU: 1514, Speed: 100Gbps, BPDU Error: None,
  Loop Detect PDU Error: None, Loopback: Disabled, Source filtering: Disabled, 
  Flow control: Enabled
  Device flags   : Present Running

  Active alarms  : None
  Active defects : None
  PCS statistics                      Seconds
    Bit errors                             0
    Errored blocks                         0
  Ethernet FEC Mode  :                  FEC91
  Ethernet FEC statistics              Errors
    FEC Corrected Errors                    0
    FEC Uncorrected Errors                  0
    FEC Corrected Errors Rate               0
    FEC Uncorrected Errors Rate             0

Solution:

The PIC (10x100/10x40/40x10GE) has a signal integrity issue on port 2 in 100Gb mode with LR4 optics. In order to make this port work reliably, the hardware design is different from the other ports. 

Junos software has made a change to run RS-FEC between PFE ASIC and Retimer. RS-FEC is always enabled on port 2 to protect the link between ASIC and Retimer. 

If we do not enable FEC, then the link becomes unstable. Do not use CLI to change FEC mode on this port. It will cause issues. If we connect port 2 to another 10x100G port, we must not configure FEC on the peer, and it will work correctly; if you set FEC, it will break this link.

If SR4 type optics is installed, the Retimer by default will not have FEC enabled, and FEC end to end will be activated. For non 100G operation, FEC will not be activated.

Normally, PTX3k/5k users do not need to specify which FEC mode they want to use via CLI. The driver software does all the work and sets FEC mode as needed. 

Although a config knob is provided, there is no need to use CLI to configure FEC mode at all.

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search