Knowledge Search


×
 

Understanding MX Fabric

  [KB23065] Show Article Properties


Summary:

This article reviews the basics behind MX Fabric Planes.  It is referenced from the KB23173 - Resolution Guide - Troubleshoot Fabric Plane.


 
Symptoms:

Describe MX Fabric (Planes and SCB)


 
Solution:

Click one of the following links to jump to that topic:


Introduction to MX SCB                

The MX Switch Control Board (SCB) provides Control Plane functions, Chassis management functions, and Switch Plane functions. It is also a carrier for the Routing Engine (RE). Below are various facts about the SCB:                  

  • Each SCB contains two fabric ASICs, and the same SCB card is usable in MX960, MX480 and MX240.

  • The Control Plane functionality is provided primarily by a RE, an ACBC FPGA, and a gigabit Ethernet switch, which all reside on the SCB.

  • The RE is a Pentium processor subsystem that can be plugged into the SCB. The RE uses two primary interfaces to the SCB: PCI bus and gigabit Ethernet.

  • On the SCB, the RE’s PCI bus interfaces to the ACBC FPGA, which aggregates most of the system I/O and provides Chassis management functions.

  • The RE’s gigabit Ethernet port connects to a gigabit Ethernet switch on the SCB. This gigabit Ethernet switch provides Ethernet connectivity to all of the processors in the Chassis, including the standby RE, for Control Plane communications.

  • There are two SCBs in the system which can operate the control plane function in redundant mode. Each SCB contains the mastership logic that determines which SCB/RE combination is the master in the system. The master is also called the Junos master. The Junos master controls all components in the system, including the standby SCB/RE.

  • Each SCB generates an output mastership signal to each FRU. Each SCB also generates an output operational signal to each FRU. On each FRU, the mastership signal is qualified by the operational signal and combined with local status signals to select the master SCB used for FRU control.

  • Each SCB contains two switch fabric chips (ASICs). However, the switch fabric ASICs are under the control of the current Junos master.

  • Each SCB provides two planes of switch fabric for packet forwarding among the DPCs/MPCs for MX960 and four planes for MX480/MX240.

  • On MX480/MX240 Each fabric ASIC is configured in virtual plane mode, where two virtual planes exist on one fabric ASIC. The MX960 chassis may contain up to three SCB's and hence 6 planes are available. The MX480/MX240 contains a maximum of 2 SCBs and hence 8 logical planes are available.

  • On MX960, the two SCBs residing in slot 6 and slot 7 provide both Control and Switch Fabric features, while the third SCB residing in slot 8 of the chassis (hybrid slot) will only do fabric functions. Either a SCB or a DPC can be plugged into slot 8. A third SCB is used only for Switch Fabric redundancy. Therefore, if an application does not require switch fabric redundancy, a DPC can be used in slot 8.


Packet Forwarding and Switch Fabric          

The MX Series uses a distributed Packet Forwarding architecture. Each DPC/MPC contains either 1/2/4 PFEs, each of which is self contained in terms of forwarding decisions. When the forwarding decision points to an output interface on a different PFE, which can either be on the same DPC/MPC or a different DPC/MPC, the packet needs to be sent across the Fabric Plane.

Since the Packet Forwarding is implemented using a distributed architecture, the fabric architecture is both distributed and fault-tolerant. The fabric interconnect ASICs are housed in SCBs that are fully redundant on every platform. Each PFE implements the fabric queuing and flow control mechanisms required to communicate to multiple other PFEs on the chassis at the same time. A request-grant mechanism is used to implement flow control. Each PFE that wants to send a packet to a destination PFE sends a request, and only when the request is granted is it allowed to send the packet to the destination PFE.          
 

MX960 Switch Fabric Diagram with DPC

 

The MX960 Switch Fabric connectivity is as follows:
  • Each PFE is connected to all Fabric Planes
  • Diagram shows only two DPCs (FPCs) in the chassis
  • All other DPCs are also connected in same way
  • 4 Active Planes and 2 Spare Planes.
For additional details on the MX960, refer to the Technical Documentation:
MX960 3D Universal Edge Router - Components


 

MX480 / MX240 Switch Fabric Diagram with DPC

The MX480/MX240 Switch Fabric connectivity is as follows:


For additional details on the MX480, refer to the Technical Documentation:
MX480 3D Universal Edge Router - Components

For additional details on the MX240, refer to the Technical Documentation:
MX240 3D Universal Edge Router - Components
 

MX960 Switch Fabric Diagram with MPC

 

The MX960 Switch Fabric connectivity in a MPC environment:
 
NOTE:  For MX960 using the Enhanced SCB or SCBE, the MPCs will have 4 ACTIVE planes, while planes 4 and 5 will remain in SPARE mode. 
 

Fabric Behavior when Chassis has mix of MPC and DPC cards and MX SCB

A chassis (with DPC cards) in the MX960 platform uses FOUR of the fabric planes in ACTIVE mode and TWO fabric planes in SPARE mode, thus providing fabric redundancy. If MPC Cards are added into the chassis, all SIX planes are in ACTIVE mode, thus providing more fabric bandwidth. However, with the MPC cards, there will not be any fabric redundancy; except the DPC cards will still have the fabric redundancy capabilities in a case where all three SCB’s are present.  The DPC cards will only use two of the SCB’s as ACTIVE and the third SCB is a SPARE. If one of the two ACTIVE cards fail, then it will use the third SCB as ACTIVE and will continue with full fabric bandwidth capability.  From a DPC card perspective, two SCB’s are ACTIVE and one SCB is SPARE. From the chassis perspective and from MPC cards perspective, all 3 SCB’s are ACTIVE.

The MX480 and MX240 have similar behavior, where 4 fabric planes from one SCB will be ACTIVE and 4 fabric planes from the second SCB will be SPARE from the DPC perspective. All eight Planes will be ACTIVE from the MPC and chassis perspective.

The behavior of the chassis will be summarized as follows (for MX960, and the 480 and 240 have similar behavior):

  • When a MPC card is inserted in the chassis, all SIX Fabric planes are marked ACTIVE in the chassis.
  • There will be NO concept of fabric redundancy from a chassis perspective. However, internally, fabric redundancy will be available for DPC cards ONLY.
  • MPC will communicate with DPC cards only through FOUR planes considered ACTIVE in the chassis. These will be the planes that I-Chip considers as ACTIVE.
  • MPC cards will communicate with other MPC cards through ALL SIX planes in the chassis.
  • CLI will indicate that ALL SIX planes are ACTIVE.
  • Active fabric LED is turned “ON” for all three SCB’s.
 

MX Fabric Plane Component Summary:

 MX SCB  MX960  MX480  MX240
 FPC (PFE) max  12(48)  6(24)  3(12)
 SCB / SF Chips  3 max / 6  2 max / 4  2 max / 4
 Total Fabric Planes  6 max  8  8
 Spare Planes for DPC  2   4   4 
 Spare Planes for MPC  0  0  0

 
Related Links: