Support Support Downloads Knowledge Base Apex Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[EX/QFX] Traffic drop due to microburst in COS Classification

1

0

Article ID: KB36942 KB Last Updated: 20 May 2021Version: 1.0
Summary:

How to identify micro burst traffic occurring in the network leading to packet drops in COS Queue.

Symptoms:

Dropped packets are seen on COS interface.

root@r-019> show interfaces queue ge-0/1/0    
Physical interface: ge-0/1/0, Enabled, Physical link is Up
Interface index: 649, SNMP ifIndex: 518
Forwarding classes: 16 supported, 5 in use
Egress queues: 12 supported, 5 in use
Queue: 0, Forwarding classes: best-effort
  Queued:
    Packets              :                     0                     0 pps
    Bytes                :                     0                     0 bps
  Transmitted:
    Packets              :           92299418556                122542 pps
    Bytes                :        92299407936208             980338976 bps
    Tail-dropped packets : Not Available  
    RL-dropped packets   :                     0                     0 pps
    RL-dropped bytes     :                     0                     0 bps
    Total-dropped packets:          830775607880               1217312 pps
    Total-dropped bytes  :       830775515656432            9738503624 bps


The total bandwidth of this interface is 10 GBPS and 5% is allocated to queue 0. When traffic exceeds 10Gbps, the COS buffers allocated for queue 0 will appear. Then the server drops the traffic that crosses the buffer limit.

Monitored the interface utilization with SNMP traps to plot a graph which will send the query varying from 30 seconds to 5 minutes. The data utilization graph will not show the BW spiking that occurred in interval of millisecond to second making users think the interface utilization never crossed the defined limit.

Solution:

If a queue drop is seen, then the following command can be used to check the buffer and data utilization on the node from the FPC mode after running 'start shell network pfe fpc0'.

set dcbcm bcmshell "for i=0,200 'sleep 0 10000; dump chg MMU_THDU_XPIPE_COUNTER_QUEUE'"

Note: Here we are executing the above BCM counter dump command 100 times at interval of 10ms. This will provide the buffer utilizations during micro burst conditions.

Example:

When the output bandwidth of an interface is set to 10Gbps and traffic is pushed to approximately 11Gbps, the command provided the following output. 

TFXPC0(vty)# ...000; dump chg MMU_THDU_XPIPE_COUNTER_QUEUE'"                  

HW (unit 0)
Sleeping for 0.010000 seconds
MMU_THDU_XPIPE_COUNTER_QUEUE.mmu0[492]: <SHARED_COUNT=0x19db,PARITY=1,MIN_COUNT=9,ECCP=0x51,ECC=0x11,DATA=0x0033b60009>
Sleeping for 0.010000 seconds
MMU_THDU_XPIPE_COUNTER_QUEUE.mmu0[492]: <SHARED_COUNT=0x19ec,MIN_COUNT=9,DATA=0x0033d80009>
Sleeping for 0.010000 seconds
MMU_THDU_XPIPE_COUNTER_QUEUE.mmu0[492]: <SHARED_COUNT=0x19b3,MIN_COUNT=9,ECCP=0xa,ECC=0xa,DATA=0x0033660009>
Sleeping for 0.010000 seconds
MMU_THDU_XPIPE_COUNTER_QUEUE.mmu0[492]: <SHARED_COUNT=0x1a06,MIN_COUNT=9,ECCP=0x23,ECC=0x23,DATA=0x00340c0009>
MMU_THDU_XPIPE_COUNTER_QUEUE.mmu0[504]: <PARITY=1,MIN_COUNT=1,ECCP=0x43,ECC=3,DATA=0x0000000001>


When the traffic is brought down to a lower value, around 10Gbps, then the output indicates there was no micro burst:

Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds
Sleeping for 0.010000 seconds​
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search