[Subscriber Management] MX bbesmgd daemon reports 'RTT exceeded threshold for FPC X, throttle_phase_count xxx'

Summary:

This article explains the reason for 'RTT exceeded threshold' message on MX broadband network gateway (BNG), and introduces the feature of RTT Load Throttle of resource-monitor.

Symptoms:

Example 'RTT exceeded threshold' message:

Jan  6 02:09:27 2020  MX-BNG bbesmgd[18216]: %DAEMON-4-SMD_RSMON_RTT_EXCEEDED: RTT exceeded threshold for FPC 3, throttle_phase_count 1
Jan  6 02:09:30 2020  MX-BNG bbesmgd[18216]: %DAEMON-4-SMD_RSMON_RTT_NOMINAL: RTT returned below threshold for FPC 3
Jan  6 12:27:36 2020  MX-BNG bbesmgd[18216]: %DAEMON-4-SMD_RSMON_RTT_EXCEEDED: RTT exceeded threshold for FPC 3, throttle_phase_count 2
Jan  6 12:28:18 2020  MX-BNG bbesmgd[18216]: %DAEMON-4-SMD_RSMON_RTT_NOMINAL: RTT returned below threshold for FPC 3

When this message is seen, the subscriber login may be failed.

Cause:

With RTT Load Throttle feature enabled, when PFE becomes too busy, RSMON RTT Load Throttle feature kicks in and deny the new subscriber login; on subscriber login retry and PFE CPU in normal level, login will succeed. Without this feature, the subscriber would have gone to active state but no traffic will be flowing for a period of time.

Use the command below to check if RTT Load Throttle feature has been enabled:

user@MX-BNG> show system resource-monitor summary 
Resource Usage Summary

Throttle                       : Enabled      
Load Throttle                  : Enabled  <-- indicates RTT Load Throttle has been enabled   
Heap Mem Threshold             : 70  %
IFL Counter Threshold          : 95  %
Round Trip Delay Threshold(ms) : 1000     <-- RTT Throttle value
Filter Counter Threshold       : 100 %
Expansion Threshold            : 95  %
CoS Queue Threshold            : 100 %
MFS threshold                  : 70  %        Used : 0   

Slot # 0   
     Client allowed                   : Yes  
     Service allowed                  : Yes  
     Heap memory used                 : 339204848       In % : 18  
     Average Round-trip Delay(ms)     : 103  (30  )     Round-trip Delay(ms) : 103  <-- RTT Delay, the 30 in bracket means that the average is taken for last 30 secs

     MAX session rate allowed(%)      : 100 
     Client denied                    : 1524  <-- the counter of new subscribers have been denied    
     Service Denied                   : 0     
     Performance Denial Client        : 1524  <-- 
     Performance Denial Service       : 0   
     IFL Denied                       : 0   

                Filter counter memory      IFL counter memory   Expansion memory
      PFE #         used  |   %             used  |   %          used  |   %
          0       17848880     44           304096      0             0      0
          1       17754480     44           274832      0             0      0

     CoS Queue Utilization
     PFE #   Scheduler Block #        Used      %
         0                   0          88      0
         0                   1          88      0
         1                   0          24      0
         1                   1          24      0

If 'Average Round-trip Delay' is 'exceeded Round Trip Delay Threshold', the 'RSMON RTT Load Throttle' is triggered and system reports the log message.

Solution:

There is a hidden command to adjust RTT Threshold value. Setting it to a large value can decrease the frequency of logging.

user@MX-BNG# set system services resource-monitor rtt-parameters rt-delay-threshold <value-in-millisecs>

If you prefer to disable this feature, use command below:

user@MX-BNG# set system services resource-monitor no-load-throttle

Knowledge Base