Support Support Downloads Knowledge Base Juniper Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[MX] Logs incrementing with log messages "rt_pfe_veto:"

1

0

Article ID: KB36114 KB Last Updated: 11 Nov 2021Version: 2.0
Summary:

This article explains the meaning of the "rt_pfe_veto" error messages in the logs and details the steps to perform when you observe these messages.

 

Symptoms:

Users may notice a network impact that shows tasks are piling up in the KRT queue, upon investigation. Review of the log messages reveals the following:

Jun 24 09:13:29  lab-re0 /kernel: rt_pfe_veto: Too many delayed route/nexthop unrefs. Op 2 err 55, rtsm_id 5:-1, msg type 2
Jun 24 09:13:29  lab-re0 /kernel: rt_pfe_veto: Memory usage of M_RTNEXTHOP type = (0) Max size possible for  M_RTNEXTHOP type = (16711550976) Current delayed unref = (40151), Current unique delayed unref = (40000), Max delayed unref on this platform = (40000) Current delayed weight unref = (40151) Max delayed weight unref on this platform= (400000) curproc = rpd
Jun 24 09:13:29  lab-re0 /kernel: rt_pfe_veto: Possible slowest client is xdpc3. States processed - 82282463. States to be processed - 2445
Jun 24 09:13:29  lab-re0 /kernel: rt_pfe_veto: Possible second slowest client is xdpc4. States processed - 82282463. States to be processed - 2445

What is the meaning of these messages and why are the jobs not being processed?

 

Cause:

The above messages ("rt_pfe_veto") are logged due to congestion in relation to the updates from RPD to kernel, and then to the PFE so the forwarding tables could be programmed.

When a major change occurs, the routing protocol daemon (RPD) sends updates to the kernel to maintain the current status of the routing tables. These changes can include:

  • Routing Engine mastership switchover
  • Routing Engine reboot
  • Restart of the routing daemon, which causes a rebuild of the routing tables
  • Links to next hops flapping
  • IGP/BGP convergence
  • Memory pressure due to the large number of routes or prefixes

The "veto" system is a memory system to protect the kernel memory from allocations for various processes and without this system, all the kernel memory may get used up.

These messages are generally generated when the router has a large number of updates to process that need to be sent to the PFE. After the router processes all the available updates, the above logs will get cleared.

 

Solution:

To determine if these log messages are reporting stalling on your device, collect the following information at the moment of congestion, and check if there are any "stuck" jobs at the time.

If there are "0 queued" jobs and the "rt_pfe_veto:" messages have stopped incrementing in the logs, it means that the router has processed all the updates and the instability period has ended. In this case, these messages can be safely ignored and treated as being cosmetic.

cli> show krt queue

Routing table add queue: 0 queued
Interface add/delete/change queue: 0 queued
Top-priority deletion queue: 0 queued
Top-priority change queue: 0 queued
Top-priority add queue: 0 queued
high priority V4oV6 tcnh delete queue: 0 queued
high priority anchor gencfg delete queue: 0 queued
High-priority multicast add/change: 0 queued
Indirect next hop top priority add/change: 0 queued
Indirect next hop add/change: 0 queued
high priority anchor gencfg add-change queue: 0 queued
MPLS add queue: 0 queued
Indirect next hop delete: 0 queued
High-priority deletion queue: 0 queued
MPLS change queue: 0 queued
High-priority change queue: 0 queued
High-priority add queue: 0 queued
Normal-priority indirect next hop queue: 0 queued
Normal-priority deletion queue: 0 queued
Normal-priority composite next hop deletion queue: 0 queued
Low priority Statistics-id-group deletion queue: 0 queued
Normal-priority change queue: 0 queued
Normal-priority add queue: 0 queued
Least-priority delete queue: 0 queued
Least-priority change queue: 0 queued
Least-priority add queue: 0 queued
Normal-priority pfe table nexthop queue: 0 queued
EVPN gencfg queue: 0 queued
Normal-priority gmp queue: 0 queued
Routing table delete queue: 0 queued
Low priority route retry queue: 0 queued

However, if you find jobs that are still stuck and not recovering, contact Support.

 

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search