Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Junos] Validation fails during upgrade due to LPDFD

0

0

Article ID: KB24154 KB Last Updated: 27 Mar 2014Version: 1.0
Summary:

When Junos OS is upgraded on backup RE with GRES disabled, the validation fails and the following error message appears:

Validating against /config/juniper.conf.gz Local Policy Decision Function: LPDFD failed to initialize the shared-memory.
mgd: error: configuration check-out failed

This article explains the details of the issue and provides a solution.

Symptoms:

To reproduce the problem, follow these steps:

  1. Deactivate chassis redundancy graceful-switchover on master RE for upgrade preparation
  2. Note that the following files have been deleted manually before this upgrade on backup RE:

    /var/tmp/shm_ipc_sem_<*>
  3. Log in to the backup RE and execute the command "request system software add  jinstall-<version>"

  4. The following error message is reported and validation fails:
Validating against /config/juniper.conf.gz Local Policy Decision Function: LPDFD failed to initialize the shared-memory.
mgd: error: configuration check-out failed
Cause:

The cause is related to the leak of system v kernel semaphores and default value is 10. The shared-memory system call returns errors when the limit is exceeded.

In Junos OS 9.3, 6 out of 10 semaphores are part of normal allocations every time the host is booted. In /var/tmp, shm-ipc semaphore files named shm_ipc_sem* represent those semaphores. If somehow those files get deleted and one of the related daemons (dcd, cosd, dfwd, rpd) is restarted, (for example, disabled GRES on master) the routing engine (RE) will cause these daemons restart on the backup RE. This action will introduce a leak of 1 semaphore. These files do not get deleted on a request system storage cleanup. The only way to delete these files is if someone executes a rm command from the /var/tmp directory.

One way to find out if these files got deleted for the life time of the router is to look at the files' timestamps. The timestamp should match the router boot up time.

root@router% ls -l /var/tmp/shm_ipc_sem_*
-rw-r--r-- 1 root field 0 Jun 5 15:54 /var/tmp/shm_ipc_sem_1   <<< Jun 5 15:54 around should be the router boot up time, if not it means these files get deleted.
-rw-r--r-- 1 root field 0 Jun 5 15:54 /var/tmp/shm_ipc_sem_2
-rw-r--r-- 1 root field 0 Jun 5 15:54 /var/tmp/shm_ipc_sem_3
-rw-r--r-- 1 root field 0 Jun 5 15:54 /var/tmp/shm_ipc_sem_4
Solution:

A graceful routing engine switchover (GRES) switchover does not recover this semaphore leak. The only way to recover is to reboot the routing engine.

As a workaround, just perform a GRES and reboot the new backup. Then the validation of the Junos OS upgrade should pass.

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search