Support Support Downloads Knowledge Base Apex Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[PTX] Junos vmhost commands timeout or upgrade failure with vhclient daemon connection refused error[1]

0

0

Article ID: KB34513 KB Last Updated: 21 Sep 2019Version: 1.0
Summary:

The vhclient connection is refused and ':error[1]' errors are continuously logged in the following format:

[date][time][nodename] vhclient.[pid].daemon: [em1 internal communication v4 address]: [error]

Symptoms:

The device will not connect and the following messages are continuously logged by default in /var/log/messages:

May 30 14:16:28.637 lab-router vhclient.18619.daemon: 192.168.1.1: Connection refused :error[1]

This may cause some 'show vmhost <*>' commands to hang without ever displaying output and require ^c on the CLI to break out and regain CLI prompt.

labroot@lab-router> show vmhost version 
^C[abort]

The Junos VM compute node will show Failed status.

labroot@lab-router> show vmhost    status
Compute cluster: rainier-re-cc
  Compute Node: rainier-re-cn, Online
    Note: Compute node has the following errors:
          - Failed

While in this state, Junos upgrades may fail as the upgrade requires communication between the Host OS and Junos VM. Pings to the Host OS over em1/virbr0 port (192.168.1.0/24) private instance network may succeed while in this state.

labroot@lab-router> ping 192.168.1.1 routing-instance __juniper_private4__
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.389 ms
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.708 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.423 ms

Other system daemons will have established connections and will be in functional state.

labroot@lab-router> show system connections | match "192\.168\.1\."

tcp4       0      0  192.168.1.2.53729     192.168.1.1.33081      ESTABLISHED
tcp4       0      0  192.168.1.2.33081     192.168.1.1.53239      ESTABLISHED
tcp4       0      0  192.168.1.2.6022      192.168.1.1.54327      ESTABLISHED
tcp4       0      0  192.168.1.2.5014      192.168.1.1.49843      ESTABLISHED
tcp4       0      0  192.168.1.2.58000     192.168.1.1.58064      ESTABLISHED
Cause:

The communication failure between the Host OS and Junos VM occurs over the internal network on the platform over the em1 and virbr0 bridged connection in the __juniper_priivate4__ routing instance. This does not affect forwarding-plane of the device. However, it will interfere with Junos upgrades and will prevent some 'show vmhost' commands from completed.

Junos VM Architecture
Solution:

Due to communication failure with the Host OS, data collection at the host is not possible.

Steps to take if this issue occurs:

  1. Collect the following outputs:

    • request system information
    • Tarfile of /var/log/*
  2. Contact your JTAC Representative

  3. Power cycle the device.

Note: Please refer to Compressing Troubleshooting Logs from /var/logs to Send to Juniper Networks Technical Support

In addition to providing output from request support information and logs, the data collection below should also be provided to facilitate troubleshooting. These commands execute on the Routing-Engine Shell and provides stats for the internal connection between the Host and VM. 

start shell
ifconfig
netstat -a
netstat -i em1
netstat -s
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search