Support Support Downloads Knowledge Base Service Request Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[CSO]: Components health check fails with "No minions matched the target. Please check the minion running status" error

0

0

Article ID: KB34848 KB Last Updated: 12 Aug 2019Version: 1.0
Summary:

This article covers the specific scenario of a components health check failing in the installer VM and a workaround for it.

Note: This article is applicable only to CSO version 4.0.2 or earlier.

 

Symptoms:

The following error is reported during a components health check:

root@cso-installer-vm:~/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2# ./components_health.sh
Syncing salt data, this will take a while..

************************************************************************

HEALTH CHECK FOR INFRASTRUCTURE COMPONENTS STARTED IN CENTRAL ENVIRONMENT

************************************************************************

INFO     Health Check for Infrastructure Component Cassandra Started
ERROR    No minions matched the target. Please check the minion running status.

Please Check the logs:
===========================
Console Logs Path: /root/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2/logs/health_check_console.log
Error Logs Path: /root/CSO-4.0.2-/Contrail_Service_Orchestration_4.0.2/logs/health_check_error.log
Info Logs Path: /root/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2/logs/health_check.log

 

Cause:

This is due to a bug in the salt version ( 2017.7.5) used in CSO 4.0.2, due to which the data that is cached for minions by the salt master in a long-running setup is lost.

 

Solution:

The following commands need to be run in the installer VM.

  1. Run the salt-key command to check whether all the VMs are listed in the output.
# salt-key
Accepted Keys:
csp-central-infravm.R82ZN0.central
csp-central-k8mastervm.R82ZN0.central
csp-central-msvm.R82ZN0.central
csp-contrailanalytics-1.R82ZN0.central
csp-regional-sblb.KNVKBT.regional
Denied Keys:
Unaccepted Keys:
Rejected Keys:
  1. Initiate the salt '*' test.ping command from the installer VM to run a simple ping. A value of true indicates that the test was successful.

# salt '*' test.ping
csp-regional-sblb.KNVKBT.regional:
    True
csp-central-infravm.R82ZN0.central:
    True
csp-central-k8mastervm.R82ZN0.central:
    True
csp-contrailanalytics-1.R82ZN0.central:
    True
csp-central-msvm.R82ZN0.central:
    True
  1. Run the salt '*' deployutils.get_role_ips 'cassandra' command to list the IP addresses assigned to the VM/role.

In the following output, no IP address assignment is seen for the VMs.

# salt '*' deployutils.get_role_ips 'cassandra'
csp-central-k8mastervm.R82ZN0.central:
csp-central-infravm.R82ZN0.central:
csp-regional-sblb.KNVKBT.regional:
csp-central-msvm.R82ZN0.central:
csp-contrailanalytics-1.R82ZN0.central:

There is a known workaround for this problem for CSO 4.0.2 and earlier versions, which is to run the following command:


salt '*' cmd.run "service salt-minion restart"

The above command restarts the minions and makes the salt master fetch data from these minions again to keep in the local cache.


# salt '*' cmd.run "service salt-minion restart"
csp-regional-sblb.KNVKBT.regional:
    salt-minion stop/waiting
    salt-minion start/running, process 22374
csp-central-infravm.R82ZN0.central:
    salt-minion stop/waiting
    salt-minion start/running, process 28947
csp-central-k8mastervm.R82ZN0.central:
    salt-minion stop/waiting
    salt-minion start/running, process 16572
csp-contrailanalytics-1.R82ZN0.central:
    salt-minion stop/waiting
    salt-minion start/running, process 4634
csp-central-msvm.R82ZN0.central:
    salt-minion stop/waiting
    salt-minion start/running, process 28515

Note: This issue is not seen in CSO 4.1.1 versions because the components health check script first checks the salt master cache and if the cache data is missing, it syncs the minion data before running the components health check, which prevents the specified error. Refer to Salt 2017.7.5 Release Notes for more information.

 

Run the components health script again after executing the above workaround and verify the output:

root@cso-installer-vm:~/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2# ./components_health.sh
Syncing salt data, this will take a while..

************************************************************************
HEALTH CHECK FOR INFRASTRUCTURE COMPONENTS STARTED IN CENTRAL ENVIRONMENT
************************************************************************

INFO     Health Check for Infrastructure Component Cassandra Started
INFO     The Infrastructure Component Cassandra is Healthy

INFO     Health Check for Infrastructure Component ElasticSearch Started
INFO     The Infrastructure Component ElasticSearch is Healthy

INFO     Health Check for Infrastructure Component Etcd Started
INFO     The Infrastructure Component Etcd is Healthy

INFO     Health Check for Infrastructure Component MariaDb Started
INFO     The Infrastructure Component MariaDb is Healthy

INFO     Health Check for Infrastructure Component RabbitMQ Started
INFO     The Infrastructure Component RabbitMQ is Healthy

INFO     Health Check for Infrastructure Component ZooKeeper Started
INFO     The Infrastructure Component ZooKeeper is Healthy

INFO     Health Check for Infrastructure Component Redis Started
INFO     The Infrastructure Component Redis is Healthy

INFO     Health Check for Infrastructure Component ArangoDb Started
INFO     The Infrastructure Component ArangoDb is Healthy

INFO     Health Check for Infrastructure Component Sim_Cluster Started
INFO     The Infrastructure Component Sim_Cluster is Healthy

INFO     Health Check for Infrastructure Component Elk_Logstash Started
INFO     The Infrastructure Component Elk_Logstash is Healthy

INFO     Health Check for Infrastructure Component Elk_Kibana Started
INFO     The Infrastructure Component Elk_Kibana is Healthy

INFO     Health Check for Infrastructure Component Keystone Started
INFO     The Infrastructure Component Keystone is Healthy

INFO     Health Check for Infrastructure Component Swift Started
INFO     The Infrastructure Component Swift is Healthy

INFO     Health Check for Infrastructure Component Kubernetes Started
INFO     The Infrastructure Component Kubernetes is Healthy

INFO     Health Check for Infrastructure Component Contrail_Analytics Started
INFO     The Infrastructure Component Contrail_Analytics is Healthy

Overall result:

         The following Infrastructure Components are Healthy:
                 ['Cassandra', 'ElasticSearch', 'Etcd', 'MariaDb', 'RabbitMQ', 'ZooKeeper', 'Redis', 'ArangoDb', 'Sim_Cluster', 'Elk_Logstash', 'Elk_Kibana', 'Keystone', 'Swift', 'Kubernetes', 'Contrail_Analytics']

HEALTH CHECK NOT REQUIRED FOR COLLOCATED REGION

Please check the logs:
Console Logs Path: /root/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2/logs/health_check_console.log
Error Logs Path: /root/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2/logs/health_check_error.log
Info Logs Path: /root/CSO-4.0.2/Contrail_Service_Orchestration_4.0.2/logs/health_check.log

INFO     =============== Health Check start time: 2019-07-23 17:51:52.029308 ==============
INFO     =============== Health Check end time: 2019-07-23 17:56:06.896536   ==============
INFO     =============== Health Check duration: 0:04:14.867228   ==============

 

Expected output (where X.X.X.X is the IP address)

# salt '*' deployutils.get_role_ips 'cassandra'
csp-regional-sblb.KNVKBT.regional:
csp-central-k8mastervm.R82ZN0.central:
    - X.X.X.X
csp-central-infravm.R82ZN0.central:
    - X.X.X.X
csp-contrailanalytics-1.R82ZN0.central:
    - X.X.X.X
csp-central-msvm.R82ZN0.central:
    - X.X.X.X

# salt '*' deployutils.get_role_ips 'haproxy_confd'
csp-regional-sblb.KNVKBT.regional:
csp-contrailanalytics-1.R82ZN0.central:
    - X.X.X.X
csp-central-k8mastervm.R82ZN0.central:
    - X.X.X.X
csp-central-infravm.R82ZN0.central:
    - X.X.X.X
csp-central-msvm.R82ZN0.central:
    - X.X.X.X

 

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Security Alerts and Vulnerabilities

Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search