Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Junos Space] jingest service fails to start after Log Collector upgrade from Junos OS 17.1R2

0

0

Article ID: KB35149 KB Last Updated: 30 Mar 2020Version: 2.0
Summary:

This article explains why the jingest service may fail to start after upgrading Log Collector from Junos OS release 17.1R2 to Junos OS release 17.2R1/17.2R2/18.1R1, and what must be done to resolve the issue.

 

Symptoms:

When users try to upgrade Log Collector from Junos OS release 17.1R2 to Junos OS release 17.2R1/17.2R2/18.1R1, the upgrade finishes with the following errors:

Contacting elasticsearch cluster 'log-collector' and wait for YELLOW clusterstate ...
ERR: Timed out while waiting for a green or yellow cluster state. 
* Try running sgadmin.sh with -icl (but no -cl) and -nhnv (If thats works you need to check your clustername as well as hostnames in your SSL certificates)
* Make also sure that your keystore or cert is a client certificate (not a node certificate) and configured properly in elasticsearch.yml
* If this is not working, try running sgadmin.sh with --diagnose and see diagnose trace log file)
* Add --accept-red-cluster to allow sgadmin to operate on a red cluster. 

Error: Error in loading ssl configuration, Please look for missing configurations and then execute below script
(Note: If you have configured multinode setup, then run below script in Receiver node)
sh /opt/jnpr/bin/ssl_configuration.sh 

log-collector.log shows an error similar to the following:

[2019-07-26T10:48:27,422][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)
[2019-07-26T10:48:27,424][ERROR][c.f.s.a.BackendRegistry ] Not yet initialized (you may need to run sgadmin)
[2019-07-26T10:48:27,535][ERROR][c.f.s.h.SearchGuardHttpServerTransport] [nGQOcK5] SSL Problem Received fatal alert: unknown_ca
javax.net.ssl.SSLException: Received fatal alert: unknown_ca 
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) ~[?:?] 
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?] 
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634) ~[?:?] 
at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800) ~[?:?] 
at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083) ~[?:?] 
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907) ~[?:?] 
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781) ~[?:?] 
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624) ~[?:1.8.0-jdk8u132-b00] 

jingest shows the following status:

[root@LOG-COLLECTOR]# service jingest status
jingest is not running

While trying to start the jingest service, the following message is displayed:

[root@LOG-COLLECTOR]# /etc/init.d/jingest restart
Killing jingest (pid 6359) with SIGTERM
Waiting jingest (pid 6359) to die...
Waiting jingest (pid 6359) to die...
jingest stopped.
Error:
######
Elastic Search is not active
Collector will not be active until you configure Indexer properly.

 

Cause:

This issue is usually observed if there are any red indices in the setup before upgrade, which are caused due to any unparsed logs. 

 

Solution:

Perform the following steps to resolve the issue:

  1. Rename log-collector to log-collector_old by using mv /var/lib/elasticsearch/log-collector /var/lib/elasticsearch/log-collector_old.
  2. Run the following script: sh /opt/jnpr/bin/ssl_configuration.sh.

  3. Check the status of the Elasticsearch and jingest services:
/etc/init.d/jingest status
/etc/init.d/elasticsearch status
  1. To move the contents back, use the following command: mv /var/lib/elasticsearch/log-collector_old/nodes/0/indices/* /var/lib/elasticsearch/log-collector/nodes/0/indices/.

The red indices will turn green in a few hours depending on the size of the indices following which the Log Collector will be UP and functional.

 

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search