Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Contrail] Contrail-schema restarts every 15 minutes, causing high RabbitMQ message queues

1

0

Article ID: KB34225 KB Last Updated: 07 Sep 2019Version: 2.0
Summary:

This article covers a specific scenario in an environment with an increasing number of Access Control Lists (ACL) in version 3.2.9. It provides the root cause and solution for such scenarios.

Symptoms:

The queues in RabbitMQ are high, and the contrail-schema status reports 'timeout':

The schema-zk.log indicates the timeout as shown below:

# grep -5 "Zookeeper connection lost" schema-zk.log
04/18/2019 07:17:25 PM [schema]: Connection dropped: socket connection broken
04/18/2019 07:17:25 PM [schema]: Transition to CONNECTING
04/18/2019 07:17:25 PM [schema]: Zookeeper connection lost

Cause:

Due to an increasing number of ACLs in the environment, Zookeeper may exhibit a timeout behavior because the schema process can take longer time to complete the transactions.

Solution:

This issue is fixed in version 3.2.14.

Work-around:

The following work-around can be added in version 3.2.9. This is not applicable to Contrail version 4.x and above.

Note: In a multi-node setup, the changes need to be added to all config nodes.
 

A. Zookeeper timeout value needs to be added in schema-transformer:
Conf file: /etc/contrail/contrail-schema.conf
Section: DEFAULTS
Variable: zk_timeout=4000


 

B. The max session value needs to be increased to 4000000 in the Zookeeper configuration file:
Conf file: /etc/zookeeper/conf/zoo.cfg
Variable: maxSessionTimeout=4000000


 

C. The services need to be restarted to reflect the above changes:

service contrail-schema restart
service contrail-schema status
service zookeeper restart
service zookeeper status


Modification History:
2019-09-07: Minor, non-technical edit.
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search