Support Support Downloads Knowledge Base Juniper Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Contrail] The 'rabbitmqctl list_queues' command times out

0

0

Article ID: KB36066 KB Last Updated: 09 Jul 2020Version: 1.0
Summary:

In the Contrail Cloud environment, it is observed that the 'rabbitmqctl list_queues' command always get stuck at following last line:

    [root@overcloud-cc-0 ~]# docker exec -it contrail_config_rabbitmq rabbitmqctl list_queues

    Timeout: 60.0 seconds ...
    Listing queues for vhost / ...
    name    messages
    ......
    {:badrpc, {:timeout, 60.0, "Some queue(s) are unresponsive, use list_unresponsive_queues command."}} <---
Cause:

Since RabbitMQ release 3.7.4, the rabbitmq CLI tools parallelize the querying of information from cluster members. In order to receive stream data back, the CLI instance binds and registers itself on an available port (default between 35672 and 35682, inclusive).  If these ports are firewalled off, then rabbitmqctl commands such as 'list_queues' will hang waiting for data from remote cluster members. That is why "timeout" is seen in the above example. The issue is fixed in higher releases.

Solution:

This issue is seen in Contrail Cloud 13.2.1 with Contrail Networking 1912.  The rabbitmq version used in this release is 3.7.20, which can be verified with the following command:

    root@overcloudsd9-cc-2:/#  rabbitmqctl status
    Status of node contrail@overclouds-cc-2 ...
    [{pid,581},
    {running_applications,
        [{rabbitmq_management,"RabbitMQ Management Console","3.7.20"},
        {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.7.20"},
        {amqp_client,"RabbitMQ AMQP Client","3.7.20"},
        {rabbitmq_management_agent,"RabbitMQ Management Agent","3.7.20"},
        {rabbit,"RabbitMQ","3.7.20"},
        {rabbit_common,
            "Modules shared by rabbitmq-server and rabbitmq-erlang-client",
            "3.7.20"},
        {cowboy,"Small, fast, modern HTTP server.","2.6.1"},

A workaround is to ensure ports 35672 - 35682 are open in all three controllers. Enforce this with the 'iptables' command:

    iptables -I INPUT 1 -p tcp --match multiport --dports 35672:35682 -j ACCEPT
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search