Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Contrail] Zookeeper Session ID increments after reconnection

0

0

Article ID: KB35219 KB Last Updated: 16 Nov 2019Version: 1.0
Summary:

This article demonstrates how session ID increases after Zookeeper client reconnects. 

Solution:

Observe how a znode changes after Zookeper node communication fails.

CAUTION: Do NOT use this process on a production system.

Initially, svc-monitor is running with pid 5060:

# contrail-status -d | grep svc
contrail-svc-monitor        active       pid 5060, uptime 0:05:20  

Also, the related znode in zookeeper has session ID 0x16e18e9152f000c:

# /usr/share/zookeeper/bin/zkCli.sh
<snipped>
[zk: localhost:2181(CONNECTED) 1] ls /svc-monitor
[71805c1e12da4aa29c15558b4ba332ba__lock__0000000003]

[zk: localhost:2181(CONNECTED) 11] get /svc-monitor/71805c1e12da4aa29c15558b4ba332ba__lock__0000000003
5060
cZxid = 0x401
ctime = Tue Oct 29 12:13:53 PDT 2019
mZxid = 0x401
mtime = Tue Oct 29 12:13:53 PDT 2019
pZxid = 0x401
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x16e18e9152f000c
dataLength = 4
numChildren = 0

Using local firewall, blocking port 2181 to BREAK the connection on Contrail node running svc-monitor until the connection is down.

# iptables -A OUTPUT -p tcp --dport 2181 -j DROP
# netstat -alnp | grep 5060 | grep 2181
tcp        0     12 10.176.0.250:41068      10.176.0.250:2181       ESTABLISHED 5060/python
# netstat -alnp | grep 5060 | grep 2181

Now, unblock port 2181 again:

# iptables -D OUTPUT -p tcp --dport 2181 -j DROP

When checking znode again with command '/usr/share/zookeeper/bin/zkCli.sh', the old znode is gone. Instead, a new znode with sequence number and session ID increased. The sequence number increased from 0000000003 to 0000000004, and session ID increased from 0x16e18e9152f000c to 0x16e18e9152f0028.

[zk: localhost:2181(CONNECTED) 13] get /svc-monitor/71805c1e12da4aa29c15558b4ba332ba__lock__0000000003
Node does not exist: /svc-monitor/71805c1e12da4aa29c15558b4ba332ba__lock__0000000003

[zk: localhost:2181(CONNECTED) 14] ls /svc-monitor
[2471d34cf45d4061ab05c2860ff3138b__lock__0000000004]
[zk: localhost:2181(CONNECTED) 17] get /svc-monitor/2471d34cf45d4061ab05c2860ff3138b__lock__0000000004
18621
cZxid = 0x524
ctime = Tue Oct 29 12:27:34 PDT 2019
mZxid = 0x524
mtime = Tue Oct 29 12:27:34 PDT 2019
pZxid = 0x524
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x16e18e9152f0028
dataLength = 5
numChildren = 0

For more on troubleshooting Zookeeper, refer to KB31144 - Contrail Getting Started - Administration, Configuration & Troubleshooting (JumpStation)

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search