Support Support Downloads Knowledge Base Service Request Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

QFX5000 is unable to send BGP notification messages to peer device after rebooting

0

0

Article ID: KB34306 KB Last Updated: 04 Jun 2019Version: 1.0
Summary:

After rebooting a QFX5000 device, it is unable to send BGP notification messages to the peer device.

Symptoms:

Topology:

QFX5100-r019 (xe-0/1/2, 192.168.20.1/30) --- (xe-0/0/0:0, 192.168.20.2/30 ) QFX5100-r030

Setting eBGP and iBGP between two QFX5100s, eBGP uses physical interfaces and iBGP uses logical interfaces (Loopback interface).
Both lo0 routes will be exported to each other via eBGP.

root@QFX5100-r019# show protocols bgp | display set 
set protocols bgp traceoptions file bgp.txt
set protocols bgp traceoptions flag all
set protocols bgp group ibgp type internal
set protocols bgp group ibgp local-address 192.168.10.11
set protocols bgp group ibgp family evpn signaling
set protocols bgp group ibgp multipath
set protocols bgp group ibgp neighbor 192.168.10.1 peer-as 64512
set protocols bgp group ebgp type external
set protocols bgp group ebgp family inet unicast
set protocols bgp group ebgp export export-bgp
set protocols bgp group ebgp local-as 65101
set protocols bgp group ebgp multipath multiple-as
set protocols bgp group ebgp neighbor 192.168.20.2 peer-as 65201

root@QFX5100-r019# show interfaces lo0 | display set 
set interfaces lo0 unit 0 family inet address 192.168.10.11/32

[master:0][edit]
root@QFX5100-r019# show policy-options | display set 
set policy-options policy-statement export-bgp term 1 from protocol direct
set policy-options policy-statement export-bgp term 1 from protocol static
set policy-options policy-statement export-bgp term 1 from protocol local
set policy-options policy-statement export-bgp term 1 from interface lo0.0
set policy-options policy-statement export-bgp term 1 then accept

**************

root@QFX5100-r030# show protocols bgp | display set 
set protocols bgp traceoptions file bgp.txt
set protocols bgp traceoptions flag all
set protocols bgp group ibgp type internal
set protocols bgp group ibgp local-address 192.168.10.1
set protocols bgp group ibgp family evpn signaling
set protocols bgp group ibgp multipath
set protocols bgp group ibgp neighbor 192.168.10.11 peer-as 64512
set protocols bgp group ebgp type external
set protocols bgp group ebgp family inet unicast
set protocols bgp group ebgp export export-bgp
set protocols bgp group ebgp local-as 65201
set protocols bgp group ebgp multipath multiple-as
set protocols bgp group ebgp neighbor 192.168.20.1 peer-as 65101

root@QFX5100-r030# show interfaces lo0 | display set 
set interfaces lo0 unit 0 family inet address 192.168.10.1/32

[master:0][edit]
root@QFX5100-r030# show policy-options | display set 
set policy-options policy-statement export-bgp term 1 from protocol direct
set policy-options policy-statement export-bgp term 1 from protocol local
set policy-options policy-statement export-bgp term 1 from interface lo0.0
set policy-options policy-statement export-bgp term 1 then accept

When rebooting r019 via the command 'request system reboot', then checking the BGP status on r030, IBGP (192.168.10.11) will not change to down immediately.
 

root@QFX5100-r030# run show bgp summary                
Groups: 2 Peers: 2 Down peers: 1
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0               
                       0          0          0          0          0          0
bgp.evpn.0           
                       0          0          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.10.11         64512          6          6       0       0        2:57 Establ
  bgp.evpn.0: 0/0/0/0
192.168.20.1          65101         18         17       0       2        1:07 Idle  

[master:0][edit]
root@QFX5100-r030# run show bgp summary    
Groups: 2 Peers: 2 Down peers: 2
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0               
                       0          0          0          0          0          0
bgp.evpn.0           
                       0          0          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
192.168.10.11         64512          0          0       0       1          49 Connect
192.168.20.1          65101         18         17       0       2        2:14 Idle  

[master:0][edit]
root@QFX5100-r030# run show bgp neighbor 
Peer: 192.168.10.11+179 AS 64512 Local: 192.168.10.1 AS 64512
  Group: ibgp                  Routing-Instance: master
  Forwarding routing-instance: master  
  Type: Internal    State: Connect        Flags: <>
  Last State: Active        Last Event: ConnectRetry
  Last Error: Hold Timer Expired Error
  Options: <Preference LocalAddress AddressFamily PeerAS Multipath Rib-group Refresh>
  Address families configured: evpn
  Local Address: 192.168.10.1 Holdtime: 90 Preference: 170
  Number of flaps: 1
  Last flap event: HoldTime   <-- it is not RecvNotify
  Error: 'Hold Timer Expired Error' Sent: 1 Recv: 0
  Trace options:  all
  Trace file: /var/log/bgp.txt size 131072 files 10

Peer: 192.168.20.1 AS 65101    Local: 192.168.20.2 AS 65201
  Group: ebgp                  Routing-Instance: master
  Forwarding routing-instance: master  
  Type: External    State: Idle           Flags: <PeerInterfaceError>
  Last State: Established   Last Event: Stop
  Last Error: Cease
  Export: [ export-bgp ] 
  Options: <Preference AddressFamily PeerAS Multipath LocalAS Refresh>
  Options: <MultipathAs>
  Address families configured: inet-unicast
  Holdtime: 90 Preference: 170 Local AS: 65201 Local System AS: 64512
  Number of flaps: 2
  Last flap event: Stop   <-- it is not RecvNotify
  Error: 'Cease' Sent: 2 Recv: 0
  Trace options:  all
  Trace file: /var/log/bgp.txt size 131072 files 10

The same symptom is also seen on QFX110 and QFX5200.
Cause:

This is a limitation because there is no hypervisor on QFX5000 PFE, so PFE will go down immediately.

From the following messages, the physical interface is down before sending notifications. Thus, notification messages will not be sent to the peer.

Mar  1 06:23:16  QFX5100-r019 fpc0  fxpc_check_signal: Going to shutdown network ports
Mar  1 06:23:16  QFX5100-r019 fpc0 ifp xe-0/1/2 ifd_mdown
Mar  1 06:23:21  QFX5100-r019 rpd[2053]: bgp_peer_delete:9410: NOTIFICATION sent to 192.168.10.1 (Internal AS 64512): code 6 (Cease) subcode 3 (Peer Unconfigured), Reason: Peer Deletion
Mar  1 06:23:21  QFX5100-r019 rpd[2053]: bgp_peer_delete:9410: NOTIFICATION sent to 192.168.20.2 (External AS 65201): code 6 (Cease) subcode 3 (Peer Unconfigured), Reason: Peer Deletion
Solution:

The Bidirectional Forwarding Detection (BFD) protocol is a simple hello mechanism that detects failures in a network。

Please add BFD, which will let all BGP sessions go down immediately when the peer device reboots.

Related Links

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Security Alerts and Vulnerabilities

Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search