Support Support Downloads Knowledge Base Case Manager My Juniper Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[vRR] Bidirectional Forwarding Detection in Virtual Route Reflector

0

0

Article ID: KB35345 KB Last Updated: 29 Jun 2020Version: 2.0
Summary:

This article explains behavior when running Bidirectional Forwarding Detection (BFD) on a Virtual Route Reflector (vRR).

The vRR is a product with limited forwarding plane support, and is designed to be placed out of the data path in a route reflector role.

 

Symptoms:

Running BFD with aggressive (sub-second) timers on a vRR might cause BFD session to flap.

 

Cause:

Background information into Juniper's BFD implementation:

  • ppmd (periodic packet management daemon) is the daemon in charge of periodic packet generation to alleviate rpd (routing protocol daemon) from these tasks.

  • ppmd is used, for example to transmit and receive OSPF hello packets, BFD packets, LACPDUs, xSTP BPDUs, VRRP, and so on.

  • Bidirectional Forwarding Detection on Juniper Network products can run in three different modes, depending on the hardware and platform.

The ppmd daemon (or periodic packet management daemon) can run in these three modes:

  • Centralized (or non-distributed) BFD: BFD is run on the Routing Engine CPU.
  • Distributed (or single-hop) BFD: BFD is delegated to the line card (FPC) CPU micro-kernel.
  • Inline BFD: BFD runs in the ASIC (scales even further than distributed mode).

The vRR has limited data plane capabilities.

Therefore, the BFD protocol and ppmd run purely in centralized mode on the vRR. BFD and ppmd are running in user space ("RE" CPU space). Ppmd interacts with RPD and kernel through Unix sockets. There is a bottleneck with sub-second packet generation and "higher scale." This means that BFD running in control plane usually cannot offer failure detection in the sub-second range. In other words, there is no u-kernel that can be offloaded/delegated to FPC/PFE CPU, like in distributed mode.

 

Solution:

Based on the number of BGP prefixes in the RIB and RAM assigned to the vRR, the BFD timer might need to be adjusted. BFD minimum-interval might need to be adjusted to an optimal level. This optimal level depends on the load on the system (traffic and RPD current load), which would be to around ~1sec. Then BFD should be monitored to verify its stability. If it is not stable, the minimum-interval might need to be further adjusted.

The minimum-interval might differ based on the number of BGP sessions, the size of the RIB, the amount of RAM, and so on.

  • 32GB RAM for VRR is usually recommended for higher/max scale 30 million routes.
  • 16GB RAM should be sufficient for a BGP scale of 3 million routes.

Failure Scenarios when a vRR is not configured with BFD

  • When a router learns a route via eBGP (route-type external), it receives the NEXT_HOP attribute and will typically modify the NEXT_HOP attribute to its own loopback address with a “next-hop self” iBGP export policy, before re-advertising the prefixes to its iBGP neighbors (typically route reflectors).

  • The route reflector(s) will not re-write the NEXT_HOP attribute, but instead reflect these prefixes to other routers which are route reflector clients.

  • The route reflector clients will use the IGP to recursively resolve the “protocol next-hop” in the BGP route, which is the loopback address of the original router which received the eBGP prefix, and therefore come up with the interface next hop by using the shortest IGP metric to that loopback (or an LSP, if there is one in the inet.3 table).

  • The routers install the route with the real next hop in the FIB (data plane).

  • These IGP sessions are in the data path, so they are protected with BFD and ideally will failover in sub-second interval should there be a problem.

Failure scenario 1 – A vRR not running BFD goes down

  • The prefixes learned from that vRR will time out on the route reflector clients by using the iBGP protocol hold timers; default in Junos OS is 90 seconds.

  • The vRR went down, but not the destination routes, so that route is still valid. Therefore no black-holing of traffic takes place.

  • The other route reflector clients continue forwarding traffic to the "protocol next-hop", through the data plane path (which was not going through the out-of-band vRR).

  • The session with the vRR that went down times out, and because the route reflectors were redundant and using different cluster IDs and/or add-path capability, the route reflector clients have more than 1 iBGP route for that eBGP learned prefix.

Failure scenario 2 – Router A which learned the eBGP routes goes down

  • The prefixes learned from router A will time out on the vRR(s) by using the iBGP protocol hold timers; default in Junos OS is 90 seconds.

  • However, because the IGPs are running BFD to protect and detect failures in the data plane, all IGP sessions to router A go down.

  • Therefore, the loopback address of router A is removed from the IS-IS or OSPF LS databases, and the route to the loopback is removed from the RIB and FIB, within sub-second.

  • The route reflector clients still have the iBGP prefixes advertised by router A and reflected by the vRR(s), which will take the configured iBGP hold time to be withdrawn.

  • The recursive “protocol next-hop” resolution of the iBGP prefixes advertised by router A and reflected by the vRR(s) will fail, because the loopback of router A is no longer in the RIB.

  • The iBGP routes advertised by router A and reflected by the vRR(s) will become hidden, as it is not possible to resolve the “protocol next-hop”, failing over to the alternate paths. Therefore, no blackholing of traffic will take place.

​Failure scenario 3 - A vRR not running BFD goes down AND Router A which learned the eBGP routes goes down

  • No BFD was configured on the iBGP sessions between the vRR and the route reflector clients. The default BGP hold-time is 90s.

  • Suppose there are two route reflector clients, Router-A and Router-B.

  • Suppose the vRR goes down.

  • In the 90s hold-time that iBGP would take to detect that the vRR is down, a second failure occurs and Router-A goes down.

In this scenario, Router-B would have routing information originating from Router-A, reflected by vRR (which is down) but whose BGP hold-time has not expired yet. However, the routes with a "protocol next-hop" of Router-A loopback address in Router-B's RIB would become hidden because Router-A is down, and its loopback is no longer reachable via IGP (via the data path, protected with BFD). Therefore, Router-B is no longer able to recursively resolve the "protocol next-hop" in the reflected BGP routes, learned from Router-A. As a result those routes become hidden, and alternate routing information is used. Therefore no black-holing of traffic would take place.

 

Modification History:

2020-06-29: Corrected some wording and removed the part about BFD and link checks

 

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search