[Subscriber Management] AE subscriber interface targeting rebalance does not work

Summary:

Customers may want to add a new member link to an existing Aggregated Ethernet (AE) interface, and then want to rebalance subscribers across the new member link. However, this operation may not work sometimes.

This article gives the reason, while also detailing the configuration that ensures that subscribers are rebalanced across a new member link automatically.

Symptoms:

When you configure targeted-distribution in a dynamic-profile and apply the configuration on a subscriber interface, subscribers will be distributed to each link automatically.

dynamic-profiles {
    pppoe {
        interfaces {
            pp0 {
                unit "$junos-interface-unit" {
                    no-traps;
                    ppp-options {
                        chap;
                        pap;
                    }
                    pppoe-options {
                        underlying-interface "$junos-underlying-interface";
                        server;
                    }
                    targeted-distribution;       <<<<<< targeted-distribution configuration in dynamic-profile
                    family inet {
                        unnumbered-address lo0.0;
                    }
                }
            }
        }
    }
}

Now when you bring up a single link AE interface and connect 100 subscribers, you will find that all the subscribers are distributed on that active link.

user@mx> show lacp interfaces ae0
Aggregated interface: ae0
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/2/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/2/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/2/1       Actor    No   Yes    No   No   No   Yes     Fast    Active
      xe-0/2/1     Partner    No   Yes    No   No   No   Yes     Fast   Passive
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/2/0                  Current   Fast periodic Collecting distributing
      xe-0/2/1            Port disabled     No periodic           Detached

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100

Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 100       <<<<<< All subscribers distributed on this link
  Number of backup  distributions: 0
Physical interface: xe-0/2/1, Link status: Down
  Number of primary distributions: 0
  Number of backup  distributions: 100

When you bring up a new member link on the AE interface now, and try to rebalance subscriber distribution, the operation may not work as is shown below:

user@mx> show lacp interfaces ae0
Aggregated interface: ae0
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      xe-0/2/0       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/2/0     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/2/1       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      xe-0/2/1     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State
      xe-0/2/0                  Current   Fast periodic Collecting distributing
      xe-0/2/1                  Current   Fast periodic Collecting distributing

user@mx> request interface rebalance ae0     <<<<<< Rebalance operation
Rebalance operation on interface ae0 started

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100

Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 100        <<<<<< All subscribers are still distributed on the original link.
  Number of backup  distributions: 0
Physical interface: xe-0/2/1, Link status: Up
  Number of primary distributions: 0
  Number of backup  distributions: 100

Cause:

As per Junos OS design, if the subscriber count difference between the members of the AE interface is below 500, rebalance is not done.

A configurable knob, rebalance-subscriber-granularity, controls the aggregate client weight for each targeted pseudo IFL. This knob's default value is 500, which is the reason that rebalance works at a higher number (> 500) of subscribers.

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 1000

Distribution Group name: default
Number of distributed interfaces: 1000
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 1000
  Number of backup  distributions: 0
Physical interface: xe-0/2/1, Link status: Down
  Number of primary distributions: 0
  Number of backup  distributions: 1000

user@mx> request interface rebalance ae0
Rebalance operation on interface ae0 started

{MASTER}
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 1000

Distribution Group name: default
Number of distributed interfaces: 1000
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 500
  Number of backup  distributions: 500
Physical interface: xe-0/2/1, Link status: Up
  Number of primary distributions: 500    <<<<<<500 subscribers rebalanced to new member link
  Number of backup  distributions: 500

Solution:

When you set rebalance-subscriber-granularity to 1, the rebalance operation always take place.

user@mx# set interfaces ae0 aggregated-ether-options targeted-options rebalance-subscriber-granularity 1

{MASTER}[edit]
user@mx# show | compare
[edit interfaces ae0 aggregated-ether-options]
+     targeted-options {
+         rebalance-subscriber-granularity 1;
+     }

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100

Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 100
  Number of backup  distributions: 0
Physical interface: xe-0/2/1, Link status: Down
  Number of primary distributions: 0
  Number of backup  distributions: 100

user@mx> request interface rebalance ae0
Rebalance operation on interface ae0 started

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100

Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 50
  Number of backup  distributions: 50
Physical interface: xe-0/2/1, Link status: Up
  Number of primary distributions: 50
  Number of backup  distributions: 50

Note: Setting this knob to a low value, as shown above, can result in undesirable effects such as the system running out of pseudo IFLs, the convergence time increasing for rebalance, and so on. It is therefore NOT recommended that the value should be set to less than a few hundreds.

However, customers need not worry about subscriber distribution difference being below 500 because new subscribers will always be distributed to a new member link that has low weight.

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100

Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 100
  Number of backup  distributions: 0
Physical interface: xe-0/2/1, Link status: Down
  Number of primary distributions: 0
  Number of backup  distributions: 100

For example, when you bring up a new member link and connect 100 new subscribers, all new subscribers will distribute to the new member link.

user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 200

Distribution Group name: default
Number of distributed interfaces: 200
Physical interface: xe-0/2/0, Link status: Up
  Number of primary distributions: 100
  Number of backup  distributions: 100
Physical interface: xe-0/2/1, Link status: Up
  Number of primary distributions: 100            <<<<<< 100 new subscribers distributed to new member link
  Number of backup  distributions: 100

Knowledge Base