Customers may want to add a new member link to an existing Aggregated Ethernet (AE) interface, and then want to rebalance subscribers across the new member link. However, this operation may not work sometimes.
This article gives the reason, while also detailing the configuration that ensures that subscribers are rebalanced across a new member link automatically.
When you configure targeted-distribution in a dynamic-profile and apply the configuration on a subscriber interface, subscribers will be distributed to each link automatically.
dynamic-profiles {
pppoe {
interfaces {
pp0 {
unit "$junos-interface-unit" {
no-traps;
ppp-options {
chap;
pap;
}
pppoe-options {
underlying-interface "$junos-underlying-interface";
server;
}
targeted-distribution; <<<<<< targeted-distribution configuration in dynamic-profile
family inet {
unnumbered-address lo0.0;
}
}
}
}
}
}
Now when you bring up a single link AE interface and connect 100 subscribers, you will find that all the subscribers are distributed on that active link.
user@mx> show lacp interfaces ae0
Aggregated interface: ae0
LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
xe-0/2/0 Actor No No Yes Yes Yes Yes Fast Active
xe-0/2/0 Partner No No Yes Yes Yes Yes Fast Active
xe-0/2/1 Actor No Yes No No No Yes Fast Active
xe-0/2/1 Partner No Yes No No No Yes Fast Passive
LACP protocol: Receive State Transmit State Mux State
xe-0/2/0 Current Fast periodic Collecting distributing
xe-0/2/1 Port disabled No periodic Detached
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100
Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 100 <<<<<< All subscribers distributed on this link
Number of backup distributions: 0
Physical interface: xe-0/2/1, Link status: Down
Number of primary distributions: 0
Number of backup distributions: 100
When you bring up a new member link on the AE interface now, and try to rebalance subscriber distribution, the operation may not work as is shown below:
user@mx> show lacp interfaces ae0
Aggregated interface: ae0
LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
xe-0/2/0 Actor No No Yes Yes Yes Yes Fast Active
xe-0/2/0 Partner No No Yes Yes Yes Yes Fast Active
xe-0/2/1 Actor No No Yes Yes Yes Yes Fast Active
xe-0/2/1 Partner No No Yes Yes Yes Yes Fast Active
LACP protocol: Receive State Transmit State Mux State
xe-0/2/0 Current Fast periodic Collecting distributing
xe-0/2/1 Current Fast periodic Collecting distributing
user@mx> request interface rebalance ae0 <<<<<< Rebalance operation
Rebalance operation on interface ae0 started
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100
Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 100 <<<<<< All subscribers are still distributed on the original link.
Number of backup distributions: 0
Physical interface: xe-0/2/1, Link status: Up
Number of primary distributions: 0
Number of backup distributions: 100
As per Junos OS design, if the subscriber count difference between the members of the AE interface is below 500, rebalance is not done.
A configurable knob, rebalance-subscriber-granularity
, controls the aggregate client weight for each targeted pseudo IFL. This knob's default value is 500, which is the reason that rebalance works at a higher number (> 500) of subscribers.
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 1000
Distribution Group name: default
Number of distributed interfaces: 1000
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 1000
Number of backup distributions: 0
Physical interface: xe-0/2/1, Link status: Down
Number of primary distributions: 0
Number of backup distributions: 1000
user@mx> request interface rebalance ae0
Rebalance operation on interface ae0 started
{MASTER}
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 1000
Distribution Group name: default
Number of distributed interfaces: 1000
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 500
Number of backup distributions: 500
Physical interface: xe-0/2/1, Link status: Up
Number of primary distributions: 500 <<<<<<500 subscribers rebalanced to new member link
Number of backup distributions: 500
When you set rebalance-subscriber-granularity
to 1, the rebalance operation always take place.
user@mx# set interfaces ae0 aggregated-ether-options targeted-options rebalance-subscriber-granularity 1
{MASTER}[edit]
user@mx# show | compare
[edit interfaces ae0 aggregated-ether-options]
+ targeted-options {
+ rebalance-subscriber-granularity 1;
+ }
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100
Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 100
Number of backup distributions: 0
Physical interface: xe-0/2/1, Link status: Down
Number of primary distributions: 0
Number of backup distributions: 100
user@mx> request interface rebalance ae0
Rebalance operation on interface ae0 started
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100
Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 50
Number of backup distributions: 50
Physical interface: xe-0/2/1, Link status: Up
Number of primary distributions: 50
Number of backup distributions: 50
Note: Setting this knob to a low value, as shown above, can result in undesirable effects such as the system running out of pseudo IFLs, the convergence time increasing for rebalance, and so on. It is therefore NOT recommended that the value should be set to less than a few hundreds.
However, customers need not worry about subscriber distribution difference being below 500 because new subscribers will always be distributed to a new member link that has low weight.
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 100
Distribution Group name: default
Number of distributed interfaces: 100
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 100
Number of backup distributions: 0
Physical interface: xe-0/2/1, Link status: Down
Number of primary distributions: 0
Number of backup distributions: 100
For example, when you bring up a new member link and connect 100 new subscribers, all new subscribers will distribute to the new member link.
user@mx> show interfaces targeting ae0
Aggregated interface: ae0
Targeting Type: Auto
Redundancy mode: Link Level Redundancy
Total number of distribution groups: 1
Total number of distributed interfaces: 200
Distribution Group name: default
Number of distributed interfaces: 200
Physical interface: xe-0/2/0, Link status: Up
Number of primary distributions: 100
Number of backup distributions: 100
Physical interface: xe-0/2/1, Link status: Up
Number of primary distributions: 100 <<<<<< 100 new subscribers distributed to new member link
Number of backup distributions: 100