[MX] The CPU utilization is more than 100% after upgrading to BSDx based Junos 15.1

Summary:

This article explains why CPU utilization increases over 100% after upgrading to BSDx based Junos 15.1.

Symptoms:

After upgrading from legacy (non-BSDx) Junos to BSDx based Junos 15.1, the idle percentage increases more than 100%:

lab@MX240-RE0> show system processes extensive
last pid: 13207;  load averages:  0.21,  0.26,  0.25  up 0+23:01:49    09:36:54
141 processes: 2 running, 138 sleeping, 1 waiting
Mem: 91M Active, 1777M Inact, 1339M Wired, 1666M Buf, 28G Free
Swap: 8192M Total, 8192M Free
  PID USERNAME  THR PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND

   10 root        4 155 ki31     0K    64K CPU3    3  90.6H 398.63% idle  <-- over 100%
 3378 root        2 -26  r26   821M 25908K nanslp  0  50:20   3.17% chassisd
   11 root       30 -72    -     0K   480K WAIT    0   5:51   0.00% intr
 3389 root        1  20    0   721M 12408K select  1   3:44   0.00% ppmd
 3403 root        1  20    0   722M  7196K select  3   2:19   0.00% clksyncd
 3490 root        5  20    0   939M    98M kqread  0   2:06   0.00% rpd
**snip**

Cause:

This behavior is by design.

Solution:

Junos is based on FreeBSD, which currently have two versions.

Legacy: FREEBSD 6.1 <-- before Junos 15.1
BSDx: FREEBSD 10.1 <-- Junos 15.1 and after

The reason you see 400% value is because BSDx images have Symmetric multiprocessing (SMP) and 4 cores enabled on RE1800x4. However, SMP is not enabled on Legacy images, so there is only 1 core running. This means you will only see 100% max in legacy Junos.

Output from BSDx:

lab@MX240-RE0> start shell

% top
last pid: 13621;  load averages:  0.46,  0.39,  0.34    up 0+23:30:30  10:05:35
88 processes:  1 running, 86 sleeping, 1 zombie
CPU 0:  0.0% user,  0.0% nice,  0.8% system,  0.0% interrupt, 99.2% idle
CPU 1:  0.8% user,  0.0% nice,  0.4% system,  0.0% interrupt, 98.8% idle
CPU 2:  0.4% user,  0.0% nice,  0.4% system,  0.0% interrupt, 99.2% idle
CPU 3:  0.4% user,  0.0% nice,  1.2% system,  0.0% interrupt, 98.3% idle
Mem: 209M Active, 1824M Inact, 1381M Wired, 1648M Buf, 28G Free
Swap: 8192M Total, 8192M Free

In BSDx images, it is expected to see the Weighted CPU (WCPU) idle value reach nearly 400% with RE1800x4. Unix/Linux sums up the total 4 RE cores together when displaying the WCPU idle value.

Use the following method to verify your Junos version:

In some special versions, it may still be using legacy Junos even after 15.1. So it is recommended to double check:

Method 1

lab@MX240-RE0> start shell

% sysctl kern.osreldate
kern.osreldate: 601000  <-- start with 6, this is FreeBSD6 base (6.1)

lab@MX240-RE0> start shell

% sysctl kern.osreldate
kern.osreldate: 1001510  <-- start with 10, FreeBSD10 base.

Method 2

lab@MX240-RE0> show version | match kernel
JUNOS Kernel Software Suite [16.1X70-D10.3] <-- This is legacy output

lab@MX240-RE0> show version | match kernel
JUNOS OS Kernel 64-bit  [20160616.329709_builder_stable_10] <-- BSDx shows 32 or 64 bit

Modification History:

2018-09-26: Minor, non-technical edit.

Knowledge Base