This article explains why high CPU utilization may be seen in the Juniper Linux Guest OS for the lcore (SRX PFE) process, and indicates whether any action is necessary.
When performance monitoring tools such as top -H, which track CPU usage, are used for Juniper Linux of vSRX, SRX1500, SRX4100, SRX4200, or SRX4600, it is observed that the srxpfe or lcore-slave processes are consuming a high percentage of the CPU.
For vSRX systems, the srxpfe processes are associated with vCPUs, thereby impacting monitoring reporting of the vCPU and the associated physical CPU core.
To view the process utilization, log in to Juniper Linux from the Junos shell and run the following ssh and top commands. The root username and password must be used to log in to Juniper Linux.
ssh -JU __juniper_private4__ root@192.168.1.1
root@localhost:~# top -H
top - 16:59:47 up 197 days, 1:53, 1 user, load average: 1.89, 2.08, 2.04
Tasks: 141 total, 6 running, 135 sleeping, 0 stopped, 0 zombie
Cpu(s): 59.2%us, 2.9%sy, 0.0%ni, 38.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 4047688k total, 4002976k used, 44712k free, 299996k buffers
Swap: 15620k total, 0k used, 15620k free, 389328k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3673 root 20 0 3868m 255m 153m R 100 6.5 14741:35 srxpfe
3604 root 20 0 3868m 255m 153m S 11 6.5 679:01.48 srxpfe
3214 root 20 0 7943m 245m 101m R 98 3.1 11200:47 lcore-slave-1
3455 root 20 0 2114m 986m 3852 S 6 25.0 10286:51 kvm
3450 root 20 0 2114m 986m 3852 R 4 25.0 11610:44 kvm
High CPU usage of up to 100% is due to the use of the poll mode driver by the flow processes srxpfe and lcore-slave that are running in a loop, polling for packets from the NIC/vNIC for data plane processing.
The high CPU is from the Juniper Linux and the underlying physical host of vSRX systems, and it is not for the data plane flow process itself.
The lcore process is used by the Data Plane Development Kit (DPDK) mechanism in vSRX and consists of libraries to accelerate packet processing workloads.
DPDK is supposed to poll certain ports for incoming packets in an infinite loop, pinned to a certain logical core. Due to this kind of a polling model, the Linux OS faces a challenge while measuring effective core utilization. As a result, the CPU time on the core where the polling loop is running is always close to 100%. This is expected behavior.
For the vSRX-S flavor, this process has a nano-sleep mechanism when there is no traffic. Therefore, the effects on the S flavor may not be significant.
This is expected behavior by design.
Although the data plane CPU level will reflect as being high on the monitoring tools and on Juniper Linux, data collected by the Junos CLI commands reflects the data plane CPU processing based on traffic processing and will not reflect the same high CPU utilization due to the poll mode driver behavior.
Junos commands collect CPU utilization from the data plane, as shown with the following commands:
root@vSRX> show security monitoring performance spu
fpc 0 pic 0
Last 60 seconds:
0: 0 1: 0 2: 0 3: 0 4: 0 5: 0
6: 0 7: 0 8: 0 9: 0 10: 0 11: 0
12: 0 13: 0 14: 0 15: 0 16: 0 17: 0
18: 0 19: 0 20: 0 21: 0 22: 0 23: 0
24: 0 25: 0 26: 0 27: 0 28: 0 29: 0
30: 0 31: 0 32: 0 33: 0 34: 0 35: 0
36: 0 37: 0 38: 0 39: 0 40: 0 41: 0
42: 0 43: 0 44: 0 45: 0 46: 0 47: 0
48: 0 49: 0 50: 0 51: 0 52: 0 53: 0
54: 0 55: 0 56: 0 57: 0 58: 0 59: 0
root@vSRX> show security monitoring fpc 0
FPC 0
PIC 0
CPU utilization : 0 %
Memory utilization : 40 %
Current flow session : 0
Current flow session IPv4: 0
Current flow session IPv6: 0
Max flow session : 1048576
Total Session Creation Per Second (for last 96 seconds on average): 0
IPv4 Session Creation Per Second (for last 96 seconds on average): 0
IPv6 Session Creation Per Second (for last 96 seconds on average): 0
2020-09-08: Merged content from another KB; article relevant and valid