Support Support Downloads Knowledge Base Juniper Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Contrail] How to understand high CPU consumed by DPDK binary

0

0

Article ID: KB33831 KB Last Updated: 13 Feb 2019Version: 1.0
Summary:

On a typical DPDK compute node, you may notice "high" CPU utilization when checking DPDK process as shown in the following example:

root@bcomp80:~# ps aux|grep dpdk
root     14550  804  0.0 55518320 18844 ?      Sl    2018 464540:56 /usr/bin/contrail-vrouter-dpdk --no-daemon --vdev eth_bond_bond0,mode=4,xmit_policy=l34,socket_id=1,mac=14:02:ec:66:b8:dc,slave=0000:87:00.0,slave=0000:09:00.1 --vlan_tci 722 --vlan_fwd_intf_name bond0 --socket-mem 1024 1024
root     38458  0.0  0.0  11740   932 pts/20   S+   13:11   0:00 grep --color=auto dpdk​

root@bcomp80:~# top -n1 -p 14550
top - 13:14:58 up 63 days, 20:36,  2 users,  load average: 9.79, 9.66, 9.55
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s): 13.0 us,  8.1 sy,  0.0 ni, 79.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  19770470+total, 11202083+used, 85683872 free,   174204 buffers
KiB Swap: 15626236 total,  2954936 used, 12671300 free.  1387732 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                                                                             
14550 root      20   0 52.946g  18844   1932 S 803.4  0.0 464559:57 contrail-vroute

This is working as designed because all 8 CPU cores dedicated to DPDK binary are constantly polling for packets. Therefore, they are displayed as 100% busy in the "top" output.

Symptoms:
root@bcomp80:~# top -n1 -p <dpdk tid>
top - 13:14:58 up 63 days, 20:36,  2 users,  load average: 9.79, 9.66, 9.55
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
%Cpu(s): 13.0 us,  8.1 sy,  0.0 ni, 79.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  19770470+total, 11202083+used, 85683872 free,   174204 buffers
KiB Swap: 15626236 total,  2954936 used, 12671300 free.  1387732 cached Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
14550 root      20   0 52.946g  18844   1932 S 803.4  0.0 464559:57 contrail-vroute
Solution:

Looking at the top display in thread mode, i.e., top -H, there are 8 CPU threads dedicated for DPDK binary, which consumes nearly 100% CPU individually.  Therefore, 8 CPU threads contribute to the 800% CPU utilization.

​root@bcomp80:~# top -n1 -H -p <dpdk tid>
top - 13:16:06 up 63 days, 20:37,  2 users,  load average: 9.75, 9.71, 9.58
Threads:  15 total,   8 running,   7 sleeping,   0 stopped,   0 zombie
%Cpu(s): 13.0 us,  8.1 sy,  0.0 ni, 79.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  19770470+total, 11202089+used, 85683808 free,   174204 buffers
KiB Swap: 15626236 total,  2954936 used, 12671300 free.  1387748 cached Mem
  PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
14733 root      20   0 52.946g  18844   1932 R 99.9  0.0  57687:01 lcore-slave-12
14738 root      20   0 52.946g  18844   1932 R 99.9  0.0  57687:31 lcore-slave-17
14731 root      20   0 52.946g  18844   1932 R 99.3  0.0  57687:09 lcore-slave-10
14732 root      20   0 52.946g  18844   1932 R 99.3  0.0  57687:15 lcore-slave-11
14734 root      20   0 52.946g  18844   1932 R 99.3  0.0  57687:17 lcore-slave-13
14735 root      20   0 52.946g  18844   1932 R 99.3  0.0  57687:33 lcore-slave-14
14736 root      20   0 52.946g  18844   1932 R 99.3  0.0  57687:36 lcore-slave-15
14737 root      20   0 52.946g  18844   1932 R 92.7  0.0  57687:30 lcore-slave-16
14727 root      20   0 52.946g  18844   1932 S  6.6  0.0   1670:59 lcore-slave-1
14550 root      20   0 52.946g  18844   1932 S  0.0  0.0   1198:50 contrail-vroute
14726 root      20   0 52.946g  18844   1932 S  0.0  0.0  13:01.47 eal-intr-thread
14728 root      20   0 52.946g  18844   1932 S  0.0  0.0   0:00.11 lcore-slave-2
14729 root      20   0 52.946g  18844   1932 S  0.0  0.0  21:28.30 lcore-slave-8
14730 root      20   0 52.946g  18844   1932 S  0.0  0.0 161:49.57 lcore-slave-9 
18924 root      20   0 52.946g  18844   1932 S  0.0  0.0   3:55.42 lcore-slave-9 

The following excerpt is from the document, Configuring the Data Plane Development Kit (DPDK) Integrated with Contrail vRouter.

coremask—Specify a CPU affinity mask with which vRouter will run. vRouter will use only the CPUs specified for its threads of execution. These CPU cores will be constantly polling for packets, and they will be displayed as 100% busy in the output of “top”.

In this example, because our DPDK compute node deployment uses default coremask value as 0xf, which corresponds to CPU core 0 to 7, all 8 CPU cores will then contribute to 800% CPU in top output.

env.dpdk = {
    host1: { 'huge_pages' : '50', 'coremask' : '0xf'},
    host2: { 'huge_pages' : '50', 'coremask' : '0xf'},
}
Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search