Support Support Downloads Knowledge Base Juniper Support Portal Community

Knowledge Base

Search our Knowledge Base sites to find answers to your questions.

Ask All Knowledge Base Sites All Knowledge Base Sites JunosE Defect (KA)Knowledge BaseSecurity AdvisoriesTechnical BulletinsTechnotes Sign in to display secure content and recently viewed articles

[Contrail] How to check the load and status of each vrouter DPDK thread

0

0

Article ID: KB36062 KB Last Updated: 09 Jul 2020Version: 1.0
Summary:

This article explores useful commands to check the various running status of vrouter DPDK threads.

KB36056 explains the roles of different threads in dpdk-vrouter and shows how to use the 'ps' command to list them.

KB35517 explains how CPU pinning works with examples.

Solution:

The 'ps' command is a powerful, versatile tool that give all kinds of useful information about a running process.  With different parameters of the 'ps' command, we can check the load of each vrouter dpdk threads:

    $ ps -mo pid,tid,comm,psr,pcpu -p $(ps -ef | awk '$8=="/usr/bin/contrail-vrouter-dpdk" {print $2}')
      PID   TID COMMAND         PSR %CPU
     3638     - contrail-vroute   -  803
        -  3638 -                 0  2.2
        -  4026 -                15  0.0
        -  4027 -                23  2.8
        -  4028 -                34  0.0
        -  4029 -                21  0.0
        -  4030 -                12  0.2
        -  4031 -                 1 99.9        #<---forwarding thread
        -  4032 -                 2 99.9        #<---forwarding thread
        -  4033 -                13 99.9        #<---forwarding thread
        -  4034 -                14 99.8        #<---forwarding thread
        -  4035 -                25 99.9        #<---forwarding thread
        -  4036 -                26 99.9        #<---forwarding thread
        -  4037 -                37 99.9        #<---forwarding thread
        -  4038 -                38 99.9        #<---forwarding thread
        -  5941 -                20  0.0

The output of this command shows, the 8 forwarding threads are running with full CPU utilizations. The reason is that with DPDK vrouter design, each forwarding thread keeps polling the queue to see if any packets need to be received/sent in a indefinite loop, regardless of the actual traffic volumn. This is part of the performance boost method in the design. Other threads, instead, does not consume high CPU because they are not directly used to handle the user packets. Another useful information is the CPU pinning status. In KB35517, the 'taskset' command is used to show the same result. For example, we can tell thread 4031 is pinned to CPU#1, 4032 is pinned to CPU#2, 4033 is pinned to CPU#13, etc. These matches to the CPU assignment configuration in '/etc/contrail/supervisord_vrouter_files/contrail-vrouter-dpdk.ini'.

command=taskset -c 1,25,2,26,13,37,14,38 ...


Similar result can be displayed with the command, 'pidstat' as shown below:

    $ pidstat -t -p `pidof contrail-vrouter-dpdk`
    Linux 3.13.0-128-generic (bcomp80)      06/23/2020      _x86_64_        (48 CPU)

    02:56:41 PM   UID      TGID       TID    %usr %system  %guest    %CPU   CPU  Command
    02:56:41 PM     0      3638         -  494.60  308.70    0.00  803.30     0  contrail-vroute
    02:56:41 PM     0         -      3638    0.69    1.52    0.00    2.21     0  |__contrail-vroute
    02:56:41 PM     0         -      4026    0.01    0.01    0.00    0.02    17  |__eal-intr-thread
    02:56:41 PM     0         -      4027    0.57    2.29    0.00    2.87    19  |__lcore-slave-1
    02:56:41 PM     0         -      4028    0.00    0.00    0.00    0.00    34  |__lcore-slave-2
    02:56:41 PM     0         -      4029    0.01    0.03    0.00    0.04    17  |__lcore-slave-8
    02:56:41 PM     0         -      4030    0.11    0.11    0.00    0.22    43  |__lcore-slave-9
    02:56:41 PM     0         -      4031   62.08   37.66    0.00   99.74     1  |__lcore-slave-10
    02:56:41 PM     0         -      4032   60.74   39.00    0.00   99.74     2  |__lcore-slave-11
    02:56:41 PM     0         -      4033   61.92   37.82    0.00   99.74    13  |__lcore-slave-12
    02:56:41 PM     0         -      4034   61.91   37.83    0.00   99.73    14  |__lcore-slave-13
    02:56:41 PM     0         -      4035   61.53   38.21    0.00   99.74    25  |__lcore-slave-14
    02:56:41 PM     0         -      4036   60.79   38.95    0.00   99.74    26  |__lcore-slave-15
    02:56:41 PM     0         -      4037   61.96   37.79    0.00   99.74    37  |__lcore-slave-16
    02:56:41 PM     0         -      4038   61.37   38.38    0.00   99.74    38  |__lcore-slave-17
    02:56:41 PM     0         -      5941    0.00    0.00    0.00    0.00    20  |__lcore-slave-9

It is better than the 'ps' command at least in one aspect that the tree structure of the threads and their parent process (dpdk-vrouter) is remained, along with the threads name, so you can easily tell who is who. The drawback is that this command does not come with the standard Linux distribution and requires to be be installed before it can be used.

Comment on this article > Affected Products Browse the Knowledge Base for more articles related to these product categories. Select a category to begin.

Getting Up and Running with Junos

Getting Up and Running with Junos Security Alerts and Vulnerabilities Product Alerts and Software Release Notices Problem Report (PR) Search Tool EOL Notices and Bulletins JTAC User Guide Customer Care User Guide Pathfinder SRX High Availability Configurator SRX VPN Configurator Training Courses and Videos End User Licence Agreement Global Search