| John DeNisco | 06dcd45 | 2018-07-26 12:45:10 -0400 | [diff] [blame] | 1 | .. _cpuusage: |
| 2 | |
| 3 | ************** |
| 4 | CPU Load/Usage |
| 5 | ************** |
| 6 | |
| 7 | There are various commands and tools that can help users see FD.io VPP CPU and memory usage at runtime. |
| 8 | |
| 9 | Linux top/htop |
| 10 | ============== |
| 11 | |
| 12 | The Linux top and htop are decent tools to look at FD.io VPP cpu and memory usage, but they will only show |
| 13 | preallocated memory and total CPU usage. These commands can be useful to show which cores VPP is running on. |
| 14 | |
| 15 | This is an example of VPP instance that is running on cores 8 and 9. For this output type **top** and then |
| 16 | type **1** when the tool starts. |
| 17 | |
| 18 | .. code-block:: console |
| 19 | |
| 20 | $ top |
| 21 | |
| 22 | top - 11:04:04 up 35 days, 3:16, 5 users, load average: 2.33, 2.23, 2.16 |
| 23 | Tasks: 435 total, 2 running, 432 sleeping, 1 stopped, 0 zombie |
| 24 | %Cpu0 : 1.0 us, 0.7 sy, 0.0 ni, 98.0 id, 0.0 wa, 0.0 hi, 0.3 si, 0.0 st |
| 25 | %Cpu1 : 2.0 us, 0.3 sy, 0.0 ni, 97.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 26 | %Cpu2 : 0.7 us, 1.0 sy, 0.0 ni, 98.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 27 | %Cpu3 : 1.7 us, 0.7 sy, 0.0 ni, 97.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 28 | %Cpu4 : 2.0 us, 0.7 sy, 0.0 ni, 97.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 29 | %Cpu5 : 3.0 us, 0.3 sy, 0.0 ni, 96.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 30 | %Cpu6 : 2.3 us, 0.7 sy, 0.0 ni, 97.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 31 | %Cpu7 : 2.6 us, 0.3 sy, 0.0 ni, 97.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 32 | %Cpu8 : 96.0 us, 0.3 sy, 0.0 ni, 3.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 33 | %Cpu9 :100.0 us, 0.0 sy, 0.0 ni, 0.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 34 | %Cpu10 : 1.0 us, 0.3 sy, 0.0 ni, 98.7 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st |
| 35 | .... |
| 36 | |
| 37 | VPP Memory Usage |
| 38 | ================ |
| 39 | |
| 40 | For details on VPP memory usage you can use the **show memory** command |
| 41 | |
| 42 | This is the example VPP memory usage on 2 cores. |
| 43 | |
| 44 | .. code-block:: console |
| 45 | |
| 46 | # vppctl show memory verbose |
| 47 | Thread 0 vpp_main |
| 48 | 22043 objects, 17878k of 20826k used, 2426k free, 2396k reclaimed, 346k overhead, 1048572k capacity |
| 49 | alloc. from small object cache: 22875 hits 39973 attempts (57.23%) replacements 5143 |
| 50 | alloc. from free-list: 44732 attempts, 26017 hits (58.16%), 528461 considered (per-attempt 11.81) |
| 51 | alloc. from vector-expand: 3430 |
| 52 | allocs: 52324 2027.84 clocks/call |
| 53 | frees: 30280 594.38 clocks/call |
| 54 | Thread 1 vpp_wk_0 |
| 55 | 22043 objects, 17878k of 20826k used, 2427k free, 2396k reclaimed, 346k overhead, 1048572k capacity |
| 56 | alloc. from small object cache: 22881 hits 39984 attempts (57.23%) replacements 5148 |
| 57 | alloc. from free-list: 44736 attempts, 26021 hits (58.17%), 528465 considered (per-attempt 11.81) |
| 58 | alloc. from vector-expand: 3430 |
| 59 | allocs: 52335 2027.54 clocks/call |
| 60 | frees: 30291 594.36 clocks/call |
| 61 | |
| 62 | VPP CPU Load |
| 63 | ============ |
| 64 | |
| 65 | To find the VPP CPU load or how busy VPP is use the **show runtime** command. |
| 66 | |
| 67 | With at least one interface in polling mode, the VPP CPU utilization is always 100%. |
| 68 | |
| 69 | A good indicator of CPU load is **"average vectors/node"**. A bigger number means VPP |
| 70 | is more busy but also more efficient. The Maximum value is 255 (unless you change VLIB_FRAME_SIZE in code). |
| 71 | It basically means how many packets are processed in batch. |
| 72 | |
| 73 | If VPP is not loaded it will likely poll so fast that it will just get one or few |
| 74 | packets from the rx queue. This is the case shown below on Thread 1. As load goes up vpp |
| 75 | will have more work to do, so it will poll less frequently, and that will result in more |
| 76 | packets waiting in rx queue. More packets will result in more efficient execution of the |
| 77 | code so number of clock cycles / packet will go down. When "average vectors/node" goes up |
| 78 | close to 255, you will likely start observing rx queue tail drops. |
| 79 | |
| 80 | .. code-block:: console |
| 81 | |
| 82 | # vppctl show run |
| 83 | Thread 0 vpp_main (lcore 8) |
| 84 | Time 6152.9, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00 |
| 85 | vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0 |
| 86 | Name State Calls Vectors Suspends Clocks Vectors/Call |
| 87 | acl-plugin-fa-cleaner-process event wait 0 0 1 3.66e4 0.00 |
| 88 | admin-up-down-process event wait 0 0 1 2.54e3 0.00 |
| 89 | .... |
| 90 | --------------- |
| 91 | Thread 1 vpp_wk_0 (lcore 9) |
| 92 | Time 6152.9, average vectors/node 1.00, last 128 main loops 0.00 per node 0.00 |
| 93 | vector rates in 1.3073e2, out 1.3073e2, drop 6.5009e-4, punt 0.0000e0 |
| 94 | Name State Calls Vectors Suspends Clocks Vectors/Call |
| 95 | TenGigabitEthernet86/0/0-outpu active 804395 804395 0 6.17e2 1.00 |
| 96 | TenGigabitEthernet86/0/0-tx active 804395 804395 0 7.29e2 1.00 |
| 97 | arp-input active 2 2 0 3.82e4 1.00 |
| 98 | dpdk-input polling 24239296364 804398 0 1.59e7 0.00 |
| 99 | error-drop active 4 4 0 4.65e3 1.00 |
| 100 | ethernet-input active 2 2 0 1.08e4 1.00 |
| 101 | interface-output active 1 1 0 3.78e3 1.00 |
| 102 | ip4-glean active 1 1 0 6.98e4 1.00 |
| 103 | ip4-icmp-echo-request active 804394 804394 0 5.02e2 1.00 |
| 104 | ip4-icmp-input active 804394 804394 0 4.63e2 1.00 |
| 105 | ip4-input-no-checksum active 804394 804394 0 8.51e2 1.00 |
| 106 | ip4-load-balance active 804394 804394 0 5.46e2 1.00 |
| 107 | ip4-local active 804394 804394 0 5.79e2 1.00 |
| 108 | ip4-lookup active 804394 804394 0 5.71e2 1.00 |
| 109 | ip4-rewrite active 804393 804393 0 5.69e2 1.00 |
| 110 | ip6-input active 2 2 0 5.72e3 1.00 |
| 111 | ip6-not-enabled active 2 2 0 1.56e4 1.00 |
| 112 | unix-epoll-input polling 835722 0 0 3.03e-3 0.00 |