Gitiles
Code Review Sign In
gerrit.nordix.org / fdio / vpp / ee94ffa35d69a56df894a31147a106f2a7d4c3de / . / docs / usecases / simpleperf / iperf3.rst
blob: 1d159d0eef262786afc7e92a693b64b8ac1328b9 [file] [log] [blame]
jdenisco1511a4e2018-10-30 08:46:02 -0400[diff] [blame]1.. _iperf3:
2
3Introduction
4============
5
6This tutorial shows how to use VPP use iperf3 and Trex to get some basic peformance
7numbers from a few basic configurations. Four examples are shown. In the first two
8examples, the **iperf3** tool is used to generate traffic, and in the last two examples
9the Cisco's `TRex Realistic Traffic Generator <http://trex-tgn.cisco.com/>`_ is used. For
10comparison purposes, the first example shows packet forwarding using ordinary kernel
11IP forwarding, and the second example shows packet forwarding using VPP.
12
13Three Intel Xeon processor platform systems are used to connect to the VPP host to pass traffic
14using **iperf3** and Cisco’s `TRex <http://trex-tgn.cisco.com/>`_.
15
16Intel 40 Gigabit Ethernet (GbE) network interface cards (NICs) are used to connect the hosts.
17
18
19Using Kernel Packet Forwarding with Iperf3
20===========================================
21
22In this test, 40 GbE Intel Ethernet Network Adapters are used to connect the three
23systems. Figure 1 illustrates this configuration.
24
25.. figure:: /_images/iperf3fig1.png
26
27Figure 1: VPP runs on a host that connects to two other systems via 40 GbE NICs.
28
29For comparison purposes, in the first example, we configure kernel forwarding in
30*csp2s22c03* and use the **iperf3** tool to measure network bandwidth between
31*csp2s22c03* and *net2s22c05*.
32
33In the second example, we start the VPP engine in *csp2s22c03* instead of using
34kernel forwarding. On *csp2s22c03*, we configure the system to have the addresses
3510.10.1.1/24 and 10.10.2.1/24 on the two 40-GbE NICs. To find all network interfaces
36available on the system, use the lshw Linux command to list all network interfaces
37and the corresponding slots *[0000:xx:yy.z]*.
38
39In this example, the 40-GbE interfaces are *ens802f0* and *ens802f1*.
40
41.. code-block:: console
42
43 csp2s22c03$ sudo lshw -class network -businfo
44 Bus info Device Class Description
45 ========================================================
46 pci@0000:03:00.0 enp3s0f0 network Ethernet Controller 10-Gig
47 pci@0000:03:00.1 enp3s0f1 network Ethernet Controller 10-Gig
48 pci@0000:82:00.0 ens802f0 network Ethernet Controller XL710
49 pci@0000:82:00.1 ens802f1 network Ethernet Controller XL710
50 pci@0000:82:00.0 ens802f0d1 network Ethernet interface
51 pci@0000:82:00.1 ens802f1d1 network Ethernet interface
52
53
54Configure the system *csp2s22c03* to have 10.10.1.1 and 10.10.2.1 on the two 40-GbE NICs
55*ens802f0* and *ens802f1*, respectively.
56
57.. code-block:: console
58
59 csp2s22c03$ sudo ip addr add 10.10.1.1/24 dev ens802f0
60 csp2s22c03$ sudo ip link set dev ens802f0 up
61 csp2s22c03$ sudo ip addr add 10.10.2.1/24 dev ens802f1
62 csp2s22c03$ sudo ip link set dev ens802f1 up
63
64List the route table:
65
66.. code-block:: console
67
68 csp2s22c03$ route
69 Kernel IP routing table
70 Destination Gateway Genmask Flags Metric Ref Use Iface
71 default jf111-ldr1a-530 0.0.0.0 UG 0 0 0 enp3s0f1
72 default 192.168.0.50 0.0.0.0 UG 100 0 0 enp3s0f0
73 10.10.1.0 * 255.255.255.0 U 0 0 0 ens802f0
74 10.10.2.0 * 255.255.255.0 U 0 0 0 ens802f1
75 10.23.3.0 * 255.255.255.0 U 0 0 0 enp3s0f1
76 link-local * 255.255.0.0 U 1000 0 0 enp3s0f1
77 192.168.0.0 * 255.255.255.0 U 100 0 0 enp3s0f0
78
79.. code-block:: console
80
81 csp2s22c03$ ip route
82 default via 10.23.3.1 dev enp3s0f1
83 default via 192.168.0.50 dev enp3s0f0 proto static metric 100
84 10.10.1.0/24 dev ens802f0 proto kernel scope link src 10.10.1.1
85 10.10.2.0/24 dev ens802f1 proto kernel scope link src 10.10.2.1
86 10.23.3.0/24 dev enp3s0f1 proto kernel scope link src 10.23.3.67
87 169.254.0.0/16 dev enp3s0f1 scope link metric 1000
88 192.168.0.0/24 dev enp3s0f0 proto kernel scope link src 192.168.0.142 metric 100
89
90On *csp2s22c04*, we configure the system to have the address 10.10.1.2 and use
91the interface *ens802* to route IP packets 10.10.2.0/24. Use the lshw Linux
92command to list all network interfaces and the corresponding slots *[0000:xx:yy.z]*.
93
94For example, the interface *ens802d1* *(ens802)* is connected to slot *[82:00.0]*:
95
96.. code-block:: console
97
98 csp2s22c04$ sudo lshw -class network -businfo
99 Bus info Device Class Description
100 =====================================================
101 pci@0000:03:00.0 enp3s0f0 network Ethernet Controller 10-Gigabit X540-AT2
102 pci@0000:03:00.1 enp3s0f1 network Ethernet Controller 10-Gigabit X540-AT2
103 pci@0000:82:00.0 ens802d1 network Ethernet Controller XL710 for 40GbE QSFP+
104 pci@0000:82:00.0 ens802 network Ethernet interface
105
106For kernel forwarding, set 10.10.1.2 to the interface *ens802*, and add a static
107route for IP packet 10.10.2.0/24:
108
109.. code-block:: console
110
111 csp2s22c04$ sudo ip addr add 10.10.1.2/24 dev ens802
112 csp2s22c04$ sudo ip link set dev ens802 up
113 csp2s22c04$ sudo ip route add 10.10.2.0/24 via 10.10.1.1
114
115.. code-block:: console
116
117 csp2s22c04$ ifconfig
118 enp3s0f0 Link encap:Ethernet HWaddr a4:bf:01:00:92:73
119 inet addr:10.23.3.62 Bcast:10.23.3.255 Mask:255.255.255.0
120 inet6 addr: fe80::a6bf:1ff:fe00:9273/64 Scope:Link
121 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
122 RX packets:3411 errors:0 dropped:0 overruns:0 frame:0
123 TX packets:1179 errors:0 dropped:0 overruns:0 carrier:0
124 collisions:0 txqueuelen:1000
125 RX bytes:262230 (262.2 KB) TX bytes:139975 (139.9 KB)
126
127 ens802 Link encap:Ethernet HWaddr 68:05:ca:2e:76:e0
128 inet addr:10.10.1.2 Bcast:0.0.0.0 Mask:255.255.255.0
129 inet6 addr: fe80::6a05:caff:fe2e:76e0/64 Scope:Link
130 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
131 RX packets:0 errors:0 dropped:0 overruns:0 frame:0
132 TX packets:40 errors:0 dropped:0 overruns:0 carrier:0
133 collisions:0 txqueuelen:1000
134 RX bytes:0 (0.0 B) TX bytes:5480 (5.4 KB)
135
136 lo Link encap:Local Loopback
137 inet addr:127.0.0.1 Mask:255.0.0.0
138 inet6 addr: ::1/128 Scope:Host
139 UP LOOPBACK RUNNING MTU:65536 Metric:1
140 RX packets:31320 errors:0 dropped:0 overruns:0 frame:0
141 TX packets:31320 errors:0 dropped:0 overruns:0 carrier:0
142 collisions:0 txqueuelen:1
143 RX bytes:40301788 (40.3 MB) TX bytes:40301788 (40.3 MB)
144
145After setting the route, we can ping from *csp2s22c03* to *csp2s22c04*, and vice versa:
146
147.. code-block:: console
148
149 csp2s22c03$ ping 10.10.1.2 -c 3
150 PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
151 64 bytes from 10.10.1.2: icmp_seq=1 ttl=64 time=0.122 ms
152 64 bytes from 10.10.1.2: icmp_seq=2 ttl=64 time=0.109 ms
153 64 bytes from 10.10.1.2: icmp_seq=3 ttl=64 time=0.120 ms
154
155.. code-block:: console
156
157 csp2s22c04$ ping 10.10.1.1 -c 3
158 PING 10.10.1.1 (10.10.1.1) 56(84) bytes of data.
159 64 bytes from 10.10.1.1: icmp_seq=1 ttl=64 time=0.158 ms
160 64 bytes from 10.10.1.1: icmp_seq=2 ttl=64 time=0.096 ms
161 64 bytes from 10.10.1.1: icmp_seq=3 ttl=64 time=0.102 ms
162
163Similarly, on *net2s22c05*, we configure the system to have the address *10.10.2.2*
164and use the interface *ens803f0* to route IP packets *10.10.1.0/24*. Use the lshw
165Linux command to list all network interfaces and the corresponding slots
166*[0000:xx:yy.z]*. For example, the interface *ens803f0* is connected to slot *[87:00.0]*:
167
168.. code-block:: console
169
170 NET2S22C05$ sudo lshw -class network -businfo
171 Bus info Device Class Description
172 ========================================================
173 pci@0000:03:00.0 enp3s0f0 network Ethernet Controller 10-Gigabit X540-AT2
174 pci@0000:03:00.1 enp3s0f1 network Ethernet Controller 10-Gigabit X540-AT2
175 pci@0000:81:00.0 ens787f0 network 82599 10 Gigabit TN Network Connection
176 pci@0000:81:00.1 ens787f1 network 82599 10 Gigabit TN Network Connection
177 pci@0000:87:00.0 ens803f0 network Ethernet Controller XL710 for 40GbE QSFP+
178 pci@0000:87:00.1 ens803f1 network Ethernet Controller XL710 for 40GbE QSFP+
179
180For kernel forwarding, set 10.10.2.2 to the interface ens803f0, and add a static
181route for IP packet 10.10.1.0/24:
182
183.. code-block:: console
184
185 NET2S22C05$ sudo ip addr add 10.10.2.2/24 dev ens803f0
186 NET2S22C05$ sudo ip link set dev ens803f0 up
187 NET2S22C05$ sudo ip route add 10.10.1.0/24 via 10.10.2.1
188
189After setting the route, you can ping from *csp2s22c03* to *net2s22c05*, and vice
190versa. However, in order to ping between *net2s22c05* and *csp2s22c04*, kernel IP
191forwarding in *csp2s22c03* has to be enabled:
192
193.. code-block:: console
194
195 csp2s22c03$ sysctl net.ipv4.ip_forward
196 net.ipv4.ip_forward = 0
197 csp2s22c03$ echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward
198 csp2s22c03$ sysctl net.ipv4.ip_forward
199 net.ipv4.ip_forward = 1
200
201If successful, verify that now you can ping between *net2s22c05* and *csp2s22c04*:
202
203.. code-block:: console
204
205 NET2S22C05$ ping 10.10.1.2 -c 3
206 PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
207 64 bytes from 10.10.1.2: icmp_seq=1 ttl=63 time=0.239 ms
208 64 bytes from 10.10.1.2: icmp_seq=2 ttl=63 time=0.224 ms
209 64 bytes from 10.10.1.2: icmp_seq=3 ttl=63 time=0.230 ms
210
211We use the **iperf3** utility to measure network bandwidth between hosts. In this
212test, we download the **iperf3** utility tool on both *net2s22c05* and *csp2s22c04*.
213On *csp2s22c04*, we start the **iperf3** server with “iperf3 –s”, and then on *net2s22c05*,
214we start the **iperf3** client to connect to the server:
215
216.. code-block:: console
217
218 NET2S22C05$ iperf3 -c 10.10.1.2
219 Connecting to host 10.10.1.2, port 5201
220 [ 4] local 10.10.2.2 port 54074 connected to 10.10.1.2 port 5201
221 [ ID] Interval Transfer Bandwidth Retr Cwnd
222 [ 4] 0.00-1.00 sec 936 MBytes 7.85 Gbits/sec 2120 447 KBytes
223 [ 4] 1.00-2.00 sec 952 MBytes 7.99 Gbits/sec 1491 611 KBytes
224 [ 4] 2.00-3.00 sec 949 MBytes 7.96 Gbits/sec 2309 604 KBytes
225 [ 4] 3.00-4.00 sec 965 MBytes 8.10 Gbits/sec 1786 571 KBytes
226 [ 4] 4.00-5.00 sec 945 MBytes 7.93 Gbits/sec 1984 424 KBytes
227 [ 4] 5.00-6.00 sec 946 MBytes 7.94 Gbits/sec 1764 611 KBytes
228 [ 4] 6.00-7.00 sec 979 MBytes 8.21 Gbits/sec 1499 655 KBytes
229 [ 4] 7.00-8.00 sec 980 MBytes 8.22 Gbits/sec 1182 867 KBytes
230 [ 4] 8.00-9.00 sec 1008 MBytes 8.45 Gbits/sec 945 625 KBytes
231 [ 4] 9.00-10.00 sec 1015 MBytes 8.51 Gbits/sec 1394 611 KBytes
232 - - - - - - - - - - - - - - - - - - - - - - - - -
233 [ ID] Interval Transfer Bandwidth Retr
234 [ 4] 0.00-10.00 sec 9.45 GBytes 8.12 Gbits/sec 16474 sender
235 [ 4] 0.00-10.00 sec 9.44 GBytes 8.11 Gbits/sec receiver
236
237 iperf Done.