Gitiles
Code Review Sign In
gerrit.nordix.org / oransc / pti / rtp / 8ff63e9822783c2535bcb4f8dfee6da299607531 / . / docs / installation-guide.rst
blob: e8dc09bd3f834db27ba0b38d6389aa62fd439c0f [file] [log] [blame]
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]1.. This work is licensed under a Creative Commons Attribution 4.0 International License.
2.. SPDX-License-Identifier: CC-BY-4.0
3.. Copyright (C) 2019 Wind River Systems, Inc.
4
5
6Installation Guide
7==================
8
9.. contents::
10 :depth: 3
11 :local:
12
13Abstract
14--------
15
16This document describes how to install O-RAN INF image, example configuration for better
17real time performance, and example deployment of Kubernetes cluster and plugins.
18
19The audience of this document is assumed to have basic knowledge in Yocto/Open-Embedded Linux
20and container technology.
21
22Version history
23
24+--------------------+--------------------+--------------------+--------------------+
25| **Date** | **Ver.** | **Author** | **Comment** |
26| | | | |
27+--------------------+--------------------+--------------------+--------------------+
28| 2019-11-02 | 1.0.0 | Jackie Huang | Initail version |
29| | | | |
30+--------------------+--------------------+--------------------+--------------------+
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]31| 2020-06-03 | 2.0.0 (Bronze) | Xiaohua Zhang | Bronze version |
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]32| | | | |
33+--------------------+--------------------+--------------------+--------------------+
34| | | | |
35| | | | |
36+--------------------+--------------------+--------------------+--------------------+
37
38
39Preface
40-------
41
42Before starting the installation and deployment of O-RAN INF, you need to download the ISO image or build from source as described in developer-guide.
43
44
45Hardware Requirements
46---------------------
47
48Following minimum hardware requirements must be met for installation of O-RAN INF image:
49
50+--------------------+----------------------------------------------------+
51| **HW Aspect** | **Requirement** |
52| | |
53+--------------------+----------------------------------------------------+
54| **# of servers** | 1 |
55+--------------------+----------------------------------------------------+
Xiaohua Zhang8ff63e92020-06-01 23:24:41 -0400[diff] [blame^]56| **CPU** | 4 |
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]57| | |
58+--------------------+----------------------------------------------------+
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]59| **RAM** | 16G |
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]60| | |
61+--------------------+----------------------------------------------------+
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]62| **Disk** | 500G * 2 |
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]63| | |
64+--------------------+----------------------------------------------------+
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]65| **NICs** | 2 |
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]66| | |
67+--------------------+----------------------------------------------------+
68
69
70
71Software Installation and Deployment
72------------------------------------
73
741. Installation from the O-RAN INF ISO image
75````````````````````````````````````````````
76
77- Please see the README.md file for how to build the image.
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]78- The Image is a live ISO image with CLI installer: inf-image-aio-installer-intel-corei7-64.iso
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]79
801.1 Burn the image to the USB device
81''''''''''''''''''''''''''''''''''''
82
83- Assume the the usb device is /dev/sdX here
84
85::
86
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]87 $ sudo dd if=/path/to/inf-image-aio-installer-intel-corei7-64.iso of=/dev/sdX bs=1M
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]88
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]891.2 Install the server
90''''''''''''''''''''''
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]91
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]92- Reboot the target from the USB device.
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]93
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]94- Select "Graphics console install" or "Serial console install" and press ENTER
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]95
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]96- Remove the USB device and press ENTER to reboot
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]97
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]98
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]992. Configuration and initialize the bootstrap
100`````````````````````````````````````````````
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]101
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]1022.1 First Login
103```````````````
104Example:
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]105
106::
107
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]108 ifconfig eno1 128.224.180.14/24 up
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]109
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]110 ip route add default via 128.224.180.1
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]111
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]112 ping 8.8.8.8
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]113
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]1142.3 Login the server through SSH
115````````````````````````````````
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]116
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]1172.4 Prepare the localhost.yml for bootstrap
118```````````````````````````````````````````
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]119
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]120Example:
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]121
122::
123
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]124 system_mode: simplex
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]125
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]126 external_oam_subnet: 128.224.180.0/24
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]127
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]128 external_oam_gateway_address: 128.224.180.1
129
130 external_oam_floating_address: 128.224.180.14
131
132 admin_username: admin
133
134 admin_password: Wind123!
135
136 ansible_become_pass: Wind123!
137
138 docker_http_proxy: http://xxx.xxx.xxx.xxx:8080
139
140 docker_https_proxy: https://xxx.xxx.xxx.xxx:8080
141
1422.5 Run the bootstrap
143`````````````````````
144
145Use the following command:
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]146
147::
148
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]149 ansible-playbook /usr/share/ansible/stx-ansible/playbooks/bootstrap.yml -vvv
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]150
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]151After the bootstrap successfully finish, it will show as following:
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]152
153::
154
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]155 PLAY RECAP ******************************************************************************************************************************************
156 localhost : ok=256 changed=135 unreachable=0 failed=0 skipped=220 rescued=0 ignored=0
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]157
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]1582.6 Configure the OAM network
159`````````````````````````````
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]160
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]161Example:
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]162
163::
164
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]165 controller-0:~$ source /etc/platform/openrc
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]166
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]167 [sysadmin@controller-0 ~(keystone_admin)]$ OAM_IF=eno1
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]168
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]169 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify controller-0 $OAM_IF -c platform
170
171 +-----------------+--------------------------------------+
172 | Property | Value |
173 +-----------------+--------------------------------------+
174 | ifname | eno1 |
175 +-----------------+--------------------------------------+
176 | iftype | ethernet |
177 +-----------------+--------------------------------------+
178 | ports | [u'eno1'] |
179 +-----------------+--------------------------------------+
180 | imac | 3c:a8:2a:1d:31:d0 |
181 +-----------------+--------------------------------------+
182 | imtu | 1500 |
183 +-----------------+--------------------------------------+
184 | ifclass | platform |
185 +-----------------+--------------------------------------+
186 | aemode | None |
187 +-----------------+--------------------------------------+
188 | schedpolicy | None |
189 +-----------------+--------------------------------------+
190 | txhashpolicy | None |
191 +-----------------+--------------------------------------+
192 | uuid | 95a32d13-01fc-43d2-90ab-801970fb3859 |
193 +-----------------+--------------------------------------+
194 | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
195 +-----------------+--------------------------------------+
196 | vlan_id | None |
197 +-----------------+--------------------------------------+
198 | uses | [] |
199 +-----------------+--------------------------------------+
200 | used_by | [] |
201 +-----------------+--------------------------------------+
202 | created_at | 2020-06-03T08:21:17.526303+00:00 |
203 +-----------------+--------------------------------------+
204 | updated_at | 2020-06-03T10:02:54.682501+00:00 |
205 +-----------------+--------------------------------------+
206 | sriov_numvfs | 0 |
207 +-----------------+--------------------------------------+
208 | sriov_vf_driver | None |
209 +-----------------+--------------------------------------+
210 | accelerated | [False] |
211 +-----------------+--------------------------------------+
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]212
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]213 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-assign controller-0 $OAM_IF oam
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]214
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]215 +--------------+--------------------------------------+
216 | Property | Value |
217 +--------------+--------------------------------------+
218 | hostname | controller-0 |
219 +--------------+--------------------------------------+
220 | uuid | 32592cc3-9d66-4961-a51e-23686da00290 |
221 +--------------+--------------------------------------+
222 | ifname | eno1 |
223 +--------------+--------------------------------------+
224 | network_name | oam |
225 +--------------+--------------------------------------+
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]226
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]227
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]228 [sysadmin@controller-0 ~(keystone_admin)]$
229 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-network-list controller-0
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]230
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]231 +--------------+--------------------------------------+--------+--------------+
232 | hostname | uuid | ifname | network_name |
233 +--------------+--------------------------------------+--------+--------------+
234 | controller-0 | 2478f4c0-a7b4-4656-aa2a-97a0e8891955 | lo | mgmt |
235 +--------------+--------------------------------------+--------+--------------+
236 | controller-0 | 32592cc3-9d66-4961-a51e-23686da00290 | eno1 | oam |
237 +--------------+--------------------------------------+--------+--------------+
238 | controller-0 | 43721e4d-4f52-4b4b-bd02-0efa00ef2124 | lo | cluster-host |
239 +--------------+--------------------------------------+--------+--------------+
240
2412.7 Configure the NTP server
242````````````````````````````
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]243
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]244Example:
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]245
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]246::
247
248 [sysadmin@controller-0 ~(keystone_admin)]$
249 [sysadmin@controller-0 ~(keystone_admin)]$ system ntp-modify ntpservers=0.pool.ntp.org,1.pool.ntp.org
250
251 +--------------+--------------------------------------+
252 | Property | Value |
253 +--------------+--------------------------------------+
254 | uuid | 8fbe5712-6ba0-4e61-b174-ed60f152f616 |
255 +--------------+--------------------------------------+
256 | ntpservers | 0.pool.ntp.org,1.pool.ntp.org |
257 +--------------+--------------------------------------+
258 | isystem_uuid | 802ee0be-3728-4df4-b9f7-1533c67fd960 |
259 +--------------+--------------------------------------+
260 | created_at | 2020-06-03T08:19:47.812249+00:00 |
261 +--------------+--------------------------------------+
262 | updated_at | None |
263 +--------------+--------------------------------------+
264
2652.8 Configure the Storage by using the second hard drive as ceph backend
266````````````````````````````````````````````````````````````````````````
267
268Example:
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]269
270::
271
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]272 [sysadmin@controller-0 ~(keystone_admin)]$
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]273
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]274 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]275
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]276 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
277 | uuid | device_no | device_ | device_ | size_ | available_ | rpm | serial_ | device_path |
278 | | de | num | type | gib | gib | | id | |
279 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
280 | 3884f906-cb9b-4052-bcb8-8be70e599b9f | /dev/sda | 2048 | HDD | 1117. | 868.759 | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
281 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
282 | | | | | 782 | | | RH8W4QQ | 03:00.0-scsi-0:1:0:0 |
283 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
284 | | | | | | | | | |
285 | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/sdb | 2064 | HDD | 1117. | 0.0 | Undetermined | PDNLH0B | /dev/disk/by-path/pci-0000: |
286 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
287 | | | | | 782 | | | RH8W4QQ | 03:00.0-scsi-0:1:0:1 |
288 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
289 | | | | | | | | | |
290 +--------------------------------------+-----------+---------+---------+-------+------------+--------------+---------+-----------------------------+
291
292 [sysadmin@controller-0 ~(keystone_admin)]$ system host-disk-list controller-0 | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add controller-0 {}
293
294 +------------------+-------------------------------------------------------+
295 | Property | Value |
296 +------------------+-------------------------------------------------------+
297 | osdid | 0 |
298 +------------------+-------------------------------------------------------+
299 | function | osd |
300 +------------------+-------------------------------------------------------+
301 | state | configuring-on-unlock |
302 +------------------+-------------------------------------------------------+
303 | journal_location | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf |
304 +------------------+-------------------------------------------------------+
305 | journal_size_gib | 1024 |
306 +------------------+-------------------------------------------------------+
307 | journal_path | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 |
308 +------------------+-------------------------------------------------------+
309 | journal_node | /dev/sdb2 |
310 +------------------+-------------------------------------------------------+
311 | uuid | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf |
312 +------------------+-------------------------------------------------------+
313 | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
314 +------------------+-------------------------------------------------------+
315 | idisk_uuid | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 |
316 +------------------+-------------------------------------------------------+
317 | tier_uuid | db6eabf9-9922-4d8e-9740-c0f9878e692d |
318 +------------------+-------------------------------------------------------+
319 | tier_name | storage |
320 +------------------+-------------------------------------------------------+
321 | created_at | 2020-06-03T10:03:55.645744+00:00 |
322 +------------------+-------------------------------------------------------+
323 | updated_at | None |
324 +------------------+-------------------------------------------------------+
325
326 [sysadmin@controller-0 ~(keystone_admin)]$ system host-stor-list controller-0
327
328 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
329 | uuid | function | osdid | state | idisk_uuid | journal_path | journal_node | journal_size_gib | tier_name |
330 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
331 | 71a9d508-06f6-4f5a-84f2-4723e70b1ecf | osd | 0 | configuring-on-unlock | 4e6c4b16-11e6-40b7-a2ff-751b14b8f0e9 | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:1-part2 | /dev/sdb2 | 1 | storage |
332 +--------------------------------------+----------+-------+-----------------------+--------------------------------------+-------------------------------------------------------+--------------+------------------+-----------+
333
3342.9 Configure the SR-IOV enabled network
335````````````````````````````````````````
336It need :
337 - assign the label of SR-IOV
338 - change the CPU policy to static
339 - change the hugepage setting
340 - create the data network
341 - assign the SR-IOV interface to data network
342
343Example:
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]344
345::
346
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]347 [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]348
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]349 [sysadmin@controller-0 ~(keystone_admin)]$
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]350
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]351 [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER sriovdp=enabled
352
353 +-------------+--------------------------------------+
354 | Property | Value |
355 +-------------+--------------------------------------+
356 | uuid | 7101010d-1897-4cc8-8c87-7dffaebf848b |
357 +-------------+--------------------------------------+
358 | host_uuid | 08c23843-afa4-4b80-a285-be126365276e |
359 +-------------+--------------------------------------+
360 | label_key | sriovdp |
361 +-------------+--------------------------------------+
362 | label_value | enabled |
363 +-------------+--------------------------------------+
364
365 [sysadmin@controller-0 ~(keystone_admin)]$ system host-label-assign $WORKER kube-cpu-mgr-policy=static
366
367 +-------------+--------------------------------------+
368 | Property | Value |
369 +-------------+--------------------------------------+
370 | uuid | 4936f32b-6939-4115-b643-846bd50e7738 |
371 +-------------+--------------------------------------+
372 | host_uuid | 08c23843-afa4-4b80-a285-be126365276e |
373 +-------------+--------------------------------------+
374 | label_key | kube-cpu-mgr-policy |
375 +-------------+--------------------------------------+
376 | label_value | static |
377 +-------------+--------------------------------------+
378
379
380 [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7 0
381
382 +-------------------------------------+--------------------------------------+
383 | Property | Value |
384 +-------------------------------------+--------------------------------------+
385 | Memory: Usable Total (MiB) | 0 |
386 +-------------------------------------+--------------------------------------+
387 | Platform (MiB) | 7000 |
388 +-------------------------------------+--------------------------------------+
389 | Available (MiB) | 0 |
390 +-------------------------------------+--------------------------------------+
391 | Huge Pages Configured | True |
392 +-------------------------------------+--------------------------------------+
393 | vSwitch Huge Pages: Size (MiB) | 1024 |
394 +-------------------------------------+--------------------------------------+
395 | Total | 0 |
396 +-------------------------------------+--------------------------------------+
397 | Available | 0 |
398 +-------------------------------------+--------------------------------------+
399 | Required | None |
400 +-------------------------------------+--------------------------------------+
401 | Application Pages (4K): Total | None |
402 +-------------------------------------+--------------------------------------+
403 | Application Huge Pages (2M): Total | 0 |
404 +-------------------------------------+--------------------------------------+
405 | Available | 0 |
406 +-------------------------------------+--------------------------------------+
407 | Application Huge Pages (1G): Total | 0 |
408 +-------------------------------------+--------------------------------------+
409 | Total Pending | 7 |
410 +-------------------------------------+--------------------------------------+
411 | Available | 0 |
412 +-------------------------------------+--------------------------------------+
413 | uuid | 15702f44-2281-41f3-8c83-a1e9a300ad01 |
414 +-------------------------------------+--------------------------------------+
415 | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
416 +-------------------------------------+--------------------------------------+
417 | inode_uuid | d52648bc-dfa6-4214-afa3-5d2148ee4180 |
418 +-------------------------------------+--------------------------------------+
419 | created_at | 2020-06-03T08:21:27.468357+00:00 |
420 +-------------------------------------+--------------------------------------+
421 | updated_at | 2020-06-03T10:10:49.159080+00:00 |
422 +-------------------------------------+--------------------------------------+
423
424
425 [sysadmin@controller-0 ~(keystone_admin)]$ system host-memory-modify $WORKER -1G 7 1
426
427 +-------------------------------------+--------------------------------------+
428 | Property | Value |
429 +-------------------------------------+--------------------------------------+
430 | Memory: Usable Total (MiB) | 0 |
431 +-------------------------------------+--------------------------------------+
432 | Platform (MiB) | 1000 |
433 +-------------------------------------+--------------------------------------+
434 | Available (MiB) | 0 |
435 +-------------------------------------+--------------------------------------+
436 | Huge Pages Configured | True |
437 +-------------------------------------+--------------------------------------+
438 | vSwitch Huge Pages: Size (MiB) | 1024 |
439 +-------------------------------------+--------------------------------------+
440 | Total | 0 |
441 +-------------------------------------+--------------------------------------+
442 | Available | 0 |
443 +-------------------------------------+--------------------------------------+
444 | Required | None |
445 +-------------------------------------+--------------------------------------+
446 | Application Pages (4K): Total | None |
447 +-------------------------------------+--------------------------------------+
448 | Application Huge Pages (2M): Total | 0 |
449 +-------------------------------------+--------------------------------------+
450 | Available | 0 |
451 +-------------------------------------+--------------------------------------+
452 | Application Huge Pages (1G): Total | 0 |
453 +-------------------------------------+--------------------------------------+
454 | Total Pending | 7 |
455 +-------------------------------------+--------------------------------------+
456 | Available | 0 |
457 +-------------------------------------+--------------------------------------+
458 | uuid | 544580b9-0f2a-41a1-b4e3-90fdf5c8f7e0 |
459 +-------------------------------------+--------------------------------------+
460 | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
461 +-------------------------------------+--------------------------------------+
462 | inode_uuid | e02c3c41-03b7-4f49-8244-bca851fa2d2f |
463 +-------------------------------------+--------------------------------------+
464 | created_at | 2020-06-03T08:21:27.566279+00:00 |
465 +-------------------------------------+--------------------------------------+
466 | updated_at | 2020-06-03T10:10:49.220907+00:00 |
467 +-------------------------------------+--------------------------------------+
468
469
470 [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET0='physnet0'
471
472 [sysadmin@controller-0 ~(keystone_admin)]$ PHYSNET1='physnet1'
473
474 [sysadmin@controller-0 ~(keystone_admin)]$ WORKER=controller-0
475
476 [sysadmin@controller-0 ~(keystone_admin)]$
477
478 [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET0} flat
479
480 +--------------+--------------------------------------+
481 | Property | Value |
482 +--------------+--------------------------------------+
483 | id | 1 |
484 +--------------+--------------------------------------+
485 | uuid | 0ffe0b0a-35ff-45b2-9a62-63a494d2d18b |
486 +--------------+--------------------------------------+
487 | name | physnet0 |
488 +--------------+--------------------------------------+
489 | network_type | flat |
490 +--------------+--------------------------------------+
491 | mtu | 1500 |
492 +--------------+--------------------------------------+
493 | description | None |
494 +--------------+--------------------------------------+
495
496 [sysadmin@controller-0 ~(keystone_admin)]$ system datanetwork-add ${PHYSNET1} flat
497
498 +--------------+--------------------------------------+
499 | Property | Value |
500 +--------------+--------------------------------------+
501 | id | 2 |
502 +--------------+--------------------------------------+
503 | uuid | 899ca8ba-6800-4d62-aed4-3770878da738 |
504 +--------------+--------------------------------------+
505 | name | physnet1 |
506 +--------------+--------------------------------------+
507 | network_type | flat |
508 +--------------+--------------------------------------+
509 | mtu | 1500 |
510 +--------------+--------------------------------------+
511 | description | None |
512 +--------------+--------------------------------------+
513
514
515 [sysadmin@controller-0 ~(keystone_admin)]$ system host-port-list controller-0
516
517 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
518 | uuid | name | type | pci address | device | processor | accelerated | device type |
519 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
520 | 7007dfb1-ea55-4355-9d27-3ed4c6696283 | eno1 | ethernet | 0000:02:00.0 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
521 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
522 | 51f62ecf-7897-43e3-beb8-46ae5550ef05 | eno2 | ethernet | 0000:02:00.1 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
523 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
524 | e987725e-ab45-4151-b53d-94ba21706eb4 | eno3 | ethernet | 0000:02:00.2 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
525 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
526 | f4a8115a-ca0b-4336-a634-2cad77c5a88c | eno4 | ethernet | 0000:02:00.3 | 0 | 0 | False | NetXtreme BCM5719 Gigabit Ethernet PCIe [1657] |
527 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
528 | 88965882-c816-42e8-ba0e-d3ff6f9f553e | eno49 | ethernet | 0000:04:00.0 | 0 | 0 | False | OneConnect NIC (Skyhawk) [0720] |
529 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
530 | 98e2cedc-9ef8-4ca0-b3e0-db3a2bd0273a | eno50 | ethernet | 0000:04:00.1 | 0 | 0 | False | OneConnect NIC (Skyhawk) [0720] |
531 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
532 | e75d7aa6-7832-4ec4-9dd4-ea8300773fd8 | ens2f0 | ethernet | 0000:05:00.0 | 0 | 0 | True | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
533 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
534 | 8ac4ffca-4a61-4b47-8e17-171f2e6cea66 | ens2f1 | ethernet | 0000:05:00.1 | 0 | 0 | True | 82599ES 10-Gigabit SFI/SFP+ Network Connection [10fb] |
535 +--------------------------------------+--------+----------+--------------+--------+-----------+-------------+-------------------------------------------------------+
536
537
538 [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f0
539
540 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov0 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
541
542 +-----------------+--------------------------------------+
543 | Property | Value |
544 +-----------------+--------------------------------------+
545 | ifname | sriov0 |
546 +-----------------+--------------------------------------+
547 | iftype | ethernet |
548 +-----------------+--------------------------------------+
549 | ports | [u'ens2f0'] |
550 +-----------------+--------------------------------------+
551 | imac | 00:1b:21:b3:46:2c |
552 +-----------------+--------------------------------------+
553 | imtu | 1500 |
554 +-----------------+--------------------------------------+
555 | ifclass | pci-sriov |
556 +-----------------+--------------------------------------+
557 | aemode | None |
558 +-----------------+--------------------------------------+
559 | schedpolicy | None |
560 +-----------------+--------------------------------------+
561 | txhashpolicy | None |
562 +-----------------+--------------------------------------+
563 | uuid | 9450b1ae-d1a1-4ecb-811a-fa3cb1cdf362 |
564 +-----------------+--------------------------------------+
565 | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
566 +-----------------+--------------------------------------+
567 | vlan_id | None |
568 +-----------------+--------------------------------------+
569 | uses | [] |
570 +-----------------+--------------------------------------+
571 | used_by | [] |
572 +-----------------+--------------------------------------+
573 | created_at | 2020-06-03T12:34:11.633254+00:00 |
574 +-----------------+--------------------------------------+
575 | updated_at | 2020-06-04T02:01:28.168127+00:00 |
576 +-----------------+--------------------------------------+
577 | sriov_numvfs | 6 |
578 +-----------------+--------------------------------------+
579 | sriov_vf_driver | vfio |
580 +-----------------+--------------------------------------+
581 | accelerated | [True] |
582 +-----------------+--------------------------------------+
583
584 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov0 ${PHYSNET0}
585
586 +------------------+--------------------------------------+
587 | Property | Value |
588 +------------------+--------------------------------------+
589 | hostname | controller-0 |
590 +------------------+--------------------------------------+
591 | uuid | 1d35219d-3258-42e7-8f0e-41d865710d58 |
592 +------------------+--------------------------------------+
593 | ifname | sriov0 |
594 +------------------+--------------------------------------+
595 | datanetwork_name | physnet0 |
596 +------------------+--------------------------------------+
597
598
599
600 [sysadmin@controller-0 ~(keystone_admin)]$ INTL_IF=ens2f1
601
602 [sysadmin@controller-0 ~(keystone_admin)]$ system host-if-modify -m 1500 -n sriov1 -c pci-sriov -N 6 --vf-driver=vfio $WORKER $INTL_IF
603
604 +-----------------+--------------------------------------+
605 | Property | Value |
606 +-----------------+--------------------------------------+
607 | ifname | sriov1 |
608 +-----------------+--------------------------------------+
609 | iftype | ethernet |
610 +-----------------+--------------------------------------+
611 | ports | [u'ens2f1'] |
612 +-----------------+--------------------------------------+
613 | imac | 00:1b:21:b3:46:2d |
614 +-----------------+--------------------------------------+
615 | imtu | 1500 |
616 +-----------------+--------------------------------------+
617 | ifclass | pci-sriov |
618 +-----------------+--------------------------------------+
619 | aemode | None |
620 +-----------------+--------------------------------------+
621 | schedpolicy | None |
622 +-----------------+--------------------------------------+
623 | txhashpolicy | None |
624 +-----------------+--------------------------------------+
625 | uuid | 13d5ee99-0399-4b6b-a359-0f96660afdc4 |
626 | ihost_uuid | 08c23843-afa4-4b80-a285-be126365276e |
627 | vlan_id | None |
628 | uses | [] |
629 | used_by | [] |
630 | created_at | 2020-06-03T12:34:12.072624+00:00 |
631 | updated_at | 2020-06-04T02:03:51.405322+00:00 |
632 | sriov_numvfs | 6 |
633 | sriov_vf_driver | vfio |
634 | accelerated | [True] |
635 +-----------------+--------------------------------------+
636
637 [sysadmin@controller-0 ~(keystone_admin)]$ system interface-datanetwork-assign ${WORKER} sriov1 ${PHYSNET1}
638
639 +------------------+--------------------------------------+
640 | Property | Value |
641 +------------------+--------------------------------------+
642 | hostname | controller-0 |
643 +------------------+--------------------------------------+
644 | uuid | a87c6a5f-0c53-4ebf-9a32-38366700426f |
645 +------------------+--------------------------------------+
646 | ifname | sriov1 |
647 +------------------+--------------------------------------+
648 | datanetwork_name | physnet1 |
649 +------------------+--------------------------------------+
650
6512.10 Unlock the server
652``````````````````````
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]653
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]654Example:
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]655
656::
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]657
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]658 [sysadmin@controller-0 ~(keystone_admin)]$ system host-unlock controller-0
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]659
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]660 +-----------------------+-------------------------------------------------+
661 | Property | Value |
662 +-----------------------+-------------------------------------------------+
663 | action | none |
664 +-----------------------+-------------------------------------------------+
665 | administrative | locked |
666 +-----------------------+-------------------------------------------------+
667 | availability | online |
668 +-----------------------+-------------------------------------------------+
669 | bm_ip | None |
670 +-----------------------+-------------------------------------------------+
671 | bm_type | none |
672 +-----------------------+-------------------------------------------------+
673 | bm_username | None |
674 +-----------------------+-------------------------------------------------+
675 | boot_device | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
676 +-----------------------+-------------------------------------------------+
677 | capabilities | {u'stor_function': u'monitor'} |
678 +-----------------------+-------------------------------------------------+
679 | clock_synchronization | ntp |
680 +-----------------------+-------------------------------------------------+
681 | config_applied | 3daf20c7-c032-4aa2-839d-93da92051bea |
682 +-----------------------+-------------------------------------------------+
683 | config_status | Config out-of-date |
684 +-----------------------+-------------------------------------------------+
685 | config_target | bdaf20c7-c032-4aa2-839d-93da92051bea |
686 +-----------------------+-------------------------------------------------+
687 | console | tty0 |
688 +-----------------------+-------------------------------------------------+
689 | created_at | 2020-06-03T08:20:59.372602+00:00 |
690 +-----------------------+-------------------------------------------------+
691 | hostname | controller-0 |
692 +-----------------------+-------------------------------------------------+
693 | id | 1 |
694 +-----------------------+-------------------------------------------------+
695 | install_output | graphical |
696 +-----------------------+-------------------------------------------------+
697 | install_state | None |
698 +-----------------------+-------------------------------------------------+
699 | install_state_info | None |
700 +-----------------------+-------------------------------------------------+
701 | inv_state | inventoried |
702 +-----------------------+-------------------------------------------------+
703 | invprovision | provisioning |
704 +-----------------------+-------------------------------------------------+
705 | location | {} |
706 +-----------------------+-------------------------------------------------+
707 | mgmt_ip | 192.168.204.2 |
708 +-----------------------+-------------------------------------------------+
709 | mgmt_mac | 00:00:00:00:00:00 |
710 +-----------------------+-------------------------------------------------+
711 | operational | disabled |
712 +-----------------------+-------------------------------------------------+
713 | personality | controller |
714 +-----------------------+-------------------------------------------------+
715 | reserved | False |
716 +-----------------------+-------------------------------------------------+
717 | rootfs_device | /dev/disk/by-path/pci-0000:03:00.0-scsi-0:1:0:0 |
718 +-----------------------+-------------------------------------------------+
719 | serialid | None |
720 +-----------------------+-------------------------------------------------+
721 | software_load | 19.12 |
722 +-----------------------+-------------------------------------------------+
723 | subfunction_avail | online |
724 +-----------------------+-------------------------------------------------+
725 | subfunction_oper | disabled |
726 +-----------------------+-------------------------------------------------+
727 | subfunctions | controller,worker |
728 +-----------------------+-------------------------------------------------+
729 | task | Unlocking |
730 +-----------------------+-------------------------------------------------+
731 | tboot | false |
732 +-----------------------+-------------------------------------------------+
733 | ttys_dcd | None |
734 +-----------------------+-------------------------------------------------+
735 | updated_at | 2020-06-04T02:04:59.514854+00:00 |
736 +-----------------------+-------------------------------------------------+
737 | uptime | 49475 |
738 +-----------------------+-------------------------------------------------+
739 | uuid | 08c23843-afa4-4b80-a285-be126365276e |
740 +-----------------------+-------------------------------------------------+
741 | vim_progress_status | None |
742 +-----------------------+-------------------------------------------------+
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]743
Xiaohua Zhang8ff63e92020-06-01 23:24:41 -0400[diff] [blame^]7443. Simple use case for sriov
745````````````````````````````
746
7473.1 After controller-0 is rebooted and up running, download the DPDK
748````````````````````````````````````````````````````````````````````
749
750::
751
752 [sysadmin@controller-0 ~(keystone_admin)]$ cd /opt
753 [sysadmin@controller-0 opt(keystone_admin)]$ sudo wget https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
754 Password:
755 --2020-06-04 02:35:30-- https://fast.dpdk.org/rel/dpdk-17.11.10.tar.xz
756 Resolving fast.dpdk.org... 151.101.2.49, 151.101.66.49, 151.101.130.49, ...
757 Connecting to fast.dpdk.org|151.101.2.49|:443... connected.
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]758
Xiaohua Zhang8ff63e92020-06-01 23:24:41 -0400[diff] [blame^]759 HTTP request sent, awaiting response... 200 OK
760 Length: 10251680 (9.8M) [application/octet-stream]
761 Saving to: ‘dpdk-17.11.10.tar.xz’
762
763 dpdk-17.11.10.tar.xz 100%
764 [========================================================================================>] 9.78M
765 1.48MB/s in 6.8s
766
767 2020-06-04 02:35:40 (1.43 MB/s) - ‘dpdk-17.11.10.tar.xz’ saved [10251680/10251680]
768
769 sudo tar xvf dpdk-17.11.10.tar.xz
770
771 sudo ln -s dpdk-stable-17.11.10 dpdk-stable
772
7733.1 Prepare the yaml file for the network assignment container
774``````````````````````````````````````````````````````````````
775
776The following the exmaple of the yaml file:
777
778::
779
780 [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > netdef-data-dpdk.yaml
781 > apiVersion: "k8s.cni.cncf.io/v1"
782 > kind: NetworkAttachmentDefinition
783 > metadata:
784 > name: sriov-data-dpdk-0
785 > annotations:
786 > k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet0
787 > spec:
788 > config: '{
789 > "type": "sriov",
790 > "name": "sriov-data-dpdk-0"
791 > }'
792 >
793 > ---
794 > apiVersion: "k8s.cni.cncf.io/v1"
795 > kind: NetworkAttachmentDefinition
796 > metadata:
797 > name: sriov-data-dpdk-1
798 > annotations:
799 > k8s.v1.cni.cncf.io/resourceName: intel.com/pci_sriov_net_physnet1
800 > spec:
801 > config: '{
802 > "type": "sriov",
803 > "name": "sriov-data-dpdk-1"
804 > }'
805 > EOF
806
8073.2 Run the network assignent container for the 2 VFs
808`````````````````````````````````````````````````````
809
810::
811
812 [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f netdef-data-dpdk.yaml
813 networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-0 created
814 networkattachmentdefinition.k8s.cni.cncf.io/sriov-data-dpdk-1 created
815
8163.3 Prepare the VF container yaml file
817``````````````````````````````````````
818
819::
820
821 [sysadmin@controller-0 sriov(keystone_admin)]$ cat <<EOF > pod-with-dpdk-vfs-0.yaml
822 > apiVersion: v1
823 > kind: Pod
824 metadata:
825 > metadata:
826 > name: pod-with-dpdk-vfs-0
827 > annotations:
828 > k8s.v1.cni.cncf.io/networks: '[
829 > { "name": "sriov-data-dpdk-0" },
830 { "name": "sriov-data-dpdk-1" }
831 > { "name": "sriov-data-dpdk-1" }
832 > ]'
833 > spec:
834 > restartPolicy: Never
835 > containers:
836 > - name: pod-with-dpdk-vfs-0
837 > image: wrsnfv/ubuntu-dpdk-build:v0.3
838 > env:
839 > - name: RTE_SDK
840 > value: "/usr/src/dpdk"
841 > command:
842 > - sleep
843 > - infinity
844 > stdin: true
845 > tty: true
846 > securityContext:
847 > privileged: true
848 > capabilities:
849 > add:
850 > - ALL
851 > resources:
852 > requests:
853 > cpu: 4
854 > memory: 4Gi
855 > intel.com/pci_sriov_net_physnet0: '1'
856 > intel.com/pci_sriov_net_physnet1: '1'
857 > limits:
858 > cpu: 4
859 > hugepages-1Gi: 2Gi
860 > memory: 4Gi
861 > intel.com/pci_sriov_net_physnet0: '1'
862 > intel.com/pci_sriov_net_physnet1: '1'
863 > volumeMounts:
864 > - mountPath: /mnt/huge-1048576kB
865 > name: hugepage
866 > - name: dpdk-volume
867 > mountPath: /usr/src/dpdk
868 > - name: lib-volume
869 > mountPath: /lib/modules
870 > - name: src-volume
871 > mountPath: /usr/src/
872 > volumes:
873 > - name: hugepage
874 > emptyDir:
875 > medium: HugePages
876 > - name: dpdk-volume
877 > hostPath:
878 > path: /opt/dpdk-stable/
879 > - name: lib-volume
880 > hostPath:
881 > path: /lib/modules
882 > - name: src-volume
883 > hostPath:
884 > path: /usr/src/
885 > EOF
886
8873.4 Run the VF container
888````````````````````````
889
890Start the VF container.
891
892::
893
894 [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl create -f pod-with-dpdk-vfs-0.yaml
895 pod/pod-with-dpdk-vfs-0 created
896
897 [sysadmin@controller-0 sriov(keystone_admin)]$ kubectl get pod
898 NAME READY STATUS RESTARTS AGE
899 pod-with-dpdk-vfs-0 1/1 Running 0 6m40s
900
901Login the VF container
902
903::
904
905 kubectl exec -it pod-with-dpdk-vfs-0 -- bash
906
907Build the DPDK
908
909::
910
911 cd /lib/modules/5.0.19-rt11-yocto-preempt-rt/build
912
913 root@pod-with-dpdk-vfs-0:/lib/modules/5.0.19-rt11-yocto-preempt-rt/build# make prepare
914 HOSTCC scripts/basic/fixdep
915 HOSTCC scripts/kconfig/conf.o
916 HOSTCC scripts/kconfig/confdata.o
917 HOSTCC scripts/kconfig/expr.o
918 HOSTCC scripts/kconfig/symbol.o
919 HOSTCC scripts/kconfig/preprocess.o
920 HOSTCC scripts/kconfig/zconf.lex.o
921 HOSTCC scripts/kconfig/zconf.tab.o
922 HOSTLD scripts/kconfig/conf
923 scripts/kconfig/conf --syncconfig Kconfig
924 HOSTCC arch/x86/tools/relocs_32.o
925 HOSTCC arch/x86/tools/relocs_64.o
926 HOSTCC arch/x86/tools/relocs_common.o
927 HOSTLD arch/x86/tools/relocs
928 HOSTCC scripts/genksyms/genksyms.o
929 YACC scripts/genksyms/parse.tab.c
930 HOSTCC scripts/genksyms/parse.tab.o
931 LEX scripts/genksyms/lex.lex.c
932 YACC scripts/genksyms/parse.tab.h
933 HOSTCC scripts/genksyms/lex.lex.o
934 HOSTLD scripts/genksyms/genksyms
935 HOSTCC scripts/bin2c
936 HOSTCC scripts/kallsyms
937 HOSTCC scripts/conmakehash
938 HOSTCC scripts/recordmcount
939 HOSTCC scripts/sortextable
940 HOSTCC scripts/asn1_compiler
941 HOSTCC scripts/sign-file
942 HOSTCC scripts/extract-cert
943 CC scripts/mod/empty.o
944 HOSTCC scripts/mod/mk_elfconfig
945 MKELF scripts/mod/elfconfig.h
946 HOSTCC scripts/mod/modpost.o
947 CC scripts/mod/devicetable-offsets.s
948 UPD scripts/mod/devicetable-offsets.h
949 HOSTCC scripts/mod/file2alias.o
950 HOSTCC scripts/mod/sumversion.o
951 HOSTLD scripts/mod/modpost
952 CC kernel/bounds.s
953 CC arch/x86/kernel/asm-offsets.s
954 CALL scripts/checksyscalls.sh
955
956Build the test_pmd application
957
958::
959
960 cd $RTE_SDK
961 ./usertools/dpdk-setup.sh
962 Option: 14
963 CC config.o
964 CC iofwd.o
965 CC macfwd.o
966 CC macswap.o
967 CC flowgen.o
968 CC rxonly.o
969 CC txonly.o
970 CC csumonly.o
971 CC icmpecho.o
972 CC tm.o
973 LD testpmd
974 INSTALL-APP testpmd
975 INSTALL-MAP testpmd.map
976 == Build app/proc_info
977 CC main.o
978 LD dpdk-procinfo
979 INSTALL-APP dpdk-procinfo
980 INSTALL-MAP dpdk-procinfo.map
981 == Build app/pdump
982 CC main.o
983 LD dpdk-pdump
984 INSTALL-APP dpdk-pdump
985 INSTALL-MAP dpdk-pdump.map
986 == Build app/test-crypto-perf
987 CC main.o
988 CC cperf_ops.o
989 CC cperf_options_parsing.o
990 CC cperf_test_vectors.o
991 CC cperf_test_throughput.o
992 CC cperf_test_latency.o
993 CC cperf_test_pmd_cyclecount.o
994 CC cperf_test_verify.o
995 CC cperf_test_vector_parsing.o
996 CC cperf_test_common.o
997 LD dpdk-test-crypto-perf
998 INSTALL-APP dpdk-test-crypto-perf
999 INSTALL-MAP dpdk-test-crypto-perf.map
1000 == Build app/test-eventdev
1001 CC evt_main.o
1002 CC evt_options.o
1003 CC evt_test.o
1004 CC parser.o
1005 CC test_order_common.o
1006 CC test_order_queue.o
1007 CC test_order_atq.o
1008 CC test_perf_common.o
1009 CC test_perf_queue.o
1010 CC test_perf_atq.o
1011 LD dpdk-test-eventdev
1012 INSTALL-APP dpdk-test-eventdev
1013 INSTALL-MAP dpdk-test-eventdev.map
1014 Build complete [x86_64-native-linuxapp-gcc]
1015 Installation cannot run with T defined and DESTDIR undefined
1016 ------------------------------------------------------------------------------
1017 RTE_TARGET exported as x86_64-native-linuxapp-gcc
1018 ------------------------------------------------------------------------------
1019
1020 Press enter to continue ...
1021
1022Check the VF PCI information:
1023::
1024
1025 root@pod-with-dpdk-vfs-0:/usr/src/dpdk# printenv | grep PCIDEVICE_INTEL_COM
1026 PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET1=0000:05:11.1
1027 PCIDEVICE_INTEL_COM_PCI_SRIOV_NET_PHYSNET0=0000:05:11.0
1028
1029Exit from pod back to host to find which VFs are assigned to this pod by check the pci address:
1030::
1031
1032 [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f0/device/virtfn*
1033 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn0 -> ../0000:05:10.0
1034 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn1 -> ../0000:05:10.2
1035 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn2 -> ../0000:05:10.4
1036 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn3 -> ../0000:05:10.6
1037 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn4 -> ../0000:05:11.0
1038 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f0/device/virtfn5 -> ../0000:05:11.2
1039
1040 [root@controller-0 sysadmin(keystone_admin)]# ls -l /sys/class/net/ens2f1/device/virtfn*
1041 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn0 -> ../0000:05:10.1
1042 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn1 -> ../0000:05:10.3
1043 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn2 -> ../0000:05:10.5
1044 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn3 -> ../0000:05:10.7
1045 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn4 -> ../0000:05:11.1
1046 lrwxrwxrwx 1 root root 0 Jun 4 02:12 /sys/class/net/ens2f1/device/virtfn5 -> ../0000:05:11.3
1047
1048 [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f0 vf 4 mac 9e:fd:e6:dd:c1:01
1049 [root@controller-0 sysadmin(keystone_admin)]# sudo ip link set ens2f1 vf 4 mac 9e:fd:e6:dd:c1:02
1050
1051
1052Get back to pod and run the test_pmd
1053
1054::
1055
1056 cd $RTE_SDK/x86_64-native-linuxapp-gcc/app/
1057
1058 taskset -p --cpu-list 1
1059 pid 1's current affinity list: 2,3,18,19
1060
1061 ./testpmd --socket-mem 1024,1024 -l 2,3 -w 0000:05:11.0 -w 0000:05:11.1 --file-prefix=testpmd_ -- --auto-
1062 start --tx-first --stats-period 1 --disable-hw-vlan --eth-peer=0,"9e:fd:e6:dd:c1:02" --eth-
1063 peer=1,"9e:fd:e6:dd:c1:01"
1064 EAL: Detected 32 lcore(s)
1065 EAL: No free hugepages reported in hugepages-2048kB
1066 EAL: Probing VFIO support...
1067 EAL: VFIO support initialized
1068 EAL: PCI device 0000:05:11.0 on NUMA socket 0
1069 EAL: probe driver: 8086:10ed net_ixgbe_vf
1070 EAL: using IOMMU type 1 (Type 1)
1071 EAL: PCI device 0000:05:11.1 on NUMA socket 0
1072 EAL: probe driver: 8086:10ed net_ixgbe_vf
1073 Auto-start selected
1074 Ports to start sending a burst of packets first
1075 Warning: lsc_interrupt needs to be off when using tx_first. Disabling.
1076 USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=155456, size=2176, socket=0
1077 Configuring Port 0 (socket 0)
1078 Port 0: 9E:FD:E6:DD:C1:01
1079 Configuring Port 1 (socket 0)
1080 Port 1: 9E:FD:E6:DD:C1:02
1081 Checking link statuses...
1082 Port0 Link Up. speed 10000 Mbps- full-duplex
1083 Port1 Link Up. speed 10000 Mbps- full-duplex
1084 Done
1085 No commandline core given, start packet forwarding
1086 io packet forwarding - ports=2 - cores=1 - streams=2 - NUMA support enabled, MP over anonymous pages
1087 disabled
1088 Logical Core 3 (socket 0) forwards packets on 2 streams:
1089 RX P=0/Q=0 (socket 0) -> TX P=1/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:01
1090 RX P=1/Q=0 (socket 0) -> TX P=0/Q=0 (socket 0) peer=9E:FD:E6:DD:C1:02
1091
1092 io packet forwarding packets/burst=32
1093 nb forwarding cores=1 - nb forwarding ports=2
1094 port 0:
1095 CRC stripping enabled
1096 RX queues=1 - RX desc=128 - RX free threshold=32
1097 RX threshold registers: pthresh=8 hthresh=8 wthresh=0
1098 TX queues=1 - TX desc=512 - TX free threshold=32
1099 TX threshold registers: pthresh=32 hthresh=0 wthresh=0
1100 TX RS bit threshold=32 - TXQ flags=0xf01
1101 port 1:
1102 CRC stripping enabled
1103 RX queues=1 - RX desc=128 - RX free threshold=32
1104 RX threshold registers: pthresh=8 hthresh=8 wthresh=0
1105 TX queues=1 - TX desc=512 - TX free threshold=32
1106 TX threshold registers: pthresh=32 hthresh=0 wthresh=0
1107 TX RS bit threshold=32 - TXQ flags=0xf01
1108
1109 Port statistics ====================================
1110 ######################## NIC statistics for port 0 ########################
1111 RX-packets: 56 RX-missed: 0 RX-bytes: 4096
1112 RX-errors: 0
1113 RX-nombuf: 0
1114 TX-packets: 64 TX-errors: 0 TX-bytes: 4096
1115
1116 Throughput (since last show)
1117 Rx-pps: 0
1118 Tx-pps: 0
1119 ############################################################################
1120
1121 ######################## NIC statistics for port 1 ########################
1122 RX-packets: 432 RX-missed: 0 RX-bytes: 27712
1123 RX-errors: 0
1124 RX-nombuf: 0
1125 TX-packets: 461 TX-errors: 0 TX-bytes: 30080
1126
1127 Throughput (since last show)
1128 Rx-pps: 0
1129 Tx-pps: 0
1130 ############################################################################
1131
1132 Port statistics ====================================
1133 ######################## NIC statistics for port 0 ########################
1134 RX-packets: 14124641 RX-missed: 0 RX-bytes: 903977344
1135 RX-errors: 0
1136 RX-nombuf: 0
1137 TX-packets: 14170205 TX-errors: 0 TX-bytes: 906893376
1138
1139 Throughput (since last show)
1140 Rx-pps: 7068409
1141 Tx-pps: 7091206
1142 ############################################################################
1143
1144
1145
1146
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]1147
Bin Yange71c66a2020-03-27 04:03:43 +0000[diff] [blame]1148
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]1149References
1150----------
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]1151
1152- `StarlingX`_
Jackie Huang799759c2019-11-13 15:45:51 +0800[diff] [blame]1153
Xiaohua Zhang4cf6f2d2020-06-01 20:34:21 -0400[diff] [blame]1154.. _`StarlingX`: https://docs.starlingx.io/
1155