blob: 762fb52dfdc465793d4774643d38d97d51bb97f5 [file] [log] [blame]
Petr Ospalýbe81ab02019-02-14 21:30:31 +01001.. This work is licensed under a Creative Commons Attribution 4.0 International License.
2.. http://creativecommons.org/licenses/by/4.0
3.. Copyright 2019 Samsung Electronics Co., Ltd.
4
5.. _oooi_installguide:
6
7OOM ONAP Offline Installer - Installation Guide
8===============================================
9
10This document describes the correct offline installation procedure for `OOM ONAP`_, which is done by the ansible based `offline-installer <https://gerrit.onap.org/r/#/admin/projects/oom/offline-installer>`_.
11
12Before you dive into the installation you should prepare the offline installer itself - the installer consists of at least two packages/resources. You can read about it in the `Build Guide`_, which provides the instructions for creating them.
13
Michal Ptacek26278df2019-07-09 10:46:26 +000014This current version of the *Installation Guide* supports `Dublin release`_.
Petr Ospalýbe81ab02019-02-14 21:30:31 +010015
16-----
17
18.. _oooi_installguide_preparations:
19
20Part 1. Prerequisites
21---------------------
22
Michal Ptacek26278df2019-07-09 10:46:26 +000023OOM ONAP deployment has certain hardware resource requirements - `Dublin requirements`_:
Petr Ospalýbe81ab02019-02-14 21:30:31 +010024
Michal Ptacek26278df2019-07-09 10:46:26 +000025Community recommended footprint from `Dublin requirements`_ page is 16 VMs ``224 GB RAM`` and ``112 vCPUs``. We will not follow strictly this setup due to such demanding resource consumption and so we will deploy our installation across four nodes (VMs) instead of sixteen. Our simplified setup is definitively not supported or recommended - you are free to diverge - you can follow the official guidelines or make completely different layout, but the minimal count of nodes should not drop below three - otherwise you may have to do some tweaking to make it work, which is not covered here (there is a pod count limit for a single kubernetes node - you can read more about it in this `discussion <https://lists.onap.org/g/onap-discuss/topic/oom_110_kubernetes_pod/25213556>`_).
Petr Ospalýbe81ab02019-02-14 21:30:31 +010026
27.. _oooi_installguide_preparations_k8s_cluster:
28
29Kubernetes cluster
30~~~~~~~~~~~~~~~~~~
31
32The four nodes/VMs will be running these services:
33
34- **infra-node**::
35
36 - nexus
37 - nginx proxy
38 - dns
Michal Ptacek26278df2019-07-09 10:46:26 +000039 - kubernetes-etcd
40 - kubernetes-control-plane
41
42**NOTE:** kubernetes-* roles can be collocated directly with kubernetes nodes and not necessarily on infra node.
Petr Ospalýbe81ab02019-02-14 21:30:31 +010043
44- **kubernetes node 1-3**::
45
Michal Ptacek26278df2019-07-09 10:46:26 +000046 - kubernetes worker
Petr Ospalýbe81ab02019-02-14 21:30:31 +010047
48You don't need to care about these services now - that is the responsibility of the installer (described below). Just start four VMs as seen in this table (or according to your needs as we hinted above):
49
50.. _Overview table of the kubernetes cluster:
51
52Kubernetes cluster overview
53^^^^^^^^^^^^^^^^^^^^^^^^^^^
54
Michal Ptacek26278df2019-07-09 10:46:26 +000055In Dublin we are using RKE as k8s orchestrator method, however everyone is free to diverge from this example and can set it up in own way omitting our rke playbook execution.
56
Petr Ospalýbe81ab02019-02-14 21:30:31 +010057=================== ========= ==================== ============== ============ ===============
58KUBERNETES NODE OS NETWORK CPU RAM STORAGE
59=================== ========= ==================== ============== ============ ===============
60**infra-node** RHEL 7 ``10.8.8.100/24`` ``8 vCPUs`` ``8 GB`` ``100 GB``
Michal Ptacek26278df2019-07-09 10:46:26 +000061**kube-node1** RHEL 7 ``10.8.8.101/24`` ``16 vCPUs`` ``56+ GB`` ``100 GB``
62**kube-node2** RHEL 7 ``10.8.8.102/24`` ``16 vCPUs`` ``56+ GB`` ``100 GB``
63**kube-node3** RHEL 7 ``10.8.8.103/24`` ``16 vCPUs`` ``56+ GB`` ``100 GB``
64SUM ``56 vCPUs`` ``176+ GB`` ``400 GB``
Petr Ospalýbe81ab02019-02-14 21:30:31 +010065================================================== ============== ============ ===============
66
67Unfortunately, the offline installer supports only **RHEL 7.x** distribution as of now. So, your VMs should be preinstalled with this operating system - the hypervisor and platform can be of your choosing. It is also worth knowing that the exact RHEL version (major and minor number - 7.6 for example) should match for the package build procedure and the target installation. That means: if you are building packages on RHEL 7.6 release your VMs should be RHEL 7.6 too.
68
69We will expect from now on that you installed four VMs and they are connected to the shared network. All VMs must be reachable from our *install-server* (below), which can be the hypervisor, *infra-node* or completely different machine. But in either of these cases the *install-server* must be able to connect over ssh to all of these nodes.
70
71.. _oooi_installguide_preparations_installserver:
72
73Install-server
74~~~~~~~~~~~~~~
75
76We will use distinct *install-server* and keep it separate from the four-node cluster. But if you wish so, you can use *infra-node* for this goal (if you use the default ``'chroot'`` option of the installer), but in that case double the size of the storage requirement!
77
78Prerequisites for the *install-server*:
79
80- packages described in `Build Guide`_
81- extra ``100 GB`` storage (to have space where to store these packages)
82- installed ``'chroot'`` and/or ``'docker'`` system commands
83- network connection to the nodes - especially functioning ssh client
84
85Our *install-server* will have ip: ``10.8.8.4``.
86
87**NOTE:** All the subsequent commands below, are executed from within this *install-server*.
88
89-----
90
91.. _oooi_installguide_config:
92
93Part 2. Preparation and configuration
94-------------------------------------
95
96We *MUST* do all the following instructions from the *install-server* and also we will be running them as a user ``root``. But that is not necessary - you can without any problem pick and use a regular user. The ssh/ansible connection to the nodes will also expect that we are connecting as a ``root`` - you need to elevate privileges to be able to install on them. Although it can be achieved by other means (sudo), we decided here to keep instructions simple.
97
98.. _oooi_installguide_config_packages:
99
100Installer packages
101~~~~~~~~~~~~~~~~~~
102
103As was stated above you must have prepared the installer packages (names will differ - check out the `Build Guide`_):
104
Michal Ptacek26278df2019-07-09 10:46:26 +0000105- sw_package.tar
106- resources_package.tar
107- aux_package.tar
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100108
Michal Ptacek26278df2019-07-09 10:46:26 +0000109**NOTE:** ``'aux_package.tar'`` is optional and if you don't have use for it, you can ignore it.
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100110
111We will store them in the ``/data`` directory on the *install-server* and then we will unpack the ``'sw'`` package to your home directory for example::
112
113 $ mkdir ~/onap-offline-installer
Michal Ptacek26278df2019-07-09 10:46:26 +0000114 $ tar -C ~/onap-offline-installer -xf /data/sw_package.tar
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100115
116.. _oooi_installguide_config_app:
117
118Application directory
119~~~~~~~~~~~~~~~~~~~~~
120
121Change the current directory to the ``'ansible'``::
122
123 $ cd ~/onap-offline-installer/ansible
124
125You can see multiple files and directories inside - this is the *offline-installer*. It is implemented as a set of ansible playbooks.
126
Bartek Grzybowski30b2cbf2019-03-26 16:10:10 +0100127If you created the ``'sw'`` package according to the *Build Guide* then you should have had the ``'application'`` directory populated with at least the following files:
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100128
129- ``application_configuration.yml``
130- ``hosts.yml``
131
132**NOTE:** The following paragraph describes a way how to create or fine-tune your own ``'application_configuration.yml'`` - we are discouraging you from executing this step. The recommended way is to use the packaged files inside the ``'application'`` directory.
133
134**NOT RECOMMENDED:** If for some reason you don't have these files inside the ``'application'`` directory or you simply want to do things the hard way then you can recreate them from their templates. It is better to keep the originals (templates) intact - so we will copy them to the ``'application'`` directory::
135
136 $ cp ../config/application_configuration.yml application/
137 $ cp inventory/hosts.yml application/
138
139.. _oooi_installguide_config_hosts:
140
141hosts.yml
142~~~~~~~~~
143
144We need to setup the ``'hosts.yml'`` first, the template looks like this::
145
146 ---
147 # This group contains hosts with all resources (binaries, packages, etc.)
148 # in tarball.
149 all:
150 vars:
151 # this key is supposed to be generated during setup.yml playbook execution
152 # change it just when you have better one working for all nodes
153 ansible_ssh_private_key_file: /root/.ssh/offline_ssh_key
154 ansible_ssh_common_args: '-o StrictHostKeyChecking=no'
155
156 children:
157 resources:
158 hosts:
159 resource-host:
160 ansible_host: 10.8.8.5
161
162 # This is group of hosts where nexus, nginx, dns and all other required
163 # services are running.
164 infrastructure:
165 hosts:
166 infrastructure-server:
167 ansible_host: 10.8.8.13
168 #IP used for communication between infra and kubernetes nodes, must be specified.
169 cluster_ip: 10.8.8.13
170
171 # This is group of hosts which are/will be part of Kubernetes cluster.
172 kubernetes:
Michal Zeganf9ef0c12019-06-25 11:05:05 +0200173 children:
174 # This is a group of hosts containing kubernetes worker nodes.
175 kubernetes-node:
176 hosts:
177 kubernetes-node-1:
178 ansible_host: 10.8.8.19
179 #ip of the node that it uses for communication with k8s cluster.
180 cluster_ip: 10.8.8.19
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100181
Michal Zeganf9ef0c12019-06-25 11:05:05 +0200182 # Group of hosts containing etcd cluster nodes.
183 # Defaults to infra.
184 kubernetes-etcd:
185 hosts:
186 infrastructure-server
187
188 # This is a group of hosts that are to be used as kubernetes control plane nodes.
189 # This means they host kubernetes api server, controller manager and scheduler.
190 # This example uses infra for this purpose, however note that any
191 # other host could be used including kubernetes nodes.
192 # cluster_ip needs to be set for hosts used as control planes.
193 kubernetes-control-plane:
194 hosts:
195 infrastructure-server
Bartek Grzybowskicf6797c2019-05-22 14:53:31 +0200196
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100197 nfs-server:
198 hosts:
199 kubernetes-node-1
200
201There is some ssh configuration under the ``'vars'`` section - we will deal with ssh setup a little bit later in the `SSH authentication`_.
202
203We need to first correct the ip addresses and add a couple of kubernetes nodes to match our four-node cluster:
204
205- Under the ``'resource-host'`` set the ``'ansible_host'`` address to the ip of your server, where the packages are stored - it must be reachable by ssh from the *install-server* (for ansible to run playbooks on it) **AND** *infra-node* (to extract resource data from *resource-host* to *infra-node* over ssh). In our scenario the *resource-host* is the same as the *install-server*: ``'10.8.8.4'``
206- Similarly, set the ``'ansible_host'`` to the address of the *infra-node* under the ``'infrastructure-server'``.
207- Copy the whole ``'kubernetes-node-1'`` subsection and paste it twice directly after. Change the numbers to ``'kubernetes-node-2'`` and ``'kubernetes-node-3'`` respectively and fix the addresses in the ``'ansible_host'`` variables again to match *kube-node1*, *kube-node2* and *kube-node3*.
208
209As you can see, there is another ``'cluster_ip'`` variable for each node - this serve as a designated node address in the kubernetes cluster. Make it the same as the respective ``'ansible_host'``.
210
211**NOTE:** In our simple setup we have only one interface per node, but that does not need to be a case for some other deployment - especially if we start to deal with a production usage. Basically, an ``'ansible_host'`` is an entry point for the *install-server's* ansible (*offline-installer*), but the kubernetes cluster can be communicating on a separate network to which *install-server* has no access. That is why we have this distinctive variable, so we can tell the installer that there is a different network, where we want to run the kubernetes traffic and what address each node has on such a network.
212
213After all the changes, the ``'hosts.yml'`` should look similar to this::
214
215 ---
216 # This group contains hosts with all resources (binaries, packages, etc.)
217 # in tarball.
218 all:
219 vars:
220 # this key is supposed to be generated during setup.yml playbook execution
221 # change it just when you have better one working for all nodes
222 ansible_ssh_private_key_file: /root/.ssh/offline_ssh_key
223 ansible_ssh_common_args: '-o StrictHostKeyChecking=no'
224
225 children:
226 resources:
227 hosts:
228 resource-host:
229 ansible_host: 10.8.8.4
230
231 # This is group of hosts where nexus, nginx, dns and all other required
232 # services are running.
233 infrastructure:
234 hosts:
235 infrastructure-server:
Michal Zeganf9ef0c12019-06-25 11:05:05 +0200236 ansible_host: 10.8.8.13
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100237 #IP used for communication between infra and kubernetes nodes, must be specified.
238 cluster_ip: 10.8.8.100
239
240 # This is group of hosts which are/will be part of Kubernetes cluster.
241 kubernetes:
Michal Zeganf9ef0c12019-06-25 11:05:05 +0200242 children:
243 # This is a group of hosts containing kubernetes worker nodes.
244 kubernetes-node:
245 hosts:
246 kubernetes-node-1:
247 ansible_host: 10.8.8.101
248 #ip of the node that it uses for communication with k8s cluster.
249 cluster_ip: 10.8.8.101
250 kubernetes-node-2:
251 ansible_host: 10.8.8.102
252 #ip of the node that it uses for communication with k8s cluster.
253 cluster_ip: 10.8.8.102
254 kubernetes-node-3:
255 ansible_host: 10.8.8.103
256 #ip of the node that it uses for communication with k8s cluster.
257 cluster_ip: 10.8.8.103
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100258
Michal Zeganf9ef0c12019-06-25 11:05:05 +0200259 # Group of hosts containing etcd cluster nodes.
260 # Defaults to infra.
261 kubernetes-etcd:
262 hosts:
263 infrastructure-server
264
265 # This is a group of hosts that are to be used as kubernetes control plane nodes.
266 # This means they host kubernetes api server, controller manager and scheduler.
267 # This example uses infra for this purpose, however note that any
268 # other host could be used including kubernetes nodes.
269 # cluster_ip needs to be set for hosts used as control planes.
270 kubernetes-control-plane:
271 hosts:
272 infrastructure-server
Bartek Grzybowskicf6797c2019-05-22 14:53:31 +0200273
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100274 nfs-server:
275 hosts:
276 kubernetes-node-1
277
278.. _oooi_installguide_config_appconfig:
279
280application_configuration.yml
281~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
282
283Here, we will be interested in the following variables:
284
285- ``resources_dir``
286- ``resources_filename``
287- ``aux_resources_filename``
288- ``app_data_path``
289- ``aux_data_path``
290- ``app_name``
Bartek Grzybowski30b2cbf2019-03-26 16:10:10 +0100291- ``timesync``
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100292
293``'resource_dir'``, ``'resources_filename'`` and ``'aux_resources_filename'`` must correspond to the file paths on the *resource-host* (variable ``'resource_host'``), which is in our case the *install-server*.
294
Michal Ptacek26278df2019-07-09 10:46:26 +0000295The ``'resource_dir'`` should be set to ``'/data'``, ``'resources_filename'`` to ``'resources_package.tar'`` and ``'aux_resources_filename'`` to ``'aux_package.tar'``. The values should be the same as are in the `Installer packages`_ section.
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100296
Michal Ptacek26278df2019-07-09 10:46:26 +0000297``'app_data_path'`` is the absolute path on the *infra-node* to where the package ``'resources_package.tar'`` will be extracted and similarly ``'aux_data_path'`` is another absolute path for ``'aux_package.tar'``. Both the paths are fully arbitrary, but they should point to the filesystem with enough space - the storage requirement in `Overview table of the kubernetes cluster`_.
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100298
299**NOTE:** As we mentioned in `Installer packages`_ - the auxiliary package is not mandatory and we will not utilize it in here either.
300
Bartek Grzybowski30b2cbf2019-03-26 16:10:10 +0100301The ``'app_name'`` variable should be short and descriptive. We will set it simply to: ``onap``.
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100302
Bartek Grzybowski30b2cbf2019-03-26 16:10:10 +0100303The ``'timesync'`` variable is optional and controls synchronisation of the system clock on hosts. It should be configured only if a custom NTP server is available and needed. Such a time authority should be on a host reachable from all installation nodes. If this setting is not provided then the default behavior is to setup NTP daemon on infra-node and sync all kube-nodes' time with it.
304
305If you wish to provide your own NTP servers configure their IPs as follows::
306
307 timesync:
308 servers:
309 - <ip address of NTP_1>
310 - <...>
311 - <ip address of NTP_N>
312
313Another time adjustment related variables are ``'timesync.slewclock'`` and ``'timesync.timezone'`` .
314First one can have value of ``'true'`` or ``'false'`` (default). It controls whether (in case of big time difference compared to server) time should be adjusted gradually by slowing down or speeding up the clock as required (``'true'``) or in one step (``'false'``)::
315
316 timesync:
317 slewclock: true
318
319Second one controls time zone setting on host. It's value should be time zone name according to tz database names with ``'Universal'`` being the default one::
320
321 timesync.
322 timezone: UTC
323
324``'timesync.servers'``, ``'timesync.slewclock'`` and ``'timesync.timezone'`` settings can be used independently.
325
326Final configuration can resemble the following::
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100327
328 resources_dir: /data
Michal Ptacek26278df2019-07-09 10:46:26 +0000329 resources_filename: resources-package.tar
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100330 app_data_path: /opt/onap
331 app_name: onap
Bartek Grzybowski30b2cbf2019-03-26 16:10:10 +0100332 timesync:
333 servers:
334 - 192.168.0.1
335 - 192.168.0.2
336 slewclock: true
337 timezone: UTC
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100338
Michal Zegana579c982019-04-02 15:33:30 +0200339.. _oooi_installguide_config_appconfig_overrides:
340
341Helm chart value overrides
342^^^^^^^^^^^^^^^^^^^^^^^^^^
343
Michal Ptacek26278df2019-07-09 10:46:26 +0000344In Dublin OOM charts are coming with all ONAP components disabled, this setting is also prepackaged within our sw_package.tar. Luckily there are multiple ways supported how to override this setting. It's also necessary for setting-up VIM specific entries and basically to configure any stuff with non default values.
345
346First option is to use ``overrides`` key in ``application_configuration.yml``.
Michal Zegana579c982019-04-02 15:33:30 +0200347These settings will override helm values originally stored in ``values.yaml`` files in helm chart directories.
348
349For example, the following lines could be appended to ``application_configuration.yml`` to set up managed openstack credentials for onap's so component::
350
351 overrides:
352 so:
353 config:
354 openStackUserName: "os_user"
355 openStackRegion: "region_name"
356 openStackKeyStoneUrl: "keystone_url"
357 openStackEncryptedPasswordHere: "encrypted_password"
358
Michal Ptacek26278df2019-07-09 10:46:26 +0000359In addition or alternatively to that one can configure ``helm_override_files`` key, which is new feature implemented in Change-Id: I8b8ded38b39aa9a75e55fc63fa0e11b986556cb8.
360
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100361.. _oooi_installguide_config_ssh:
362
363SSH authentication
364~~~~~~~~~~~~~~~~~~
365
366We are almost finished with the configuration and we are close to start the installation, but we need to setup password-less login from *install-server* to the nodes.
367
368You can use the ansible playbook ``'setup.yml'`` like this::
369
370 $ ./run_playbook.sh -i application/hosts.yml setup.yml -u root --ask-pass
371
372You will be asked for password per each node and the playbook will generate a unprotected ssh key-pair ``'~/.ssh/offline_ssh_key'``, which will be distributed to the nodes.
373
374Another option is to generate a ssh key-pair manually. We strongly advise you to protect it with a passphrase, but for simplicity we will showcase generating of a private key without any such protection::
375
376 $ ssh-keygen -N "" -f ~/.ssh/identity
377
378The next step will be to distribute the public key to these nodes and from that point no password is needed::
379
380 $ for ip in 100 101 102 103 ; do ssh-copy-id -i ~/.ssh/identity.pub root@10.8.8.${ip} ; done
381
382This command behaves almost identically to the ``'setup.yml'`` playbook.
383
384If you generated the ssh key manually then you can now run the ``'setup.yml'`` playbook like this and achieve the same result as in the first execution::
385
386 $ ./run_playbook.sh -i application/hosts.yml setup.yml
387
388This time it should not ask you for any password - of course this is very redundant, because you just distributed two ssh keys for no good reason.
389
390We can finally edit and finish the configuration of the ``'hosts.yml'``:
391
392- if you used the ``'setup.yml'`` playbook then you can just leave this line as it is::
393
394 ansible_ssh_private_key_file: /root/.ssh/offline_ssh_key
395
396- if you created a ssh key manually then change it like this::
397
398 ansible_ssh_private_key_file: /root/.ssh/identity
399
400-----
401
402.. _oooi_installguide_install:
403
404Part 3. Installation
405--------------------
406
407We should have the configuration complete and be ready to start the installation. The installation is done via ansible playbooks, which are run either inside a **chroot** environment (default) or from the **docker** container. If for some reason you want to run playbooks from the docker instead of chroot then you cannot use *infra-node* or any other *kube-node* as the *install-server* - otherwise you risk that installation will fail due to restarting of the docker service.
408
409If you built your ``'sw'`` package well then there should be the file ``'ansible_chroot.tgz'`` inside the ``'docker'`` directory. If not then you must create it - to learn how to do that and to get more info about the scripts dealing with docker and chroot, go to `Appendix 1. Ansible execution/bootstrap`_
410
411We will use the default chroot option so we don't need any docker service to be running.
412
413Installation is actually very straightforward now::
414
415 $ ./run_playbook.sh -i application/hosts.yml -e @application/application_configuration.yml site.yml
416
417This will take a while so be patient.
418
419``'site.yml'`` playbook actually runs in the order the following playbooks:
420
421- ``upload_resources.yml``
422- ``infrastructure.yml``
Bartek Grzybowskicf6797c2019-05-22 14:53:31 +0200423- ``rke.yml``
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100424- ``application.yml``
425
Michal Ptacekc424cff2019-03-06 16:25:43 +0000426----
427
428.. _oooi_installguide_postinstall:
429
Bartek Grzybowski32bf2fb2019-05-30 11:52:40 +0200430Part 4. Post-installation and troubleshooting
431---------------------------------------------
Michal Ptacekc424cff2019-03-06 16:25:43 +0000432
Bartek Grzybowski32bf2fb2019-05-30 11:52:40 +0200433After all of the playbooks are run successfully, it will still take a lot of time until all pods are up and running. You can monitor your newly created kubernetes cluster for example like this::
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100434
435 $ ssh -i ~/.ssh/offline_ssh_key root@10.8.8.4 # tailor this command to connect to your infra-node
436 $ watch -d -n 5 'kubectl get pods --all-namespaces'
437
Bartek Grzybowski32bf2fb2019-05-30 11:52:40 +0200438Alternatively you can monitor progress with ``helm_deployment_status.py`` script located in offline-installer directory. Transfer it to infra-node and run::
Milan Verespej1a230472019-03-20 13:51:40 +0100439
440 $ python helm_deployment_status.py -n <namespace_name> # namespace defaults to onap
441
Bartek Grzybowski32bf2fb2019-05-30 11:52:40 +0200442To automatically verify functionality with healthchecks after deployment becomes ready or after timeout period expires, append ``-hp`` switch followed by the full path to the healthcheck script and ``--health-mode`` optional switch with appropriate mode supported by that script (``health`` by default, ``--help`` displays available modes)::
Milan Verespej1a230472019-03-20 13:51:40 +0100443
Bartek Grzybowski32bf2fb2019-05-30 11:52:40 +0200444 $ python helm_deployment_status.py -hp <app_data_path>/<app_name>/helm_charts/robot/ete-k8s.sh --health-mode <healthcheck mode>
Milan Verespej1a230472019-03-20 13:51:40 +0100445
Bartek Grzybowski32bf2fb2019-05-30 11:52:40 +0200446It is strongly recommended to tailor ``helm_deployment_status.py`` to your needs since default values might not be what you'd expect. The defaults can be displayed with ``--help`` switch.
Michal Ptacekc424cff2019-03-06 16:25:43 +0000447
448Final result of installation varies based on number of k8s nodes used and distribution of pods. In some dev envs we quite frequently hit problems with not all pods properly deployed. In successful deployments all jobs should be in successful state.
449This can be verified using ::
450
451 $ kubectl get jobs -n <namespace>
452
453If some of the job is hanging in some wrong end-state like ``'BackoffLimitExceeded'`` manual intervention is required to heal this and make also dependent jobs passing. More details about particular job state can be obtained using ::
454
455 $ kubectl describe job -n <namespace> <job_name>
456
457If manual intervention is required, one can remove failing job and retry helm install command directly, which will not launch full deployment but rather check current state of the system and rebuild parts which are not up & running. Exact commands are as follows ::
458
459 $ kubectl delete job -n <namespace> <job_name>
460 $ helm deploy <env_name> <helm_chart_name> --namespace <namespace_name>
461
462 E.g. helm deploy dev local/onap --namespace onap
463
464Once all pods are properly deployed and in running state, one can verify functionality e.g. by running onap healthchecks ::
465
466 $ cd <app_data_path>/<app_name>/helm_charts/robot
467 $ ./ete-k8s.sh onap health
468
469
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100470-----
471
472.. _oooi_installguide_appendix1:
473
474Appendix 1. Ansible execution/bootstrap
475---------------------------------------
476
477There are two ways how to easily run the installer's ansible playbooks:
478
479- If you already have or can install a docker then you can build the provided ``'Dockerfile'`` for the ansible and run playbooks in the docker container.
480- Another way to deploy ansible is via chroot environment which is bundled together within this directory.
481
482(Re)build docker image and/or chroot archive
483~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
484
485Inside the ``'docker'`` directory is the ``'Dockerfile'`` and ``'build_ansible_image.sh'`` script. You can run ``'build_ansible_image.sh'`` script on some machine with the internet connectivity and it will download all required packages needed for building the ansible docker image and for exporting it into a flat chroot environment.
486
487Built image is exported into ``'ansible_chroot.tgz'`` archive in the same (``'docker'``) directory.
488
489This script has two optional arguments:
490
491#. ansible version
492#. docker image name
493
494**Note:** if optional arguments are not used, docker image name will be set to ``'ansible'`` by default.
495
496Launching ansible playbook using chroot environment
497~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
498
499This is the default and preferred way of running ansible playbooks in an offline environment as there is no dependency on docker to be installed on the system. Chroot environment is already provided by included archive ``'ansible_chroot.tgz'``.
500
501It should be available in the ``'docker'`` directory as the end-result of the packaging script or after manual run of the ``'build_ansible_image.sh'`` script referenced above.
502
503All playbooks can be executed via ``'./run_playbook.sh'`` wrapper script.
504
505To get more info about the way how the ``'./run_playbook.sh'`` wrapper script should be used, run::
506
507 $ ./run_playbook.sh
508
509The main purpose of this wrapper script is to provide the ansible framework to a machine where it was bootstrapped without need of installing additional packages. The user can run this to display ``'ansible-playbook'`` command help::
510
511 $ ./run_playbook.sh --help
512
513Developers notes
514~~~~~~~~~~~~~~~~
515
516* There are two scripts which work in tandem for creating and running chroot
517* First one can convert docker image into chroot directory
518* Second script will automate chrooting (necessary steps for chroot to work and cleanup)
519* Both of them have help - just run::
520
521 $ cd docker
522 $ ./create_docker_chroot.sh help
523 $ ./run_chroot.sh help
524
525Example usage::
526
527 $ sudo su
528 $ docker/create_docker_chroot.sh convert some_docker_image ./new_name_for_chroot
529 $ cat ./new_name_for_chroot/README.md
530 $ docker/run_chroot.sh execute ./new_name_for_chroot cat /etc/os-release 2>/dev/null
531
532Launching ansible playbook using docker container (ALTERNATIVE APPROACH)
533~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
534
535This option is here just to keep support for the older method which relies on a running docker service. For the offline deployment use the chroot option as indicated above.
536
537You will not need ``'ansible_chroot.tgz'`` archive anymore, but the new requirement is a prebuilt docker image of ansible (based on the provided ``'Dockerfile'``). It should be available in your local docker repository (otherwise the default name ``'ansible'`` may fetch unwanted image from default registry!).
538
539To trigger this functionality and to run ``'ansible-playbook'`` inside a docker container instead of the chroot environment, you must first set the ``ANSIBLE_DOCKER_IMAGE`` variable. The value must be a name of the built ansible docker image.
540
541Usage is basically the same as with the default chroot way - the only difference is the existence of the environment variable::
542
543 $ ANSIBLE_DOCKER_IMAGE=ansible ./run_playbook.sh --help
544
545-----
546
547.. _Build Guide: ./BuildGuide.rst
Michal Ptacek26278df2019-07-09 10:46:26 +0000548.. _Dublin requirements: https://onap.readthedocs.io/en/dublin/guides/onap-developer/settingup/index.html#installing-onap
549.. _Dublin release: https://docs.onap.org/en/dublin/release/
Petr Ospalýbe81ab02019-02-14 21:30:31 +0100550.. _OOM ONAP: https://wiki.onap.org/display/DW/ONAP+Operations+Manager+Project