Introduce heat as new provisioner
This change introduces OpenStack Heat as new provisioner.
Notes about how it is expected to work.
- Below heat templates are created without parameter defaults
as they differ depending on OpenStack Cloud where the instances
are created and number of instances to be created.
- provisioner/heat/playbooks/roles/install-configure-heat/files/heat-template.yaml
- provisioner/heat/playbooks/roles/install-configure-heat/files/heat-server.yaml
- Parameters in provided heat-template.yaml are expected to be set by the user
provided heat environment file using argument -e <uri to heat environment file> as
they are specific to OpenStack Cloud and the scenario. If user does not provide
heat environment file, the engine will use the default environment file to set
the parameter values which is pretty conservative.
- provisioner/heat/playbooks/roles/install-configure-heat/files/heat-environment.yaml
- As the instances are provisioned from OpenStack Cloud and also due to
not having PDF and IDF for them, configure-network role is skipped
for these instances. This could be looked into in future when the need
arises.
Another thing to note is that even though the engine is run within Python virtualenv,
Ansible OpenStack module doesn't seem to respect to this and uses system Python. This
causes execution of engine with Heat as provisioner fail with below error.
fatal: [localhost]: FAILED! => {"changed": false, "msg": "openstacksdk is required for this module"}
This is due to that Ansible OpenStack module looks in system Python which doesn't have
the package openstacksdk installed since we install it in virtualenv.
This change explicitly sets ansible_python_interpreter to be the one that is in
in virtualenv.
Change-Id: I1d9e5c722d4a0fe4dfd14ef3fa06b1e01c44aabc
diff --git a/playbooks/main.yml b/playbooks/main.yml
new file mode 100644
index 0000000..81f3508
--- /dev/null
+++ b/playbooks/main.yml
@@ -0,0 +1,34 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+- hosts: localhost
+ connection: local
+ gather_facts: true
+ become: no
+ vars_files:
+ - "{{ engine_path }}/engine/var/versions.yml"
+ - "{{ engine_path }}/engine/var/global.yml"
+ vars:
+ ansible_python_interpreter: "/usr/bin/env python"
+
+ roles:
+ - role: install-configure-heat
+ - role: create-stack
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/create-stack/defaults/main.yml b/playbooks/roles/create-stack/defaults/main.yml
new file mode 100644
index 0000000..9182a57
--- /dev/null
+++ b/playbooks/roles/create-stack/defaults/main.yml
@@ -0,0 +1,21 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+stack_name: "{{ lookup('env', 'STACK_NAME') | default('nordix-onap-custom', true) }}"
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/create-stack/tasks/create-stack.yml b/playbooks/roles/create-stack/tasks/create-stack.yml
new file mode 100644
index 0000000..c6204cf
--- /dev/null
+++ b/playbooks/roles/create-stack/tasks/create-stack.yml
@@ -0,0 +1,50 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+- name: Delete stack '{{ stack_name }}'
+ os_stack:
+ name: "{{ stack_name }}"
+ state: absent
+ wait: yes
+ ignore_errors: no
+
+- name: Create stack '{{ stack_name }}'
+ os_stack:
+ name: "{{ stack_name }}"
+ state: present
+ template: "{{ engine_cache }}/config/heat-template.yaml"
+ environment:
+ - "{{ engine_cache }}/config/heat-environment.yaml"
+ wait: yes
+ ignore_errors: no
+ register: stack_create_output
+
+- name: Log stack create output to console
+ debug:
+ msg: "{{ stack_create_output }}"
+
+- name: Get stack list
+ shell: openstack stack list --property name={{ stack_name }}
+ register: stack_list_output
+
+- name: Log stack list output to console
+ debug:
+ msg: "{{ stack_list_output.stdout_lines }}"
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/create-stack/tasks/generate-kubespray-inventory.yml b/playbooks/roles/create-stack/tasks/generate-kubespray-inventory.yml
new file mode 100644
index 0000000..3a0a6b2
--- /dev/null
+++ b/playbooks/roles/create-stack/tasks/generate-kubespray-inventory.yml
@@ -0,0 +1,38 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+- name: Get IPs of master nodes
+ shell: "openstack stack output show {{ stack_name }} master_ip -c output_value -f json"
+ register: stack_output_master_ip
+
+- name: Get IPs of worker nodes
+ shell: "openstack stack output show {{ stack_name }} worker_ip -c output_value -f json"
+ register: stack_output_worker_ip
+
+- name: Register IPs of master and worker nodes
+ set_fact:
+ master_ip: "{{ stack_output_master_ip.stdout }}"
+ worker_ip: "{{ stack_output_worker_ip.stdout }}"
+
+- name: Generate kubespray inventory
+ template:
+ src: inventory_kubespray.ini.j2
+ dest: "{{ config_path }}/inventory.ini"
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/create-stack/tasks/main.yml b/playbooks/roles/create-stack/tasks/main.yml
new file mode 100644
index 0000000..7db242d
--- /dev/null
+++ b/playbooks/roles/create-stack/tasks/main.yml
@@ -0,0 +1,27 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+# create stack
+- include: create-stack.yml
+
+# generate kubespray inventory from heat output
+- include: generate-kubespray-inventory.yml
+ when: installer_type == 'kubespray'
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/create-stack/templates/inventory_kubespray.ini.j2 b/playbooks/roles/create-stack/templates/inventory_kubespray.ini.j2
new file mode 100644
index 0000000..4bfa005
--- /dev/null
+++ b/playbooks/roles/create-stack/templates/inventory_kubespray.ini.j2
@@ -0,0 +1,38 @@
+[all:vars]
+ansible_ssh_extra_args=' -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null'
+
+[baremetal]
+# order of nodes is not significant
+{% for ip in master_ip.output_value %}
+master{{loop.index0}} ansible_host={{ ip }} ip={{ ip }} ansible_become=yes ansible_become_user=root
+{% endfor %}
+{% for ip in worker_ip.output_value %}
+worker{{loop.index0}} ansible_host={{ ip }} ip={{ ip }} ansible_become=yes ansible_become_user=root
+{% endfor %}
+
+[localhost]
+127.0.0.1 ansible_connection=local ansible_python_interpreter={{ engine_venv }}/bin/python
+
+[kube-master]
+{% for ip in master_ip.output_value %}
+master{{loop.index0}}
+{% endfor %}
+
+[kube-node]
+{% for ip in worker_ip.output_value %}
+worker{{loop.index0}}
+{% endfor %}
+
+[etcd]
+{% for ip in master_ip.output_value %}
+master{{loop.index0}}
+{% endfor %}
+
+[vault]
+{% for ip in master_ip.output_value %}
+master{{loop.index0}}
+{% endfor %}
+
+[k8s-cluster:children]
+kube-master
+kube-node
diff --git a/playbooks/roles/install-configure-heat/files/heat-environment.yaml b/playbooks/roles/install-configure-heat/files/heat-environment.yaml
new file mode 100644
index 0000000..2e0a919
--- /dev/null
+++ b/playbooks/roles/install-configure-heat/files/heat-environment.yaml
@@ -0,0 +1,32 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+parameters:
+ no_of_master_nodes: 1
+ master_image: Ubuntu 18.04 Bionic Beaver
+ master_flavor: 4C-8GB-150GB
+ master_volume_size: 20
+ no_of_worker_nodes: 2
+ worker_image: Ubuntu 18.04 Bionic Beaver
+ worker_flavor: 4C-8GB-150GB
+ worker_volume_size: 20
+ keypair: fdegir-key
+ private_network: onap-net
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/install-configure-heat/files/heat-server.yaml b/playbooks/roles/install-configure-heat/files/heat-server.yaml
new file mode 100644
index 0000000..00b3381
--- /dev/null
+++ b/playbooks/roles/install-configure-heat/files/heat-server.yaml
@@ -0,0 +1,133 @@
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+heat_template_version: 2017-02-24
+
+description: Common template for the instances
+
+parameters:
+ # parameters for instances
+ instance_name:
+ type: string
+ label: Name
+ description: Instance name
+
+ image:
+ type: string
+ label: Image name or ID
+ description: Image to use for instances
+
+ flavor:
+ type: string
+ label: Flavor
+ description: Flavor to use for instances
+
+ volume_name:
+ type: string
+ label: Volume name
+ description: Volume name
+
+ volume_size:
+ type: string
+ label: Volume
+ description: Volume size to use for instances
+
+ keypair:
+ type: string
+ label: Key name
+ description: Keypair to usedfor the instances
+
+ private_network:
+ type: string
+ label: Private network name or ID
+ description: Network to attach instances to
+
+resources:
+ wait_condition:
+ type: OS::Heat::WaitCondition
+ properties:
+ handle: { get_resource: wait_handle }
+ count: 1
+ timeout: 600
+
+ wait_handle:
+ type: OS::Heat::WaitConditionHandle
+
+ root_login:
+ type: OS::Heat::CloudConfig
+ properties:
+ cloud_config:
+ disable_root: false
+
+ boot_script:
+ type: OS::Heat::SoftwareConfig
+ properties:
+ group: ungrouped
+ config:
+ str_replace:
+ params:
+ wc_notify: { get_attr: ['wait_handle', 'curl_cli'] }
+ template: |
+ #!/bin/bash -ex
+
+ echo "Running boot script"
+
+ # we need python for ansible
+ sudo apt update
+ sudo apt install -y python python-dev
+
+ # notify completion
+ wc_notify --data-binary '{"status": "SUCCESS"}'
+
+ boot_config:
+ type: OS::Heat::MultipartMime
+ properties:
+ parts:
+ - config: {get_resource: root_login}
+ - config: {get_resource: boot_script}
+
+ volume:
+ type: OS::Cinder::Volume
+ properties:
+ name: { get_param: volume_name }
+ size: { get_param: volume_size }
+
+ volume_attachment:
+ type: OS::Cinder::VolumeAttachment
+ properties:
+ volume_id: { get_resource: volume }
+ instance_uuid: { get_resource: instance }
+ mountpoint: /dev/vdb
+
+ instance:
+ type: OS::Nova::Server
+ properties:
+ name: { get_param: instance_name }
+ image: { get_param: image }
+ flavor: { get_param: flavor }
+ key_name: { get_param: keypair }
+ networks:
+ - network: { get_param: private_network }
+ user_data_format: SOFTWARE_CONFIG
+ user_data: { get_resource: boot_config }
+
+outputs:
+ instance_ip:
+ value: {get_attr: [instance, first_address]}
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/install-configure-heat/files/heat-template.yaml b/playbooks/roles/install-configure-heat/files/heat-template.yaml
new file mode 100644
index 0000000..5b0be0e
--- /dev/null
+++ b/playbooks/roles/install-configure-heat/files/heat-template.yaml
@@ -0,0 +1,135 @@
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+heat_template_version: 2017-02-24
+
+description: Kubespray cluster template to deploy ONAP
+
+parameters:
+ # parameters for k8s master instances
+ no_of_master_nodes:
+ type: number
+ label: No of k8s master nodes
+ description: Number of master nodes in k8s cluster
+
+ master_image:
+ type: string
+ label: Image name or ID
+ description: Image to use for k8s master instances
+
+ master_flavor:
+ type: string
+ label: Flavor
+ description: Flavor to use for k8s master instances
+
+ master_volume_size:
+ type: string
+ label: Volume
+ description: Volume size to use for k8s master instances
+
+ # parameters for k8s worker instances
+ no_of_worker_nodes:
+ type: number
+ label: No of k8s worker nodes
+ description: Number of worker nodes in k8s cluster
+
+ worker_image:
+ type: string
+ label: Image name or ID
+ description: Image to use for k8s worker instances
+
+ worker_flavor:
+ type: string
+ label: Flavor
+ description: Flavor to use for k8s worker instances
+
+ worker_volume_size:
+ type: string
+ label: Volume
+ description: Size of the volume to use for k8s worker instances
+
+ keypair:
+ type: string
+ label: Key name
+ description: Keypair to use for the instances
+
+ private_network:
+ type: string
+ label: Private network name or ID
+ description: Network to attach instances to
+
+resources:
+ name_nonce:
+ type: OS::Heat::RandomString
+ properties:
+ length: 8
+ sequence: lowercase
+
+ master_nodes:
+ type: OS::Heat::ResourceGroup
+ properties:
+ count: { get_param: no_of_master_nodes }
+ resource_def:
+ type: heat-server.yaml
+ properties:
+ instance_name:
+ str_replace:
+ template: master%index%-$NONCE
+ params:
+ $NONCE: { get_resource: name_nonce }
+ image: { get_param: master_image }
+ flavor: { get_param: master_flavor }
+ keypair: { get_param: keypair }
+ volume_name:
+ str_replace:
+ template: volume-master%index%-$NONCE
+ params:
+ $NONCE: { get_resource: name_nonce }
+ volume_size: { get_param: master_volume_size }
+ private_network: { get_param: private_network }
+
+ worker_nodes:
+ type: OS::Heat::ResourceGroup
+ properties:
+ count: { get_param: no_of_worker_nodes }
+ resource_def:
+ type: heat-server.yaml
+ properties:
+ instance_name:
+ str_replace:
+ template: worker%index%-$NONCE
+ params:
+ $NONCE: { get_resource: name_nonce }
+ image: { get_param: worker_image }
+ flavor: { get_param: worker_flavor }
+ keypair: { get_param: keypair }
+ volume_name:
+ str_replace:
+ template: volume-worker%index%-$NONCE
+ params:
+ $NONCE: { get_resource: name_nonce }
+ volume_size: { get_param: worker_volume_size }
+ private_network: { get_param: private_network }
+
+outputs:
+ master_ip:
+ value: {get_attr: [master_nodes, instance_ip]}
+ worker_ip:
+ value: {get_attr: [worker_nodes, instance_ip]}
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/playbooks/roles/install-configure-heat/tasks/main.yml b/playbooks/roles/install-configure-heat/tasks/main.yml
new file mode 100644
index 0000000..8d7fd90
--- /dev/null
+++ b/playbooks/roles/install-configure-heat/tasks/main.yml
@@ -0,0 +1,41 @@
+---
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+- name: Install required packages for Heat
+ pip:
+ name:
+ - openstacksdk
+ - python-openstackclient
+ - python-heatclient
+
+- name: Copy Heat templates
+ copy:
+ src: "{{ item }}"
+ dest: "{{ engine_cache }}/config/"
+ with_items:
+ - heat-template.yaml
+ - heat-server.yaml
+
+- name: Get Heat environment file
+ get_url:
+ url: "{{ heat_env_file }}"
+ dest: "{{ engine_cache }}/config/heat-environment.yaml"
+ mode: 0644
+
+# vim: set ts=2 sw=2 expandtab:
diff --git a/provision.sh b/provision.sh
new file mode 100755
index 0000000..f172d95
--- /dev/null
+++ b/provision.sh
@@ -0,0 +1,45 @@
+#!/bin/bash
+# ============LICENSE_START=======================================================
+# Copyright (C) 2019 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+PROVISIONER_ROOT_DIR="$(dirname $(realpath ${BASH_SOURCE[0]}))"
+export ANSIBLE_ROLES_PATH="$HOME/.ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles:${ENGINE_PATH}/engine/playbooks/roles:${ENGINE_CACHE}/repos/bifrost/playbooks/roles"
+export ANSIBLE_LIBRARY="$HOME/.ansible/plugins/modules:/usr/share/ansible/plugins/modules"
+
+# set the BAREMETAL variable
+export BAREMETAL=false
+
+# source openrc file
+source $OPENRC
+
+# create stack using the provided Heat Template
+echo "Info: Install, configure heat and create stack"
+echo "-------------------------------------------------------------------------"
+cd ${ENGINE_PATH}
+ansible-playbook ${ENGINE_ANSIBLE_PARAMS} \
+ -i localhost, \
+ ${PROVISIONER_ROOT_DIR}/playbooks/main.yml
+echo "-------------------------------------------------------------------------"
+echo "Info: Nodes are provisioned using OpenStack Heat!"
+echo "-------------------------------------------------------------------------"
+
+# vim: set ts=2 sw=2 expandtab: