Switch to packer for configuring cloud infra nodes 52/5652/3
authorFatih Degirmenci <fatih.degirmenci@est.tech>
Fri, 7 Aug 2020 08:16:15 +0000 (08:16 +0000)
committerFatih Degirmenci <fatih.degirmenci@est.tech>
Fri, 7 Aug 2020 14:22:35 +0000 (14:22 +0000)
This change switches to packer for building openstack image to be
used by cloud infra nodes on jenkins. It is based on existing script
to setup openstack instances manually and uds packer. [0][1]

In addition to basic configuration, the committed authorized_keys
file can be used for adding user ssh public keys to get them added
into /home/jenkins/.ssh/authorized_keys file via cloud-init script
when the instance boots.

[0] https://gerrit.nordix.org/gitweb?p=infra/tools.git;a=blob;f=infra/jenkins/slave-setup/vm-slave/cloud-infra-slave-setup.sh
[1] https://gerrit.nordix.org/gitweb?p=infra/tools.git;a=tree;f=infra/jenkins/slave-setup/uds-build-server

Change-Id: Ie1932d7098d7c6a0c52206dce785f34076d30881

infra/jenkins/slave-setup/cloud-infra/README.md [new file with mode: 0644]
infra/jenkins/slave-setup/cloud-infra/authorized_keys [new file with mode: 0644]
infra/jenkins/slave-setup/cloud-infra/cloud-infra-base-city-fra-ubuntu1804.json [new file with mode: 0644]
infra/jenkins/slave-setup/cloud-infra/configure-image-ubuntu1804.sh [new file with mode: 0644]

diff --git a/infra/jenkins/slave-setup/cloud-infra/README.md b/infra/jenkins/slave-setup/cloud-infra/README.md
new file mode 100644 (file)
index 0000000..869c1d4
--- /dev/null
@@ -0,0 +1,15 @@
+The packer file and corresponding script in this folder are used
+for building OpenStack images to be used for Cloud Infra jobs on
+Nordix Jenkins.
+
+The packer file is specific to City Cloud Frankfurt region since
+it contains the Frankfurt region endpoint and UUID of the network
+created there. For other regions or clouds, a similar file needs
+to be created. The actual script that is executed by packer on
+provisioned temporary instances should work on any Ubuntu1804
+regardless of cloud and region.
+
+A final note is that the file authorized_keys.packer is used for
+injecting user keys into image during cloud-init phase for ssh
+access to the slaves for troubleshooting. If you want to have
+access to the slaves, please add your ssh public key in it.
diff --git a/infra/jenkins/slave-setup/cloud-infra/authorized_keys b/infra/jenkins/slave-setup/cloud-infra/authorized_keys
new file mode 100644 (file)
index 0000000..f835e17
--- /dev/null
@@ -0,0 +1,4 @@
+# injected by packer
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCZt9Ge7X3ZT/bMnYRS/zaHrM94zp+dlWH5k4LdeCpMQu9PjI3VbOqCDgemaKgKVapKRpk6etu9+Sy+P4aWnRXZsAGHEmfSwcfVaRZ2LnEUKKwbvJ8SaaSopdhkfgNV5ztkh7E+3wHIEp+fsSIC/LRXGPvcbkfNlZko96Ihz4BW0ezaeF3ikINtHb9r/cV4zNONE5NJoNkjAAE7chAiaZ+tyca4Nonb8UsCWxHofjxWNAGsC9tyuIOYIBAdXQI8kpS7VarEHK2WyOmPxNPuJqy6yQ/MYpgkg1tY5x7XsKa/O0DNl5kdH3R3ECqQ7H+AHrL5UK0l87BFan6O9xJTpb0p jenkins
+ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEArXQug0RDmcbEWKcVnTbffD8jAgzm/lVhwG+Qdlz6iZZwEyLSU4nq6ymo2ukG7xTC8yBv5T0rjTtjQollFPc6CcM4I8aPnr8adI3ajDEseI4k/9bjr8+YrnXzLEszsoeYqCEqXz8gm1tbV1MMtxN3w7IUEdekAslFeMpJ8fEzFj8Ii5IiIGVJtz2mKdhReJEmqkW0u0UBQbg8L8n4YLj2wuW4V6hkDS/p96DJeBXb/wgB4u2bRWRFB45wlHyuVImeLxlMJTvWc6G1U7B7s/mcBLFX2Fis8brvMZDMiNPaUwWfYIJN+m2wzHhx0SkeboO6Svvcn7qe2qzgnDMOEUVQjQ== fdegir
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCuEIMUpo5XvG4ZI7R7EJCikN3bDD9fbYsohyvDsOLseyfBtCauK01KELDc0H+H9j2m+icBUGBmHoQeOpVFKE0jW6Y/3qDTl5UoYV9708BMCtj9j0N7CrwrBwkLbIVIWGkf0jC5HEip9/04PE/4GSiB2ORim6I1RjOiASB52FCO+J4rc6LiQbFL+RRiimcRajwuWR4msLpr+c/S2u3mnbSnityQJc0wCR/xZWS5poOuvF4oJ5vx3Vr8RspKxyN4ooLQYXEjVjWxYb7U3PUZ7Gbiu5hiMwm+X9qyit9RmE5f27vNoqssdSdfHQ5K84txbcfRDQ19bMO2gRA+IA0xEVgZ root@esy52-afrelalto-nfs
diff --git a/infra/jenkins/slave-setup/cloud-infra/cloud-infra-base-city-fra-ubuntu1804.json b/infra/jenkins/slave-setup/cloud-infra/cloud-infra-base-city-fra-ubuntu1804.json
new file mode 100644 (file)
index 0000000..392a92f
--- /dev/null
@@ -0,0 +1,27 @@
+{
+    "builders": [
+        {
+            "type": "openstack",
+            "identity_endpoint": "https://fra1.citycloud.com:5000",
+            "image_name": "cloud-infra-base-ubuntu1804",
+            "ssh_username": "ubuntu",
+            "source_image_name": "Ubuntu 18.04 Bionic Beaver 20190724.1",
+            "flavor": "2C-4GB-50GB",
+            "networks": [
+                "3703a0fe-b900-4553-92ae-2913bba32ae1"
+            ],
+            "floating_ip_network": "ext-net"
+        }
+    ],
+    "provisioners": [
+        {
+            "type": "file",
+            "source": "authorized_keys",
+            "destination": "/home/ubuntu/authorized_keys.packer"
+        },
+        {
+            "script": "configure-image-ubuntu1804.sh",
+            "type": "shell"
+        }
+    ]
+}
diff --git a/infra/jenkins/slave-setup/cloud-infra/configure-image-ubuntu1804.sh b/infra/jenkins/slave-setup/cloud-infra/configure-image-ubuntu1804.sh
new file mode 100644 (file)
index 0000000..9243b99
--- /dev/null
@@ -0,0 +1,112 @@
+#!/bin/bash
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2020 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+
+set -o nounset
+set -o pipefail
+
+# ensure apt is not running before proceeding with the rest
+echo "Info  : Wait for completion of an existing apt process before proceeding..."
+while true; do
+  pkg_mgr_process=$(pgrep -f apt | cat)
+  if [[ -n $pkg_mgr_process ]]; then
+    sleep 10
+  else
+    break
+  fi
+done
+echo "Info  : apt process done. Continuing..."
+
+# list of basic packages to install
+PKG_LIST=(
+    apt-utils
+    apt-transport-https
+    ca-certificates
+    gnupg-agent
+    software-properties-common
+    git
+    vim
+    curl
+    wget
+    chrony
+    openjdk-11-jre-headless
+)
+
+# we need apt to proceed without any prompt asking for user input
+export DEBIAN_FRONTEND=noninteractive
+
+echo "Info  : Install packages"
+# update packages to their latest
+sudo -H -E apt update
+sudo -H -E apt upgrade -y -q=3
+
+# install packages
+sudo -H -E apt -y -q=3 install ${PKG_LIST[@]}
+
+# remove unnecessary packages
+sudo -H -E apt autoremove -y
+
+echo "Info  : Enable time sync"
+# ensure time sync is setup
+sudo systemctl enable chrony --now
+sudo chronyc -a 'burst 4/4' && sudo chronyc -a makestep
+
+echo "Info  : Enable nested virtualization"
+# enable nested virtualization
+sudo bash -c 'cat << EOF > /etc/modprobe.d/qemu-system-x86.conf
+options kvm-intel nested=y enable_apicv=n
+EOF'
+sudo modprobe -r kvm_intel kvm
+sudo modprobe -a kvm_intel kvm
+sudo lsmod | grep kvm_intel
+sudo cat /sys/module/kvm_intel/parameters/nested
+
+echo "Info  : Create and configure jenkins user"
+# create and configure jenkins user
+sudo useradd -G sudo -d /home/jenkins -m -c "jenkins user" -s /bin/bash jenkins
+sudo mkdir -p /home/jenkins/nordix/slave_root
+sudo chown -R jenkins:jenkins /home/jenkins/nordix/slave_root
+sudo chmod -R 755 /home/jenkins/nordix/slave_root
+
+# modify sudoers - disable env_reset, !requiretty and passwordless sudo
+sudo sed -i "s/^Defaults.*env_reset/#&\nDefaults:jenkins  \!requiretty/" /etc/sudoers
+sudo sed -i "s/^%sudo.*ALL/%sudo   ALL=(ALL:ALL)   NOPASSWD: ALL/" /etc/sudoers
+
+# disable ssh password login, enable ssh with keys for jenkins user
+sudo bash -c "echo PasswordAuthentication no >> /etc/ssh/sshd_config"
+sudo bash -c "echo PubkeyAuthentication yes >> /etc/ssh/sshd_config"
+sudo bash -c "echo AllowUsers jenkins >> /etc/ssh/sshd_config"
+sudo systemctl restart sshd
+
+echo "Info  : Create cloud-init script"
+# get cloud-init script in place so we can place the keys into ~jenkins/.ssh
+sudo bash -c 'cat << EOF > /var/lib/cloud/scripts/per-instance/copykeystojenkins.sh
+#!/bin/bash
+sudo mkdir -p /home/jenkins/.ssh
+# append ssh key injected by openstack to authorized_keys
+sudo cat /home/ubuntu/.ssh/authorized_keys >> /home/jenkins/.ssh/authorized_keys
+# append user ssh public keys uploaded by packer to authorized_keys
+sudo cat /home/ubuntu/authorized_keys.packer >> /home/jenkins/.ssh/authorized_keys
+# remove /home/ubuntu/authorized_keys.packer
+sudo rm -f /home/jenkins/authorized_keys.packer
+sudo chown -R jenkins:jenkins /home/jenkins/.ssh
+sudo chmod -R go-rwx /home/jenkins/.ssh
+sudo userdel -f -r ubuntu
+EOF'
+
+sudo chmod +x /var/lib/cloud/scripts/per-instance/copykeystojenkins.sh