Packer: Add Infra build slave Ubuntu 20.04 image 01/12101/8
authorrobert.tomczyk <robert.tomczyk@est.tech>
Thu, 20 Jan 2022 16:10:42 +0000 (16:10 +0000)
committerRobert Tomczyk <robert.tomczyk@est.tech>
Fri, 21 Jan 2022 11:52:30 +0000 (11:52 +0000)
Packer image for Infra slave was based on the instruction on:
https://wiki.nordix.org/x/CYBY

Change-Id: I1622e031441d90b8fb99cfa00e3d09e886ac0a79

infra/jenkins/slave-setup/geode-build-server/configure-image.sh
infra/jenkins/slave-setup/infra-slave/README.md [new file with mode: 0644]
infra/jenkins/slave-setup/infra-slave/authorized_keys [new file with mode: 0644]
infra/jenkins/slave-setup/infra-slave/configure-image.sh [new file with mode: 0644]
infra/jenkins/slave-setup/infra-slave/infra-build-ubuntu2004.json [new file with mode: 0644]
infra/jenkins/slave-setup/infra-slave/known_hosts [new file with mode: 0644]

index 09133bf4f041189443e06279dd8990028cc78f30..aa09c2cd623fe069fbd8dd405b22b81d882d3f36 100644 (file)
@@ -142,7 +142,7 @@ sudo cat /home/ubuntu/.ssh/authorized_keys >> /home/infra/.ssh/authorized_keys
 sudo cat /home/ubuntu/authorized_keys.packer >> /home/infra/.ssh/authorized_keys
 # remove /home/ubuntu/authorized_keys.packer
 sudo rm -f /home/infra/authorized_keys.packer
-sudo chown -R infra:infra /home/jenkins/.ssh
+sudo chown -R infra:infra /home/infra/.ssh
 sudo chmod -R go-rwx /home/infra/.ssh
 sudo userdel -f -r ubuntu
 EOF'
diff --git a/infra/jenkins/slave-setup/infra-slave/README.md b/infra/jenkins/slave-setup/infra-slave/README.md
new file mode 100644 (file)
index 0000000..54b3b92
--- /dev/null
@@ -0,0 +1,10 @@
+## Description
+Packer files for creating virtual machine image used for general Infrastructure jenkins build server.
+Based on Ubuntu 20.04 image with Java & Docker installed ontop.
+
+## Steps To Run
+ 1. Source OpenStack RC file `source <openstack-rc-file.sh>`to set environment variables used by packer e.g. OS_USERNAME, OS_PASSWORD and OS_AUTH_URL
+ 2. Run `packer build <packerfile.json>`
+
+
+This will create the image specified (image_name) onto the OpenStack Server (identity_endpoint). Modify packer parameters as required and add packages to be installed into the shell script. For more info see [Packer docs](https://www.packer.io/docs/builders/openstack)
diff --git a/infra/jenkins/slave-setup/infra-slave/authorized_keys b/infra/jenkins/slave-setup/infra-slave/authorized_keys
new file mode 100644 (file)
index 0000000..741ee7e
--- /dev/null
@@ -0,0 +1,5 @@
+# injected by packer
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCdyLRXDIU4w/0H/kZa+2Fw/NLPTyW83F9cQqrahGybbff/pY3CxBKUqRefIp6SLjcR5TjrWOLVO6hlan+nzUzVahlgim8YIGYiD7l6ZuWBvlt/WpqqZOswRLKsgQgTDBUNFXl4V++bsZfbJwlv1QnNaeXGYdfGulXTnQ0wk4+/rBhPXqKVIPJyjnnrz1yCxwyRGOtB9hzSiG1VoKEoddgqIydRnxPQER7K5mc3E6CmvNr8FB5sbo+urT8EVLlb9Df8a7G0XRfMrM0z+1mFjlXG6ckvbIrlfJjQkeK00cXiFwwAmRQeHKZEQrl9++uhjcueIT0TYnGMW7ZW/Hl6NfV8z21pcj+erD3ltDXn3yZjlzf44ekvtNGegJ7hlB7mSdY17rk46QPSlPtPDeibZFj33c+jhZVFeB8PV7DOfXhqtNEVCZ/GlajQlgqbJqy1ILVftD/AhZpyEK0P6j8RiP7IF+rKVrdLsnoXPuHPMV+l+JKJE7STDHiCST0Opn5+WbQhiysDkORCfZMa+q02+/V4X6Khc97ws2LVJU19EQpqGSTYRew44E/cPGFMd3MvlyvzULV56XEtGESAYGAJdgj8g+zif8R4HjfgXo0KRgnz9yk3UU9qo2QZ1aqfu7+mO5r33cVk5ZVUIp3r0f7SD6OEOpj2nlc+cIaV6XRlF/BDYQ== infra@nordix.org
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLNT328BG7z9u5RynKtMPwA5tie64UY5lcD2HGydchCRcjceIt50CpadwSA3zJcHB3XCzRYJdDlU8PTfHjNw1VX7J589j6NKm5YDJn20Kx1/S43UbFqJm+QLXcRpqTFGoqeKgNw8d52saGVzeHlfK7SkgUyz0/jrtLSrMkepwpTVTqI8rKAmDFrfHCxFZAmAb61g3xVszZnxu9GMnF5M0uaiKwARAIL6Q7vrMJHU/LDPKGpxDumxF3fYyp4gQGCvoO3RhzQ36DuA95vXymwTTqy5oDiyp0/3GhoPlDH3807at7OfHzUCuNRGgUn+wS8T0pWov6MKzH+mTiiIoH/SsD id_rsa_city_nordix_jenkins
+ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEArXQug0RDmcbEWKcVnTbffD8jAgzm/lVhwG+Qdlz6iZZwEyLSU4nq6ymo2ukG7xTC8yBv5T0rjTtjQollFPc6CcM4I8aPnr8adI3ajDEseI4k/9bjr8+YrnXzLEszsoeYqCEqXz8gm1tbV1MMtxN3w7IUEdekAslFeMpJ8fEzFj8Ii5IiIGVJtz2mKdhReJEmqkW0u0UBQbg8L8n4YLj2wuW4V6hkDS/p96DJeBXb/wgB4u2bRWRFB45wlHyuVImeLxlMJTvWc6G1U7B7s/mcBLFX2Fis8brvMZDMiNPaUwWfYIJN+m2wzHhx0SkeboO6Svvcn7qe2qzgnDMOEUVQjQ== fdegir
+ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDiCJoG0x+wVM4T8GWMTHZobCDqnrDGEkyprNHrpUGchExhYUwu7B33zbz6F/A3ScpycBgiRTc47q+W8UB+cvfpol8CuvgmF7qU2E+/pe37Uee3ZDdXsxSg5rDp2ZDY1404/m1EKCmVMzBq0292KInsS7nWu9WsgPCKdalsHEBrWgAU3ft7FJX+5ybjMqhMZ3eXs6MAC1+ppLR6JhBPRInW5s1dYNUsQMKzHZvVmVzwPsa7KZEOKfPF0IKZiGCFsWzoLiRY60tmL0r/C1HXigUuHj0vITBePKx/YV06VLbzUffH+Lpa1asA5Rm27BcQ4ucIYucdv9x/+cnum8yoxHsAmgK+PvnqPk0qa7vRqcs7crTxgNyk3fRhttg4Sq+m1o4NQncfQW+JJNO7c8gPxawZ63onPMPUnAE3WNxu8N1Z2UPtMoAwicIyldggpt8oxmjC7B3OoxM2IwmoRNvW4x2Bq/VVjALFg9pRsoIrTp4040zhKaQLIkmaTnneE85ZsTTUcOJv66odf4e9gKcHA7J0BzwYNOuwlWmDnlqC7vQmyDc7N2FoFOBYVUem4KdTvNsBG/gFpglA7NRqD0REcfNMMdxi9pltW7gbowhBD5L0rBqXDYlROhBg3jWuwTvKOmcRrCROgS0zlZxLqO7vK3c39XMpuZdsOCP2OrQORoxFaw== Robert-Nordix-City-ssh
\ No newline at end of file
diff --git a/infra/jenkins/slave-setup/infra-slave/configure-image.sh b/infra/jenkins/slave-setup/infra-slave/configure-image.sh
new file mode 100644 (file)
index 0000000..19242f5
--- /dev/null
@@ -0,0 +1,138 @@
+#!/bin/bash
+# ============LICENSE_START=======================================================
+#  Copyright (C) 2022 The Nordix Foundation. All rights reserved.
+# ================================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# SPDX-License-Identifier: Apache-2.0
+# ============LICENSE_END=========================================================
+set -o nounset
+set -o errexit
+set -o pipefail
+#set -o xtrace
+DPKG_LOCK="/var/lib/dpkg/lock-frontend"
+
+# 3PP versions
+DOCKER_VERSION="5:20.10.11~3-0~ubuntu-focal"
+
+# Wait for other apt process to finish by checking the dpkg lock file.
+try=0
+while sudo lsof ${DPKG_LOCK} >/dev/null 2>&1; do
+  echo "DPKG file locked: ${DPKG_LOCK}."
+  echo "   Waiting for another pkg instalaltion process to finish ..."
+  sleep 10
+  if [[ ${try} -gt 60 ]]; then
+    echo "ERROR: Max number of re-tries reached, exiting..."
+    exit 1
+  fi
+  try=$((try + 1))
+done
+
+# list of basic packages to install
+PKG_LIST=(
+    apt-utils
+    apt-transport-https
+    ca-certificates
+    gnupg-agent
+    software-properties-common
+    git
+    vim
+    curl
+    wget
+    zip
+    unzip
+    chrony
+    curl
+    jq
+    docker-ce=${DOCKER_VERSION}
+    docker-ce-cli=${DOCKER_VERSION}
+    containerd.io
+    openjdk-11-jre
+    openjdk-11-jdk
+)
+# we need apt to proceed without any prompt asking for user input
+export DEBIAN_FRONTEND=noninteractive
+
+echo "Info  : Install packages"
+# update packages to their latest
+sudo -H -E apt update
+sudo -H -E apt upgrade -y
+
+# install packages
+sudo apt remove -y docker docker.io containerd runc
+curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
+sudo add-apt-repository \
+  "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
+   $(lsb_release -cs) \
+   stable"
+sudo apt-get update
+sudo -H -E apt -y -q=3 --no-install-recommends install "${PKG_LIST[@]}"
+
+# If you have a issue with Let's Encrypt certificate when cloning repo due to DST Root CA X3 Expiration:
+# https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/
+# remove outdated certificate from system
+sudo rm -rf /usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt
+# update ca-certificates
+sudo update-ca-certificates --fresh --verbose
+
+# Enable and start docker
+docker --version
+sudo systemctl enable docker
+sudo systemctl start docker
+
+# remove unnecessary packages
+sudo -H -E apt autoremove -y
+
+echo "Info  : Enable time sync"
+# ensure time sync is setup
+sudo systemctl enable chrony --now
+sudo chronyc -a 'burst 4/4' && sudo chronyc -a makestep
+
+echo "Info  : Create and configure infra user"
+# create and configure infra user
+sudo useradd -G sudo,docker -d /home/infra -m -c "infra user" -s /bin/bash infra
+sudo mkdir -p /home/infra/nordix/slave_root
+sudo chown -R infra:infra /home/infra/nordix/slave_root
+sudo chmod -R 755 /home/infra/nordix/slave_root
+
+# Modify sudoers - disable env_reset, !requiretty and passwordless sudo
+sudo sed -i "s/^Defaults.*env_reset/#&\nDefaults:infra  \!requiretty/" /etc/sudoers
+sudo sed -i "s/^%sudo.*ALL/%sudo   ALL=(ALL:ALL)   NOPASSWD: ALL/" /etc/sudoers
+
+# Disable ssh password login, enable ssh with keys for infra user
+sudo bash -c "echo PasswordAuthentication no >> /etc/ssh/sshd_config"
+sudo bash -c "echo PubkeyAuthentication yes >> /etc/ssh/sshd_config"
+sudo bash -c "echo AllowUsers infra >> /etc/ssh/sshd_config"
+sudo systemctl restart sshd
+
+echo "Info  : Create cloud-init script"
+# get cloud-init script in place so we can place the keys into ~infra/.ssh
+sudo bash -c 'cat << EOF > /var/lib/cloud/scripts/per-instance/copykeystojenkins.sh
+#!/bin/bash
+sudo mkdir -p /home/infra/.ssh
+# append ssh key injected by openstack to authorized_keys
+sudo cat /home/ubuntu/.ssh/authorized_keys >> /home/infra/.ssh/authorized_keys
+# append user ssh public keys uploaded by packer to authorized_keys
+sudo cat /home/ubuntu/authorized_keys.packer >> /home/infra/.ssh/authorized_keys
+# remove /home/ubuntu/authorized_keys.packer
+sudo rm -f /home/infra/authorized_keys.packer
+# append provided known hosts list uploaded by packer to known_hosts file
+sudo cat /home/ubuntu/known_hosts.packer >> /home/infra/.ssh/known_hosts
+# remove /home/ubuntu/known_hosts.packer
+sudo rm -f /home/infra/known_hosts.packer
+sudo chown -R infra:infra /home/infra/.ssh
+sudo chmod -R go-rwx /home/infra/.ssh
+sudo userdel -f -r ubuntu
+EOF'
+
+sudo chmod +x /var/lib/cloud/scripts/per-instance/copykeystojenkins.sh
diff --git a/infra/jenkins/slave-setup/infra-slave/infra-build-ubuntu2004.json b/infra/jenkins/slave-setup/infra-slave/infra-build-ubuntu2004.json
new file mode 100644 (file)
index 0000000..83018f5
--- /dev/null
@@ -0,0 +1,34 @@
+{
+    "builders": [
+        {
+            "type": "openstack",
+            "region": "Kna1",
+            "domain_name": "CCP_Domain_34201",
+            "identity_endpoint": "https://kna1.citycloud.com:5000",
+            "image_name": "infra-build-ubuntu2004",
+            "ssh_username": "ubuntu",
+            "source_image_name": "Ubuntu 20.04 Focal Fossa 20200423",
+            "flavor": "1C-2GB",
+            "networks": [
+                "e310fab3-cfef-4107-b40e-791e9c01903d"
+            ],
+            "floating_ip_network": "ext-net"
+        }
+    ],
+    "provisioners": [
+        {
+            "type": "file",
+            "source": "authorized_keys",
+            "destination": "/home/ubuntu/authorized_keys.packer"
+        },
+        {
+            "type": "file",
+            "source": "known_hosts",
+            "destination": "/home/ubuntu/known_hosts.packer"
+        },        
+        {
+            "script": "configure-image.sh",
+            "type": "shell"
+        }
+    ]
+}
diff --git a/infra/jenkins/slave-setup/infra-slave/known_hosts b/infra/jenkins/slave-setup/infra-slave/known_hosts
new file mode 100644 (file)
index 0000000..b465bbd
--- /dev/null
@@ -0,0 +1,3 @@
+|1|qfGuo10YedyByDV7P9l+ETnd8t8=|dekeSGUxLxeuXQqz+CS5l/lqR0k= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPZTnV/GUJsWzov52aYbxZ7WquyFeBGkpUw22LXFufBLscyxqaKzHdyq/BVyX34jAG6hvPHRNlBYhw/EDBniHgQ=
+|1|gQJtn2WkiyNXH1Or3T6dbo3b2tw=|ujHKJGo0S1p0JvK4+M7m7hoxRTI= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPZTnV/GUJsWzov52aYbxZ7WquyFeBGkpUw22LXFufBLscyxqaKzHdyq/BVyX34jAG6hvPHRNlBYhw/EDBniHgQ=
+|1|YHfD0kYGT0u3szNHpg3x2XI3I8A=|eEqL99y4AS/Y+Ctn84gjEDhRtFg= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPZTnV/GUJsWzov52aYbxZ7WquyFeBGkpUw22LXFufBLscyxqaKzHdyq/BVyX34jAG6hvPHRNlBYhw/EDBniHgQ=