Copyright 2019 AT&T Intellectual Property. All rights reserved.
This file is licensed under the CREATIVE COMMONS ATTRIBUTION 4.0 INTERNATIONAL LICENSE
Full license text at https://creativecommons.org/licenses/by/4.0/legalcode
ONAP on AKS will orchestrate an Azure Kubernetes Service (AKS) deployment, a DevStack deployment, an ONAP + NFS deployment, as well as configuration to link the Azure resources together. After ONAP is installed, a cloud region will also be added to ONAP with the new DevStack details that can be used to instantiate a VNF.
The following software is required to be installed:
apt-get update && apt-get install make openjdk-8-jre openjdk-8-jdk)Check the OOM Cloud Setup Guide for the versions of kubectl and helm to use.
After installing the above software, run az login and follow the instructions to finalize the azure command line installation. You'll need to be either an owner or co-owner of the azure subscription, or some of the deployment steps may not complete successfully. If you have multiple azure subscriptions, use az account set --subscription <subscription name> prior to running az login so that resources are deployed to the correct subscription. See the azure docs for more details on using the azure command line.
cloud.sh is the main driver script, and deploys a Kubernetes Cluster (AKS), DevStack, NFS, and bootstraps ONAP with configuration needed to instantiate a VNF. The script creates ONAP in "about" an hour.
$ ./cloud.sh --help ./cloud.sh [options] options: -f, --no-prompt executes with no prompt for confirmation -n, --no-install don't install ONAP -o, --override create integration override for robot configuration -d, --no-validate dont validate pre-reqs before executing deployment -p, --post-install execute post-install scripts -h, --help provide brief overview of script This script deploys a cloud environment in Azure. It: - Uses Azure Kubernetes Service (AKS) to bootstrap a kubernetes cluster. - Creates a VM to be used as NFS storage. - Creates a VM and installs DevStack, to be used with ONAP. - Creates an openstack cli pod that can be used for cli access to devstack - Creates an integration-override.yaml file to configure robot - Launches ONAP onto the AKS Cluster via OOM. - Configures Networking, SSH Access, and Security Group Rules
$ ./cloud.sh --override
This file contains the parameters that will be used when executing cloud.sh. The parameter BUILD will be generated at runtime.
For an example with all of the parameters filled out, check here. You can copy this and modify to suit your deployment. The parameters that MUST be modified from cloud.conf.example are USER_PUBLIC_IP_PREFIX and BUILD_DIR.
All other parameters will work out of the box, however you can also customize them to suit your own deployment. See below for a description of the available parameters and how they're used.
# The variable $BUILD will be generated dynamically when this file is sourced RANDOM_STRING=`cat /dev/urandom | env LC_CTYPE=C tr -cd 'a-zA-Z0-9' | head -c 4` BUILD= This is just a helper variable to create a random string to assign to various resources. Look at cloud.conf.example to see how it can be used. # GLOBAL PARAMS LOCATION= Location in Azure to deploy resources USER_PUBLIC_IP_PREFIX= Space delimited list of ip addresses/CIDR blocks that will be added to azure secuirty groups for access. Add the CIDR blocks to grant access for ssh, ONAP portal, and DevStack horizon access. BUILD_DIR= /path/to/directory where build files, artifacts, and other files will be created. # AKS PARAMS AKS_RESOURCE_GROUP_NAME= Name of resource group in azure that will be created for the AKS resource. Must not already exist. AKS_NAME= Name of AKS resource. AKS_K8_VERSION= Kubernetes version, use az aks get-versions --location <location> to see available versions. AKS_NODE_COUNT= Number of nodes that will comprise the AKS cluster. AKS_NODE_SIZE= Flavor to use for AKS nodes. AKS_VNET_NAME= Name of VNET that AKS nodes will attach to. AKS_DNS_PREFIX= DNS prefix that will be used by kubernetes dns service. AKS_POD_CIDR= CIDR used for pod ip allocation. AKS_NODE_CIDR= CIDR used for node ip allocation. AKS_SERVICE_CIDR= CIDR used for kubernetes service allocation. AKS_DNS_IP= IP address to assign to kubernetes dns service. Should be from AKS_SERVICE_CIDR range. AKS_ADMIN_USER= User name that will be created on AKS nodes. Use this user to ssh into AKS nodes if needed. # NFS PARAMS NFS_NAME= Name of NFS VM created in Azure. NFS_RG= Name of resource group that will be created in Azure for the NFS VM. Must not already exist. NFS_VM_SIZE= Flavor to use for NFS VM. NFS_LOCATION= Azure location to deploy NFS VM. NFS_CIDR= CIDR for NFS VNET. NFS_ADMIN_USER= User name that will be created on NFS VM. Use this to ssh to NFS VM if needed. NFS_VNET_NAME= Name of VNET that NFS VM will attach to. NFS_SUBNET_NAME= Name of SUBNET attached to NFS_VNET_NAME. NFS_DISK_SIZE= Size of OS Disk for NFS VM. # DEVSTACK PARAMS DEVSTACK_NAME= Name of DevStack VM created in Azure. DEVSTACK_RG= Name of resource group that will be created in Azure for the DevStack VM. Must not already exist. DEVSTACK_VM_SIZE= Flavor to use for DevStack VM. DEVSTACK_LOCATION= Azure location to deploy DevStack VM. DEVSTACK_CIDR= CIDR for DevStack VNET. DEVSTACK_PRIVATE_IP= IP to allocate to DevStack VM. This should be from DEVSTACK_CIDR range, and will be used to communicate with DevStack from ONAP. DEVSTACK_ADMIN_USER= User name that will be created on DevStack VM. Use this to ssh to DevStack VM if needed. DEVSTACK_VNET_NAME= Name of VNET that DevStack VM will attach to. DEVSTACK_SUBNET_NAME= Name of SUBNET attached to DEVSTACK_VNET_NAME. DEVSTACK_DISK_SIZE= Size of OS Disk for DevStack VM. DEVSTACK_BRANCH= Branch to use when installing devstack. OPENSTACK_USER= User name that will be added to OpenStack after devstack has finished installing. This is also the username that will be used to create a cloud site in ONAP SO. OPENSTACK_PASS= Password to use for OPENSTACK_USER. OPENSTACK_TENANT= Tenant name that will be added to OpenStack after devstack has finished installing. This is also the username that will be used to create a cloud site in ONAP SO. OPENSTACK_REGION= Only allows RegionOne for now, future enhancements will be added to allow multi-region. IMAGE_LIST= Space delimited list of image urls to add to DevStack. Not required. # ONAP PARAMS CLLI= Name of CLLI to be created in AAI. CLOUD_OWNER= Name of Cloud Owner to be created in AAI. CLOUD_REGION= Name of Cloud Region to be created in AAI. CUSTOMER= Name of Customer to be created in AAI. SUBSCRIBER= Name of Subscriber to be created in AAI. SERVICE_TYPE= Name of Service Type to be created in AAI. AZ= Name of Availability Zone to be created in AAI. OE= Name of Owning Entity to be created in VID. LOB= Name of Line of Business to be created in VID. PROJECT= Name of Project to be created in VID. PLATFORM= Name of Platform to be created in VID. OS_ID= Primary key to be used when adding cloud site to mariadb pod. OS_TENANT_ROLE= Only supports admin for now. OS_KEYSTONE= Use KEYSTONE_V3 for now. OOM_BRANCH= Branch of OOM to clone and use to install ONAP. CHART_VERSION= Version of charts to use for ONAP install. This is needed in case multiple versions of the onap helm charts are present on the machine being used for the install. OOM_OVERRIDES= Command line overrides to use when running helm deploy. --set <override value>, etc... DOCKER_REPOSITORY= Image repository url to pull ONAP images to use for installation.
When you execute cloud.sh, you have the option to create an integration-override.yaml file that will be used during helm deploy ... to install ONAP. This is done by passing the --override flag to cloud.sh.
The template used to create the override file is ./util/integration-override.template, and is invoked by ./util/create_robot_config.sh. It's very possible this isn't complete or sufficient for how you'd like to customize your deployment. You can update the template file and/or the script to provide additional customization for your ONAP install.
In cloud.conf, there's a parameter OOM_OVERRIDES available that's used to provide command line overrides to helm deploy. This uses the standard helm syntax, so if you're using it the value should look like OOM_OVERRIDES="--set vid.enabled=false,so.image=abc". If you don't want to override anything, just set this value to an empty string.
When you run cloud.sh it will execute pre_install.sh first, which checks a few things:
cloud.conf is available in Azure.pre_install.sh). The Azure cli is introduced changes in minor versions that aren't backwards compatible.cloud.conf.If you would like to skip pre_install.sh and run the deployment anyways, pass the flag --no-validate to cloud.sh, like this:
$ ./cloud.sh --no-validate
After ONAP is deployed, you have the option of executing an arbitrary set of post-install scripts. This is enabled by passing the --post-install flag to cloud.sh, like this:
$ ./cloud.sh --post-install
These post-install scripts need to be executable from the command line, and will be provided two parameters that they can use to perform their function:
cloud.conf that's used during the original deployment.Your post-install scripts can disregard these parameters, or source them and use the parameters as-needed.
Included with this repo is one post-install script (000_bootstrap_onap.sh)that bootstraps AAI, VID, and SO with cloud and customer details so that ONAP is ready to model and instantiate a VNF.
In order to include other custom post-install scripts, simply put them in the post-install directory, and make sure to set its mode to executable. They are executed in alphabetical order.
After ONAP and DevStack are deployed, there will be a deployment.notes file with instructions on how to access the various components. The BUILD_DIR specified in cloud.conf will contain a new ssh key, kubeconfig, and other deployment artifacts as well.
All of the access information below will be in deployment.notes.
To access the Kubernetes dashboard:
az aks browse --resource-group $AKS_RESOURCE_GROUP_NAME --name $AKS_NAME
To use kubectl:
export KUBECONFIG=$BUILD_DIR/kubeconfig kubectl ...
To access Horizon:
Find the public IP address via the Azure portal, and go to http://$DEVSTACK_PUBLIC_IP
SSH access to DevStack node:
ssh -i $BUILD_DIR/id_rsa ${DEVSTACK_ADMIN_USER}@${DEVSTACK_PUBLIC_IP}
OpenStack cli access:
There's an openstack cli pod that's created in the default kubernetes default namespace. To use it, run:
kubectl exec $OPENSTACK_CLI_POD -- sh -lc "<openstack command>"
ssh -i $BUILD_DIR/id_rsa ${NFS_ADMIN_USER}@${NFS_PUBLIC_IP}
After deployment, there will be a script named $BUILD_DIR/clean.sh that can be used to delete the resource groups that were created during deployment. This script is not required; you can always just navigate to the Azure portal to delete the resource groups manually.
Below are instructions for how to create DevStack, NFS, or AKS cluster separately if you don't want to create everything all at once.
NOTE: The configuration to link components together (network peering, route table modification, NFS setup, etc...) and the onap-bootstrap will not occur if you run the scripts separately
$ ./create_devstack.sh --help ./create_devstack.sh [options] required: --public-key public key to add for admin user [required] --user-public-ip public ip that will be granted access to VM [required] -l, --location location to deploy VM [required] -u, --admin-user admin user to create on VM [required] additional options: -f, --no-prompt executes with no prompt for confirmation -h, --help provide brief overview of script -n, --name VM name [optional] -g, --resource-group provide brief overview of script [optional] -s, --size Azure flavor size for VM [optional] -c, --cidr cidr for VNET to create for VM [optional]. If provided, must also provide --devstack-private-ip from same range. -d, --directory directory to store cloud config data [optional] --vnet-name name of Vnet to create for VM [optional] --image-list space delimited list of image urls that will be added to devstack [optional] --devstack-private-ip private ip assigned to VM [optional]. If provided, this value must come from the CIDR range of VNET. --devstack-subnet-name subnet name created on VNET [optional] --devstack-disk-size size of OS disk to be allocated [optional] --openstack-username default user name for openstack [optional] --openstack-password default password for openstack [optional] --openstack-tenant default tenant name for openstack [optional]
$ ./create_nfs.sh --help ./create_nfs.sh [options] required: --public-key public key to add for admin user [required] --user-public-ip public ip that will be granted access to VM [required] -l, --location location to deploy VM [required] -u, --admin-user admin user to create on VM [required] --aks-node-cidr CIDR for Kubernetes nodes [required]. This is used during the NFS deploy to grant access to the NFS server from Kubernetes. additional options: -f, --no-prompt executes with no prompt for confirmation -h, --help provide brief overview of script -n, --name VM name [optional] -g, --resource-group resource group that will be created [optional] -s, --size Azure flavor size for VM [optional] -c, --cidr cidr for VNET to create for VM [optional]. -d, --directory directory to store cloud config data [optional] --vnet-name name of Vnet to create for VM [optional] --nfs-subnet-name subnet name created on VNET [optional] --nfs-disk-size size of external disk to be mounted on NFS VM [optional]
$ ./create_aks.sh --help ./create_aks.sh [options] required: --user-public-ip public ip that will be granted access to AKS [required] --admin-user admin user created on AKS nodes [required] --public-key public key added for admin user [required] -l, --location location to deploy AKS [required] additional options: -f, --no-prompt executes with no prompt for confirmation -h, --help provide brief overview of script -n, --name AKS name [optional] -g, --resource-group name of resource group that will be created [optional] -s, --size azure flavor size for Kube nodes [optional] -v, --kube-version version of Kubernetes for cluster [optional] -c, --node-count number of nodes for cluster [optional] --service-cidr cidr for Kuberenetes services [optional]. --dns-ip IP for Kuberenetes dns service [optional]. This should be from --service-cidr. --pod-cidr cidr for Kuberenetes pods [optional]. --node-cidr cidr for Kuberenetes nodes [optional]. --vnet-name name of Vnet to create for Kubernetes Cluster [optional]