This page is currently a WIP for the next 2 weeks starting 20180214. Deliverables will deployment scripts, arm/cli templates for various deployment scenarios (single, multiple, federated servers)
under - OOM-524Getting issue details... STATUS , - OOM-635Getting issue details... STATUS , - OOM-607Getting issue details... STATUS , - OOM-710Getting issue details... STATUS , - OOM-711Getting issue details... STATUS
Azure CLI Installation
Requirements
Azure subscription
OSX
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest
Install homebrew first (reinstall if you are on the latest OSX 10.13.2 https://github.com/Homebrew/install because of 3718)
Will install Python 3.6
$brew update $brew install azure-cli
https://docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli?view=azure-cli-latest
$ az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E..D to authenticate. [ { "cloudName": "AzureCloud", "id": "f4...b", "isDefault": true, "name": "Pay-As-You-Go", "state": "Enabled", "tenantId": "bcb.....f", "user": { "name": "michael@....org", "type": "user" }}]
Windows Powershell
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest
ONAP on Azure VMs
Pre-requisite
The supported versions are as follows:
ONAP Release | Rancher | Kubernetes | Helm | Kubectl | Docker |
---|---|---|---|---|---|
Amsterdam | 1.6.10 | 1.7.7 | 2.3.0 | 1.7.7 | 1.12.x |
Beijing | 1.6.14 | 1.8.6 | 2.6.1+ | 1.8.6 | 17.03-ce |
Reference Server
http://azure.onap.info:8880/r/projects/1a7/kubernetes-dashboard:9090/#!/pod?namespace=_all
Cluster Configuration
Single Server
Azure Artifacts
Static IP
Secondary Drive
VM
ARM Template
Follow https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-create-first-template
Create a Storage Account
$ az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...Z to authenticate. $ az group create --name examplegroup --location "South Central US" { "id": "/subscriptions/f4b...e8b/resourceGroups/examplegroup", "location": "southcentralus", "managedBy": null, "name": "examplegroup", "properties": { "provisioningState": "Succeeded" }, "tags": null } obrien:obrienlabs amdocs$ vi azuredeploy_storageaccount.json obrien:obrienlabs amdocs$ az group deployment create --resource-group examplegroup --template-file azuredeploy_storageaccount.json { "id": "/subscriptions/f4...e8b/resourceGroups/examplegroup/providers/Microsoft.Resources/deployments/azuredeploy_storageaccount", "name": "azuredeploy_storageaccount", "properties": { "additionalProperties": { "duration": "PT32.9822642S", "outputResources": [ { "id": "/subscriptions/f4..e8b/resourceGroups/examplegroup/providers/Microsoft.Storage/storageAccounts/storagekj6....kk2w", "resourceGroup": "examplegroup" }], "templateHash": "11440483235727994285"}, "correlationId": "41a0f79..90c291", "debugSetting": null, "dependencies": [], "mode": "Incremental", "outputs": {}, "parameters": {}, "parametersLink": null, "providers": [ { "id": null, "namespace": "Microsoft.Storage", "registrationState": null, "resourceTypes": [ { "aliases": null, "apiVersions": null, "locations": [ "southcentralus" ], "properties": null, "resourceType": "storageAccounts" }]}], "provisioningState": "Succeeded", "template": null, "templateLink": null, "timestamp": "2018-02-17T16:15:11.562170+00:00" }, "resourceGroup": "examplegroup"}
Create a VM
We need a 64G VM with at least 8vCores and a 100+GB drive. The sizes are detailed on https://docs.microsoft.com/en-ca/azure/virtual-machines/windows/sizes-memory - we will use the Standard_E8s_v3 type
We need an "all open 0.0.0.0/0" security group and
a reassociated data drive as boot drive
Get the ARM template
see open review in - OOM-711Getting issue details... STATUS
"ubuntuOSVersion": "16.04.0-LTS" "imagePublisher": "Canonical", "imageOffer": "UbuntuServer", "vmSize": "Standard_E8s_v3" "osDisk": {"createOption": "FromImage"},"dataDisks": [{"diskSizeGB": 511,"lun": 0, "createOption": "Empty" }]
Follow
https://github.com/Azure/azure-quickstart-templates/tree/master/101-acs-kubernetes
https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy
https://github.com/Azure/azure-quickstart-templates/tree/master/101-vm-simple-linux
It needs a security group https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-create-nsg-arm-template
"networkSecurityGroup": { "id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('frontEndNSGName'))]" }
# create a resource group if not already there az group create --name ONAPAMDOCS --location eastus "id": "/subscriptions/f.8b/resourceGroups/ONAPAMDOCS" # validate first az group deployment validate --resource-group ONAPAMDOCS --template-file oom_azure_arm_deploy.json --parameters @oom_azure_arm_deploy_parameters.json # run deployment az group deployment create --resource-group ONAPAMDOCS --template-file oom_azure_arm_deploy.json --parameters @oom_azure_arm_deploy_parameters.json
SSH into your VM and run the Kubernetes and OOM installation scripts
# clone the oom repo to get the install directory git clone https://gerrit.onap.org/r/oom # run the Rancher RI installation (to install kubernetes) oom/install/rancher/oom_rancher_install.sh -b master -s 192.168.240.32 # run the oom deployment script # get a copy of onap-parametes.yaml and place in this folder oom/install/deployment/cd.sh -b master -s 192.168.240.32 -e onap
oom_rancher_install.sh is in - OOM-715Getting issue details... STATUS under https://gerrit.onap.org/r/#/c/32019/
cd.sh in - OOM-716Getting issue details... STATUS under https://gerrit.onap.org/r/#/c/32653/
Delete the VM and resource group
# delete the vm and resources az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy # the above deletion will not delete the actual resources - only a delete of the group or each individual resource works # optionally delete the resource group az group delete --name ONAPAMDOCS -y
ONAP on Azure Container Service
AKS Installation
Follow https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster
Register for AKS preview via az cli
obrienbiometrics:obrienlabs michaelobrien$ az provider register -n Microsoft.ContainerService Registering is still on-going. You can monitor using 'az provider show -n Microsoft.ContainerService'
Create an AKS resource group
Raise your AKS vCPU quota - optional
http://aka.ms/corequotaincrease
https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest
Deployment failed. Correlation ID: 4b4707a7-2244-4557-855e-11bcced556de. Provisioning of resource(s) for container service onapAKSCluster in resource group onapAKS failed. Message: Operation results in exceeding quota limits of Core. Maximum allowed: 10, Current in use: 10, Additional requested: 1. Please read more about quota increase at http://aka.ms/corequotaincrease.. Details:
Create AKS cluster
obrienbiometrics:obrienlabs michaelobrien$ az aks create --resource-group onapAKS --name onapAKSCluster --node-count 1 --generate-ssh-keys - Running .. "fqdn": "onapaksclu-onapaks-f4....3.hcp.eastus.azmk8s.io",
AKS cluster VM granularity
The cluster will start with a 3.5G VM before scaling
Resources for your AKS cluster
Bring up AAI only for now
Design Issues
Troubleshooting
DNS propagation and caching
It takes about 2 min for DNS entries to propagate out from A record DNS changes. For example the following IP/DNS association took 2 min to appear in dig.
obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info ; <<>> DiG 9.9.7-P3 <<>> azure.onap.info ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10599 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;azure.onap.info. IN A ;; ANSWER SECTION: azure.onap.info. 251 IN A 52.224.233.230 ;; Query time: 68 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Tue Feb 20 10:26:59 EST 2018 ;; MSG SIZE rcvd: 60 obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info ; <<>> DiG 9.9.7-P3 <<>> azure.onap.info ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30447 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;azure.onap.info. IN A ;; ANSWER SECTION: azure.onap.info. 299 IN A 13.92.225.167 ;; Query time: 84 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Tue Feb 20 10:27:04 EST 2018
Corporate Firewall Access
Inside the corporate firewall - avoid it PS C:\> az login Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443) : Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by NewConnectionError('<urllib3.conne ction.VerifiedHTTPSConnection object at 0x04D18730>: Failed to establish a new connection: [Errno 11001] getaddrinfo fai led',)) at home or cell hotspot PS C:\> az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...2W to authenticate. [ { "cloudName": "AzureCloud", "id": "4...da1", "isDefault": true, "name": "Microsoft Azure Internal Consumption", "state": "Enabled", "tenantId": "72f98....47", "user": { "name": "fran...ocs.com", "type": "user" }] On corporate account (need permissions bump to be able to create a resource group prior to running an arm template https://wiki.onap.org/display/DW/ONAP+on+Kubernetes+on+Microsoft+Azure#ONAPonKubernetesonMicrosoftAzure-ARMTemplate PS C:\> az group create --name onapKubernetes --location eastus The client 'fra...s.com' with object id '08f98c7e-...ed' does not have authorization to per form action 'Microsoft.Resources/subscriptions/resourcegroups/write' over scope '/subscriptions/42e...8 7da1/resourcegroups/onapKubernetes'. try my personal = OK PS C:\> az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code EE...ULR to authenticate. Terminate batch job (Y/N)? y # hangs when first time login in a new pc PS C:\> az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E.PBKS to authenticate. [ { "cloudName": "AzureCloud", "id": "f4b...b", "isDefault": true "name": "Pay-As-You-Go", "state": "Enabled", "tenantId": "bcb...f4f", "user": "name": "michael@obrien...org", "type": "user" } }] PS C:\> az group create --name onapKubernetes2 --location eastus { "id": "/subscriptions/f4b....b/resourceGroups/onapKubernetes2", "location": "eastus", "managedBy": null, "name": "onapKubernetes2", "properties": { "provisioningState": "Succeeded" }, "tags": null}
20180228: Deployment delete does not delete resources without a resourceGroup delete
I find that a delete deployment deletes the deployment but not the actual resources. The workaround is to delete the resource group - but in some constrained subscriptions the cli user may not have the ability to create a resource group - and hence delete it.
see
https://github.com/Azure/azure-sdk-for-java/issues/1167
deleting the resources manually for now - is a workaround if you cannot create/delete resource groups
# delete the vm and resources az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy
# the above deletion will not delete the actual resources - only a delete of the group or each individual resource works
# optionally delete the resource group az group delete --name ONAPAMDOCS -y
However modifying the template to add resources works well. For example adding a reference to a network security group
20180228: Resize the OS disk
ONAP requires at least 75g - the issue is than in most VM templates on Azure - the OS disk is 30g - we need to either switch to the data disk or resize the os disk.
# add diskSizeGB to the template "osDisk": { "diskSizeGB": 255, "createOption": "FromImage" },
Links
https://azure.microsoft.com/en-us/services/container-service/
https://kubernetes.io/docs/concepts/containers/images/#using-azure-container-registry-acr