This page is currently a WIP for the next 2 weeks starting 20180214. Deliverables will deployment scripts, arm/cli templates for various deployment scenarios (single, multiple, federated servers)

under OOM-524 - Getting issue details... STATUS , OOM-635 - Getting issue details... STATUS , OOM-607 - Getting issue details... STATUS , OOM-710 - Getting issue details... STATUS , OOM-711 - Getting issue details... STATUS

Azure CLI Installation

Requirements

Azure subscription

OSX

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest

Install homebrew first (reinstall if you are on the latest OSX 10.13.2 https://github.com/Homebrew/install because of 3718)

Will install Python 3.6

$brew update
$brew install azure-cli

https://docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli?view=azure-cli-latest

$ az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E..D to authenticate.
[ {
    "cloudName": "AzureCloud",
    "id": "f4...b",
    "isDefault": true,
    "name": "Pay-As-You-Go",
    "state": "Enabled",
    "tenantId": "bcb.....f",
    "user": {
      "name": "michael@....org",
      "type": "user"
    }}]

Windows Powershell

https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest

ONAP on Azure VMs

Pre-requisite

The supported versions are as follows:

ONAP Release	Rancher	Kubernetes	Helm	Kubectl	Docker
Amsterdam	1.6.10	1.7.7	2.3.0	1.7.7	1.12.x
Beijing	1.6.14	1.8.6	2.6.1+	1.8.6	17.03-ce

Reference Server

http://azure.onap.info:8880/r/projects/1a7/kubernetes-dashboard:9090/#!/pod?namespace=_all

Cluster Configuration

Single Server

Azure Artifacts

Static IP

Secondary Drive

VM

ARM Template

Follow https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-create-first-template

Create a Storage Account

$ az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...Z to authenticate.
$ az group create --name examplegroup --location "South Central US"
{
  "id": "/subscriptions/f4b...e8b/resourceGroups/examplegroup",
  "location": "southcentralus",
  "managedBy": null,
  "name": "examplegroup",
  "properties": {
    "provisioningState": "Succeeded"
  },
  "tags": null
}
obrien:obrienlabs amdocs$ vi azuredeploy_storageaccount.json
obrien:obrienlabs amdocs$ az group deployment create --resource-group examplegroup --template-file azuredeploy_storageaccount.json
{
  "id": "/subscriptions/f4...e8b/resourceGroups/examplegroup/providers/Microsoft.Resources/deployments/azuredeploy_storageaccount",
  "name": "azuredeploy_storageaccount",
  "properties": {
    "additionalProperties": {
      "duration": "PT32.9822642S",
      "outputResources": [
        {
          "id": "/subscriptions/f4..e8b/resourceGroups/examplegroup/providers/Microsoft.Storage/storageAccounts/storagekj6....kk2w",
          "resourceGroup": "examplegroup"
        }],
      "templateHash": "11440483235727994285"},
    "correlationId": "41a0f79..90c291",
    "debugSetting": null,
    "dependencies": [],
    "mode": "Incremental",
    "outputs": {},
    "parameters": {},
    "parametersLink": null,
    "providers": [
      {
        "id": null,
        "namespace": "Microsoft.Storage",
        "registrationState": null,
        "resourceTypes": [
          {
            "aliases": null,
            "apiVersions": null,
            "locations": [
              "southcentralus"
            ],
            "properties": null,
            "resourceType": "storageAccounts"
          }]}],
    "provisioningState": "Succeeded",
    "template": null,
    "templateLink": null,
    "timestamp": "2018-02-17T16:15:11.562170+00:00"
  },
  "resourceGroup": "examplegroup"}

Create a VM

We need a 128G VM with at least 8vCores (peak is 60) and a 100+GB drive. The sizes are detailed on https://docs.microsoft.com/en-ca/azure/virtual-machines/windows/sizes-memory - we will use the Standard_D32s_v3 type

We need an "all open 0.0.0.0/0" security group and

a reassociated data drive as boot drive

Get the ARM template

see open review in OOM-711 - Getting issue details... STATUS

"ubuntuOSVersion": "16.04.0-LTS"
"imagePublisher": "Canonical",
"imageOffer": "UbuntuServer",
"vmSize": "Standard_E8s_v3"
"osDisk": {"createOption": "FromImage"},"dataDisks": [{"diskSizeGB": 511,"lun": 0, "createOption": "Empty" }]

Follow

https://github.com/Azure/azure-quickstart-templates/tree/master/101-acs-kubernetes

https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy

https://docs.microsoft.com/en-us/cli/azure/group/deployment?view=azure-cli-latest#az_group_deployment_create

https://github.com/Azure/azure-quickstart-templates/tree/master/101-vm-simple-linux

It needs a security group https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-create-nsg-arm-template

               "networkSecurityGroup": {
                  "id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('frontEndNSGName'))]"
                }

# create a resource group if not already there
az group create --name ONAPAMDOCS --location eastus
 "id": "/subscriptions/f.8b/resourceGroups/ONAPAMDOCS"

# validate first
az group deployment validate --resource-group ONAPAMDOCS --template-file oom_azure_arm_deploy.json  --parameters @oom_azure_arm_deploy_parameters.json 
# run deployment
az group deployment create --resource-group ONAPAMDOCS --template-file oom_azure_arm_deploy.json  --parameters @oom_azure_arm_deploy_parameters.json

SSH into your VM and run the Kubernetes and OOM installation scripts

# clone the oom repo to get the install directory
git clone https://gerrit.onap.org/r/oom
# run the Rancher RI installation (to install kubernetes)
oom/install/rancher/oom_rancher_install.sh -b master -s 192.168.240.32
# run the oom deployment script
# get a copy of onap-parametes.yaml and place in this folder
oom/install/deployment/cd.sh -b master -s 192.168.240.32 -e onap

oom_rancher_install.sh is in OOM-715 - Getting issue details... STATUS under https://gerrit.onap.org/r/#/c/32019/

cd.sh in OOM-716 - Getting issue details... STATUS under https://gerrit.onap.org/r/#/c/32653/

Delete the VM and resource group

# delete the vm and resources
az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy
# the above deletion will not delete the actual resources - only a delete of the group or each individual resource works
# optionally delete the resource group
az group delete --name ONAPAMDOCS -y

ONAP on Azure Container Service

AKS Installation

Follow https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster

Register for AKS preview via az cli

obrienbiometrics:obrienlabs michaelobrien$ az provider register -n Microsoft.ContainerService
Registering is still on-going. You can monitor using 'az provider show -n Microsoft.ContainerService'

Create an AKS resource group

Raise your AKS vCPU quota - optional

https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits#container-service-aks-limits

http://aka.ms/corequotaincrease

https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest

Deployment failed. Correlation ID: 4b4707a7-2244-4557-855e-11bcced556de. Provisioning of resource(s) for container service onapAKSCluster in resource group onapAKS failed. Message: Operation results in exceeding quota limits of Core. Maximum allowed: 10, Current in use: 10, Additional requested: 1. Please read more about quota increase at http://aka.ms/corequotaincrease.. Details:

Create AKS cluster

obrienbiometrics:obrienlabs michaelobrien$ az aks create --resource-group onapAKS --name onapAKSCluster --node-count 1 --generate-ssh-keys
 - Running ..
 "fqdn": "onapaksclu-onapaks-f4....3.hcp.eastus.azmk8s.io",

AKS cluster VM granularity

The cluster will start with a 3.5G VM before scaling

Resources for your AKS cluster

Bring up AAI only for now

Design Issues

Troubleshooting

DNS propagation and caching

It takes about 2 min for DNS entries to propagate out from A record DNS changes. For example the following IP/DNS association took 2 min to appear in dig.

obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info
; <<>> DiG 9.9.7-P3 <<>> azure.onap.info
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10599
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;azure.onap.info.		IN	A
;; ANSWER SECTION:
azure.onap.info.	251	IN	A	52.224.233.230
;; Query time: 68 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Feb 20 10:26:59 EST 2018
;; MSG SIZE  rcvd: 60

obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info
; <<>> DiG 9.9.7-P3 <<>> azure.onap.info
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30447
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;azure.onap.info.		IN	A
;; ANSWER SECTION:
azure.onap.info.	299	IN	A	13.92.225.167
;; Query time: 84 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Feb 20 10:27:04 EST 2018

Corporate Firewall Access

Inside the corporate firewall - avoid it

PS C:\> az login
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443)
: Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by NewConnectionError('<urllib3.conne
ction.VerifiedHTTPSConnection object at 0x04D18730>: Failed to establish a new connection: [Errno 11001] getaddrinfo fai
led',))

at home or cell hotspot

PS C:\> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...2W to authenticate.
[  {    "cloudName": "AzureCloud",    "id": "4...da1",    "isDefault": true,    "name": "Microsoft Azure Internal Consumption",    "state": "Enabled",    "tenantId": "72f98....47",    "user": {      "name": "fran...ocs.com",      "type": "user"    }]

On corporate account (need permissions bump to be able to create a resource group prior to running an arm template
https://wiki.onap.org/display/DW/ONAP+on+Kubernetes+on+Microsoft+Azure#ONAPonKubernetesonMicrosoftAzure-ARMTemplate
PS C:\> az group create --name onapKubernetes --location eastus
The client 'fra...s.com' with object id '08f98c7e-...ed' does not have authorization to per
form action 'Microsoft.Resources/subscriptions/resourcegroups/write' over scope '/subscriptions/42e...8
7da1/resourcegroups/onapKubernetes'.

try my personal = OK
PS C:\> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code EE...ULR to authenticate.
Terminate batch job (Y/N)? y
# hangs when first time login in a new pc
PS C:\> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E.PBKS to authenticate.
[  {    "cloudName": "AzureCloud",    "id": "f4b...b",    "isDefault": true    "name": "Pay-As-You-Go",    "state": "Enabled",   "tenantId": "bcb...f4f",   "user":       "name": "michael@obrien...org",    "type": "user"    }  }]
PS C:\> az group create --name onapKubernetes2 --location eastus
{  "id": "/subscriptions/f4b....b/resourceGroups/onapKubernetes2",  "location": "eastus",  "managedBy": null,  "name": "onapKubernetes2",  "properties": {    "provisioningState": "Succeeded"  },  "tags": null}

20180228: Deployment delete does not delete resources without a resourceGroup delete

I find that a delete deployment deletes the deployment but not the actual resources. The workaround is to delete the resource group - but in some constrained subscriptions the cli user may not have the ability to create a resource group - and hence delete it.

see

https://github.com/Azure/azure-sdk-for-java/issues/1167

deleting the resources manually for now - is a workaround if you cannot create/delete resource groups

# delete the vm and resources
az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy

# the above deletion will not delete the actual resources - only a delete of the group or each individual resource works

# optionally delete the resource group
az group delete --name ONAPAMDOCS -y

However modifying the template to add resources works well. For example adding a reference to a network security group

20180228: Resize the OS disk

ONAP requires at least 75g - the issue is than in most VM templates on Azure - the OS disk is 30g - we need to either switch to the data disk or resize the os disk.

# add diskSizeGB to the template
          "osDisk": {
                "diskSizeGB": 255,
                "createOption": "FromImage"
            },
ubuntu@oom-auto-deploy:~$ df
Filesystem     1K-blocks    Used Available Use% Mounted on
udev            65989400       0  65989400   0% /dev
tmpfs           13201856    8848  13193008   1% /run
/dev/sda1      259142960 1339056 257787520   1% /
tmpfs           66009280       0  66009280   0% /dev/shm
tmpfs               5120       0      5120   0% /run/lock
tmpfs           66009280       0  66009280   0% /sys/fs/cgroup
none                  64       0        64   0% /etc/network/interfaces.dynamic.d
/dev/sdb1      264091588   60508 250592980   1% /mnt
tmpfs           13201856       0  13201856   0% /run/user/1000
ubuntu@oom-auto-deploy:~$ free
              total        used        free      shared  buff/cache   available
Mem:      132018560      392336   131242164        8876      384060   131012328