| Table of Contents |
|---|
This page is currently a WIP for the next 2 weeks starting 20180214. Deliverables will deployment scripts, arm/cli templates for various deployment scenarios (single, multiple, federated servers)
under
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Azure CLI Installation
Requirements
Azure subscription
OSX
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest
Install homebrew first (reinstall if you are on the latest OSX 10.13.2 https://github.com/Homebrew/install because of 3718)
Will install Python 3.6
| Code Block |
|---|
$brew update
$brew install azure-cli |
https://docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli?view=azure-cli-latest
| Code Block |
|---|
$ az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E..D to authenticate.
[ {
"cloudName": "AzureCloud",
"id": "f4...b",
"isDefault": true,
"name": "Pay-As-You-Go",
"state": "Enabled",
"tenantId": "bcb.....f",
"user": {
"name": "michael@....org",
"type": "user"
}}] |
Windows Powershell
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest
ONAP on Azure VMs
Pre-requisite
The supported versions are as follows:
...
being migrated (version 93 - ONAP on Kubernetes on Microsoft Azure) to read the docs along with all the other pages in this section.
http://onap.readthedocs.io/en/latest/submodules/oom.git/docs/oom_cloud_setup_guide.html
Kubernetes based onap continuous deployment WIP content continues - here - Cloud Native Deployment#MicrosoftAzure
Reference Server
http://azure.onap.info:8880/r/projects/1a7/kubernetes-dashboard:9090/#!/pod?namespace=_all
Cluster Configuration
Single Server
Azure Artifacts
Static IP
Secondary Drive
VM
ARM Template
Follow https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-create-first-template
Create a Storage Account
| Code Block |
|---|
$ az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...Z to authenticate.
$ az group create --name examplegroup --location "South Central US"
{
"id": "/subscriptions/f4b...e8b/resourceGroups/examplegroup",
"location": "southcentralus",
"managedBy": null,
"name": "examplegroup",
"properties": {
"provisioningState": "Succeeded"
},
"tags": null
}
obrien:obrienlabs amdocs$ vi azuredeploy_storageaccount.json
obrien:obrienlabs amdocs$ az group deployment create --resource-group examplegroup --template-file azuredeploy_storageaccount.json
{
"id": "/subscriptions/f4...e8b/resourceGroups/examplegroup/providers/Microsoft.Resources/deployments/azuredeploy_storageaccount",
"name": "azuredeploy_storageaccount",
"properties": {
"additionalProperties": {
"duration": "PT32.9822642S",
"outputResources": [
{
"id": "/subscriptions/f4..e8b/resourceGroups/examplegroup/providers/Microsoft.Storage/storageAccounts/storagekj6....kk2w",
"resourceGroup": "examplegroup"
}],
"templateHash": "11440483235727994285"},
"correlationId": "41a0f79..90c291",
"debugSetting": null,
"dependencies": [],
"mode": "Incremental",
"outputs": {},
"parameters": {},
"parametersLink": null,
"providers": [
{
"id": null,
"namespace": "Microsoft.Storage",
"registrationState": null,
"resourceTypes": [
{
"aliases": null,
"apiVersions": null,
"locations": [
"southcentralus"
],
"properties": null,
"resourceType": "storageAccounts"
}]}],
"provisioningState": "Succeeded",
"template": null,
"templateLink": null,
"timestamp": "2018-02-17T16:15:11.562170+00:00"
},
"resourceGroup": "examplegroup"} |
Create a VM
We need a 64G VM with at least 8vCores and a 100+GB drive. The sizes are detailed on https://docs.microsoft.com/en-ca/azure/virtual-machines/windows/sizes-memory - we will use the Standard_E8s_v3 type
We need an "all open 0.0.0.0/0" security group and
a reassociated data drive as boot drive
Get the ARM template
see open review in
| Jira | ||||||
|---|---|---|---|---|---|---|
|
| Code Block |
|---|
"ubuntuOSVersion": "16.04.0-LTS"
"imagePublisher": "Canonical",
"imageOffer": "UbuntuServer",
"vmSize": "Standard_E8s_v3"
"osDisk": {"createOption": "FromImage"},"dataDisks": [{"diskSizeGB": 511,"lun": 0, "createOption": "Empty" }] |
Follow
https://github.com/Azure/azure-quickstart-templates/tree/master/101-acs-kubernetes
https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy
https://github.com/Azure/azure-quickstart-templates/tree/master/101-vm-simple-linuxIt needs a security group https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-create-nsg-arm-template
| Code Block |
|---|
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('frontEndNSGName'))]"
} |
| Code Block |
|---|
# create a resource group if not already there
az group create --name ONAPAMDOCS --location eastus
"id": "/subscriptions/f.8b/resourceGroups/ONAPAMDOCS"
# validate first
az group deployment validate --resource-group ONAPAMDOCS --template-file oom_azure_arm_deploy.json --parameters @oom_azure_arm_deploy_parameters.json
# run deployment
az group deployment create --resource-group ONAPAMDOCS --template-file oom_azure_arm_deploy.json --parameters @oom_azure_arm_deploy_parameters.json |
SSH into your VM and run the Kubernetes and OOM installation scripts
| Code Block |
|---|
# clone the oom repo to get the install directory
git clone https://gerrit.onap.org/r/oom
# run the Rancher RI installation (to install kubernetes)
oom/install/rancher/oom_rancher_install.sh -b master -s 192.168.240.32
# run the oom deployment script
# get a copy of onap-parametes.yaml and place in this folder
oom/install/deployment/cd.sh -b master -s 192.168.240.32 -e onap |
oom_rancher_install.sh is in
| Jira | ||||||
|---|---|---|---|---|---|---|
|
cd.sh in
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Delete the VM and resource group
| Code Block |
|---|
# delete the vm and resources
az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy
# the above deletion will not delete the actual resources - only a delete of the group or each individual resource works
# optionally delete the resource group
az group delete --name ONAPAMDOCS -y |
ONAP on Azure Container Service
AKS Installation
Follow https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster
Register for AKS preview via az cli
| Code Block |
|---|
obrienbiometrics:obrienlabs michaelobrien$ az provider register -n Microsoft.ContainerService
Registering is still on-going. You can monitor using 'az provider show -n Microsoft.ContainerService' |
Create an AKS resource group
Raise your AKS vCPU quota - optional
http://aka.ms/corequotaincreasehttps://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest
| Code Block |
|---|
Deployment failed. Correlation ID: 4b4707a7-2244-4557-855e-11bcced556de. Provisioning of resource(s) for container service onapAKSCluster in resource group onapAKS failed. Message: Operation results in exceeding quota limits of Core. Maximum allowed: 10, Current in use: 10, Additional requested: 1. Please read more about quota increase at http://aka.ms/corequotaincrease.. Details: |
Create AKS cluster
| Code Block |
|---|
obrienbiometrics:obrienlabs michaelobrien$ az aks create --resource-group onapAKS --name onapAKSCluster --node-count 1 --generate-ssh-keys
- Running ..
"fqdn": "onapaksclu-onapaks-f4....3.hcp.eastus.azmk8s.io", |
AKS cluster VM granularity
The cluster will start with a 3.5G VM before scaling
Resources for your AKS cluster
Bring up AAI only for now
Design Issues
Troubleshooting
DNS propagation and caching
It takes about 2 min for DNS entries to propagate out from A record DNS changes. For example the following IP/DNS association took 2 min to appear in dig.
| Code Block |
|---|
obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info
; <<>> DiG 9.9.7-P3 <<>> azure.onap.info
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10599
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;azure.onap.info. IN A
;; ANSWER SECTION:
azure.onap.info. 251 IN A 52.224.233.230
;; Query time: 68 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Feb 20 10:26:59 EST 2018
;; MSG SIZE rcvd: 60
obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info
; <<>> DiG 9.9.7-P3 <<>> azure.onap.info
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30447
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;azure.onap.info. IN A
;; ANSWER SECTION:
azure.onap.info. 299 IN A 13.92.225.167
;; Query time: 84 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Feb 20 10:27:04 EST 2018 |
Corporate Firewall Access
| Code Block |
|---|
Inside the corporate firewall - avoid it
PS C:\> az login
Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443)
: Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by NewConnectionError('<urllib3.conne
ction.VerifiedHTTPSConnection object at 0x04D18730>: Failed to establish a new connection: [Errno 11001] getaddrinfo fai
led',))
at home or cell hotspot
PS C:\> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...2W to authenticate.
[ { "cloudName": "AzureCloud", "id": "4...da1", "isDefault": true, "name": "Microsoft Azure Internal Consumption", "state": "Enabled", "tenantId": "72f98....47", "user": { "name": "fran...ocs.com", "type": "user" }]
On corporate account (need permissions bump to be able to create a resource group prior to running an arm template
https://wiki.onap.org/display/DW/ONAP+on+Kubernetes+on+Microsoft+Azure#ONAPonKubernetesonMicrosoftAzure-ARMTemplate
PS C:\> az group create --name onapKubernetes --location eastus
The client 'fra...s.com' with object id '08f98c7e-...ed' does not have authorization to per
form action 'Microsoft.Resources/subscriptions/resourcegroups/write' over scope '/subscriptions/42e...8
7da1/resourcegroups/onapKubernetes'.
try my personal = OK
PS C:\> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code EE...ULR to authenticate.
Terminate batch job (Y/N)? y
# hangs when first time login in a new pc
PS C:\> az login
To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E.PBKS to authenticate.
[ { "cloudName": "AzureCloud", "id": "f4b...b", "isDefault": true "name": "Pay-As-You-Go", "state": "Enabled", "tenantId": "bcb...f4f", "user": "name": "michael@obrien...org", "type": "user" } }]
PS C:\> az group create --name onapKubernetes2 --location eastus
{ "id": "/subscriptions/f4b....b/resourceGroups/onapKubernetes2", "location": "eastus", "managedBy": null, "name": "onapKubernetes2", "properties": { "provisioningState": "Succeeded" }, "tags": null} |
20180228: Deployment delete does not delete resources without a resourceGroup delete
I find that a delete deployment deletes the deployment but not the actual resources. The workaround is to delete the resource group - but in some constrained subscriptions the cli user may not have the ability to create a resource group - and hence delete it.
see
https://github.com/Azure/azure-sdk-for-java/issues/1167
deleting the resources manually for now - is a workaround if you cannot create/delete resource groups
# delete the vm and resources
az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy# the above deletion will not delete the actual resources - only a delete of the group or each individual resource works# optionally delete the resource group
az group delete --name ONAPAMDOCS -yHowever modifying the template to add resources works well. For example adding a reference to a network security group
20180228: Resize the OS disk
ONAP requires at least 75g - the issue is than in most VM templates on Azure - the OS disk is 30g - we need to either switch to the data disk or resize the os disk.
| Code Block |
|---|
# add diskSizeGB to the template
"osDisk": {
"diskSizeGB": 255,
"createOption": "FromImage"
}, |
Links
https://azure.microsoft.com/en-us/services/container-service/
https://kubernetes.io/docs/concepts/containers/images/#using-azure-container-registry-acr