Docker images = 30G
Shared NFS = 5G
RAM = 70G+
vCores = 12+ (max 55)
Cluster = 3 + 1 (hosts + rancher master)
Network = 170 MB/sec (2700 packets/sec at 64k/packet max) = NFS + docker pulls
Account Provider: (2) Robin of Amazon and Michael O'Brien of Amdocs
You need an NFS share between the VM's in your Kubernetes cluster - an Elastic File System share will wrap NFS
"From the NFS wizard"
Setting up your EC2 instance
Mounting your file system
If you are unable to connect, see our troubleshooting documentation.
https://docs.aws.amazon.com/efs/latest/ug/mounting-fs.html
ubuntu@ip-10-0-0-66:~$ sudo apt-get install nfs-common ubuntu@ip-10-0-0-66:~$ cd / ubuntu@ip-10-0-0-66:~$ sudo mkdir /dockerdata-nfs root@ip-10-0-0-19:/# sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 fs-43b2763a.efs.us-east-2.amazonaws.com:/ /dockerdata-nfs # write something on one vm - and verify it shows on another ubuntu@ip-10-0-0-8:~$ ls /dockerdata-nfs/ test.sh |
Subscription Sponsor: (1) Microsoft
Deliverables are deployment scripts, arm/cli templates for various deployment scenarios (single, multiple, federated servers)
In review , , ,
Automation is currently only written for single VM that hosts both the rancher server and the deployed onap pods. Use the ARM template below to deploy your VM and provision it (adjust your config parameters)
Two choices, run the single oom_deployment.sh ARM wrapper - or use it to bring up an empty vm and run oom_entrypoint.sh manually. Once the VM comes up the oom_entrypoint.sh script will run - which will download the oom_rancher_setup.sh script to setup docker, rancher, kubernetes and helm - the entrypoint script will then run the cd.sh script to bring up onap based on your values.yaml config by running helm install on it.
# login to az cli, wget the deployment script, arm template and parameters file - edit the parameters file (dns, ssh key ...) and run the arm template wget https://jira.onap.org/secure/attachment/11459/oom_deployment.sh wget https://jira.onap.org/secure/attachment/11419/arm_deploy_ons_sut_parameters.json wget https://jira.onap.org/secure/attachment/11420/arm_deploy_ons_sut.json # either run the entrypoint which creates a resource template and runs the stack - or do those two commands manually ./oom_deployment.sh -b master -s azure.onap.cloud -e onap -r a_auto-youruserid_20180421 -t arm_deploy_ons_sut.json -p arm_deploy_ons_sut_parameters.json # wait for the VM to finish in about 75 min or watch progress by ssh'ing into the vm and doing root@ons-auto-201803181110z: sudo tail -f /var/lib/waagent/custom-script/download/0/stdout # if you wish to run the oom_entrypoint script yourself - edit/break the cloud init section at the end of the arm template and do it yourself below # download and edit values.yaml with your onap preferences and openstack tenant config wget https://jira.onap.org/secure/attachment/11414/values.yaml # download and run the bootstrap and onap install script, the -s server name can be an IP, FQDN or hostname wget https://jira.onap.org/secure/attachment/11518/oom_entrypoint.sh chmod 777 oom_entrypoint.sh sudo ./oom_entrypoint.sh -b master -s devops.onap.info -e onap # wait 15 min for rancher to finish, then 30-90 min for onap to come up |
see https://jira.onap.org/secure/attachment/11455/oom_openstack.yaml and https://jira.onap.org/secure/attachment/11454/oom_openstack_oom.env
see https://jira.onap.org/secure/attachment/11439/oom_entrypoint.sh
customize your template (true/false for any components, docker overrides etc...)
https://jira.onap.org/secure/attachment/11414/values.yaml
Run oom_entrypoint.sh after you verified values.yaml - it will run both scripts below for you - a single node kubernetes setup running what you configured in values.yaml will be up in 50-90 min. If you want to just configure your vm without bringing up ONAP - comment out the cd.sh line and run that separately.
see https://jira.onap.org/secure/attachment/11438/oom_rancher_setup.sh
see https://jira.onap.org/secure/attachment/11413/cd.sh
Verify your system is up by doing a kubectl get pods --all-namespaces and checking the 8880 port to bring up the rancher or kubernetes gui.
https://portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.Resources%2Fresources
see
az group create --name onap_eastus --location eastus |
az group deployment create --resource-group onap_eastus --template-file oom_azure_arm_deploy.json --parameters @oom_azure_arm_deploy_parameters.json |
The oom_entrypoint.sh script will be run as a cloud-init script on the VM - see
which runs
see
kubectl get pods --all-namespaces # raise/lower onap components from the installed directory if using the oneclick arm template # amsterdam only root@ons-auto-master-201803191429z:/var/lib/waagent/custom-script/download/0/oom/kubernetes/oneclick# ./createAll.bash -n onap |
Azure subscription
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest
Install homebrew first (reinstall if you are on the latest OSX 10.13.2 https://github.com/Homebrew/install because of 3718)
Will install Python 3.6
$brew update $brew install azure-cli |
https://docs.microsoft.com/en-us/cli/azure/get-started-with-azure-cli?view=azure-cli-latest
$ az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E..D to authenticate. [ { "cloudName": "AzureCloud", "id": "f4...b", "isDefault": true, "name": "Pay-As-You-Go", "state": "Enabled", "tenantId": "bcb.....f", "user": { "name": "michael@....org", "type": "user" }}] |
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-apt?view=azure-cli-latest
# in root AZ_REPO=$(lsb_release -cs) echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | sudo tee /etc/apt/sources.list.d/azure-cli.list apt-key adv --keyserver packages.microsoft.com --recv-keys 52E16F86FEE04B979B07E28DB02C46DF417A0893 apt-get install apt-transport-https apt-get update && sudo apt-get install azure-cli az login # verify root@ons-dmz:~# ps -ef | grep az root 1427 1 0 Mar17 ? 00:00:00 /usr/lib/linux-tools/4.13.0-1011-azure/hv_vss_daemon -n |
https://docs.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest
Follow https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-create-first-template
$ az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...Z to authenticate. $ az group create --name examplegroup --location "South Central US" { "id": "/subscriptions/f4b...e8b/resourceGroups/examplegroup", "location": "southcentralus", "managedBy": null, "name": "examplegroup", "properties": { "provisioningState": "Succeeded" }, "tags": null } obrien:obrienlabs amdocs$ vi azuredeploy_storageaccount.json obrien:obrienlabs amdocs$ az group deployment create --resource-group examplegroup --template-file azuredeploy_storageaccount.json { "id": "/subscriptions/f4...e8b/resourceGroups/examplegroup/providers/Microsoft.Resources/deployments/azuredeploy_storageaccount", "name": "azuredeploy_storageaccount", "properties": { "additionalProperties": { "duration": "PT32.9822642S", "outputResources": [ { "id": "/subscriptions/f4..e8b/resourceGroups/examplegroup/providers/Microsoft.Storage/storageAccounts/storagekj6....kk2w", "resourceGroup": "examplegroup" }], "templateHash": "11440483235727994285"}, "correlationId": "41a0f79..90c291", "debugSetting": null, "dependencies": [], "mode": "Incremental", "outputs": {}, "parameters": {}, "parametersLink": null, "providers": [ { "id": null, "namespace": "Microsoft.Storage", "registrationState": null, "resourceTypes": [ { "aliases": null, "apiVersions": null, "locations": [ "southcentralus" ], "properties": null, "resourceType": "storageAccounts" }]}], "provisioningState": "Succeeded", "template": null, "templateLink": null, "timestamp": "2018-02-17T16:15:11.562170+00:00" }, "resourceGroup": "examplegroup"} |
az account list-locations northcentralus for example |
# create a resource group if not already there az group create --name obrien_jenkins_b_westus2 --location westus2 |
We need a 128G VM with at least 8vCores (peak is 60) and a 100+GB drive. The sizes are detailed on https://docs.microsoft.com/en-ca/azure/virtual-machines/windows/sizes-memory - we will use the Standard_D32s_v3 type
We need an "all open 0.0.0.0/0" security group and a reassociated data drive as boot drive - see the arm template in OOM-711
see open review in
"ubuntuOSVersion": "16.04.0-LTS" "imagePublisher": "Canonical", "imageOffer": "UbuntuServer", "vmSize": "Standard_E8s_v3" "osDisk": {"createOption": "FromImage"},"dataDisks": [{"diskSizeGB": 511,"lun": 0, "createOption": "Empty" }] |
Follow
https://github.com/Azure/azure-quickstart-templates/tree/master/101-acs-kubernetes
https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-template-deploy
https://github.com/Azure/azure-quickstart-templates/tree/master/101-vm-simple-linux
It needs a security group https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-create-nsg-arm-template
{ "apiVersion": "2017-03-01", "type": "Microsoft.Network/networkSecurityGroups", "name": "[variables('networkSecurityGroupName')]", "location": "[resourceGroup().location]", "tags": { "displayName": "NSG - Front End" }, "properties": { "securityRules": [ { "name": "in-rule", "properties": { "description": "All in", "protocol": "Tcp", "sourcePortRange": "*", "destinationPortRange": "*", "sourceAddressPrefix": "Internet", "destinationAddressPrefix": "*", "access": "Allow", "priority": 100, "direction": "Inbound" } }, { "name": "out-rule", "properties": { "description": "All out", "protocol": "Tcp", "sourcePortRange": "*", "destinationPortRange": "*", "sourceAddressPrefix": "Internet", "destinationAddressPrefix": "*", "access": "Allow", "priority": 101, "direction": "Outbound" } } ] } } , { "apiVersion": "2017-04-01", "type": "Microsoft.Network/virtualNetworks", "name": "[variables('virtualNetworkName')]", "location": "[resourceGroup().location]", "dependson": [ "[concat('Microsoft.Network/networkSecurityGroups/', variables('networkSecurityGroupName'))]" ], "properties": { "addressSpace": { "addressPrefixes": [ "[variables('addressPrefix')]" ] }, "subnets": [ { "name": "[variables('subnetName')]", "properties": { "addressPrefix": "[variables('subnetPrefix')]", "networkSecurityGroup": { "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]" } } } ] } }, |
# validate first (validate instead of create) az group deployment create --resource-group obrien_jenkins_b_westus2 --template-file oom_azure_arm_deploy.json --parameters @oom_azure_arm_cd_amsterdam_deploy_parameters.json |
Use the entrypoint script in
# clone the oom repo to get the install directory git clone https://gerrit.onap.org/r/oom # run the Rancher RI installation (to install kubernetes) oom/install/rancher/oom_rancher_install.sh -b master -s 192.168.240.32 -e onap # run the oom deployment script # get a copy of onap-parametes.yaml and place in this folder oom/install/deployment/cd.sh -b master -s 192.168.240.32 -e onap |
oom_rancher_install.sh is in under https://gerrit.onap.org/r/#/c/32019/
cd.sh in under https://gerrit.onap.org/r/#/c/32653/
# delete the vm and resources az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy # the above deletion will not delete the actual resources - only a delete of the group or each individual resource works # optionally delete the resource group az group delete --name ONAPAMDOCS -y |
az network public-ip create --name onap-argon --resource-group a_ONAP_argon_prod_donotdelete --location eastus --allocation-method Static
Follow https://docs.microsoft.com/en-us/azure/aks/tutorial-kubernetes-deploy-cluster
obrienbiometrics:obrienlabs michaelobrien$ az provider register -n Microsoft.ContainerService Registering is still on-going. You can monitor using 'az provider show -n Microsoft.ContainerService' |
http://aka.ms/corequotaincrease
https://portal.azure.com/#blade/Microsoft_Azure_Support/HelpAndSupportBlade/newsupportrequest
Deployment failed. Correlation ID: 4b4707a7-2244-4557-855e-11bcced556de. Provisioning of resource(s) for container service onapAKSCluster in resource group onapAKS failed. Message: Operation results in exceeding quota limits of Core. Maximum allowed: 10, Current in use: 10, Additional requested: 1. Please read more about quota increase at http://aka.ms/corequotaincrease.. Details: |
obrienbiometrics:obrienlabs michaelobrien$ az aks create --resource-group onapAKS --name onapAKSCluster --node-count 1 --generate-ssh-keys - Running .. "fqdn": "onapaksclu-onapaks-f4....3.hcp.eastus.azmk8s.io", |
The cluster will start with a 3.5G VM before scaling
Resources for your AKS cluster
A resource group makes it easier to package and remove everything for a deployment - essentially making the deployment stateless
Global or local to the resource group?
Register a CNAME for an existing domain and use the same IP address everytime the deployment comes up
How to attach the cloud init script to provision the VM
ARM template chaining
passing derived varialbles into the next arm template - for example when bringing up an entire federated set in one or more DCs
see script attached to
It takes about 2 min for DNS entries to propagate out from A record DNS changes. For example the following IP/DNS association took 2 min to appear in dig.
obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info ; <<>> DiG 9.9.7-P3 <<>> azure.onap.info ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 10599 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;azure.onap.info. IN A ;; ANSWER SECTION: azure.onap.info. 251 IN A 52.224.233.230 ;; Query time: 68 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Tue Feb 20 10:26:59 EST 2018 ;; MSG SIZE rcvd: 60 obrienbiometrics:onap_oom_711_azure michaelobrien$ dig azure.onap.info ; <<>> DiG 9.9.7-P3 <<>> azure.onap.info ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30447 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 512 ;; QUESTION SECTION: ;azure.onap.info. IN A ;; ANSWER SECTION: azure.onap.info. 299 IN A 13.92.225.167 ;; Query time: 84 msec ;; SERVER: 8.8.8.8#53(8.8.8.8) ;; WHEN: Tue Feb 20 10:27:04 EST 2018 |
Inside the corporate firewall - avoid it PS C:\> az login Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443) : Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by NewConnectionError('<urllib3.conne ction.VerifiedHTTPSConnection object at 0x04D18730>: Failed to establish a new connection: [Errno 11001] getaddrinfo fai led',)) at home or cell hotspot PS C:\> az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E...2W to authenticate. [ { "cloudName": "AzureCloud", "id": "4...da1", "isDefault": true, "name": "Microsoft Azure Internal Consumption", "state": "Enabled", "tenantId": "72f98....47", "user": { "name": "fran...ocs.com", "type": "user" }] On corporate account (need permissions bump to be able to create a resource group prior to running an arm template https://wiki.onap.org/display/DW/ONAP+on+Kubernetes+on+Microsoft+Azure#ONAPonKubernetesonMicrosoftAzure-ARMTemplate PS C:\> az group create --name onapKubernetes --location eastus The client 'fra...s.com' with object id '08f98c7e-...ed' does not have authorization to per form action 'Microsoft.Resources/subscriptions/resourcegroups/write' over scope '/subscriptions/42e...8 7da1/resourcegroups/onapKubernetes'. try my personal = OK PS C:\> az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code EE...ULR to authenticate. Terminate batch job (Y/N)? y # hangs when first time login in a new pc PS C:\> az login To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code E.PBKS to authenticate. [ { "cloudName": "AzureCloud", "id": "f4b...b", "isDefault": true "name": "Pay-As-You-Go", "state": "Enabled", "tenantId": "bcb...f4f", "user": "name": "michael@obrien...org", "type": "user" } }] PS C:\> az group create --name onapKubernetes2 --location eastus { "id": "/subscriptions/f4b....b/resourceGroups/onapKubernetes2", "location": "eastus", "managedBy": null, "name": "onapKubernetes2", "properties": { "provisioningState": "Succeeded" }, "tags": null} |
I find that a delete deployment deletes the deployment but not the actual resources. The workaround is to delete the resource group - but in some constrained subscriptions the cli user may not have the ability to create a resource group - and hence delete it.
see
https://github.com/Azure/azure-sdk-for-java/issues/1167
deleting the resources manually for now - is a workaround if you cannot create/delete resource groups
# delete the vm and resources az group deployment delete --resource-group ONAPAMDOCS --name oom_azure_arm_deploy
# the above deletion will not delete the actual resources - only a delete of the group or each individual resource works
# optionally delete the resource group az group delete --name ONAPAMDOCS -y
However modifying the template to add resources works well. For example adding a reference to a network security group
20180228: Resize the OS disk
ONAP requires at least 75g - the issue is than in most VM templates on Azure - the OS disk is 30g - we need to either switch to the data disk or resize the os disk.
# add diskSizeGB to the template "osDisk": { "diskSizeGB": 255, "createOption": "FromImage" }, ubuntu@oom-auto-deploy:~$ df Filesystem 1K-blocks Used Available Use% Mounted on udev 65989400 0 65989400 0% /dev tmpfs 13201856 8848 13193008 1% /run /dev/sda1 259142960 1339056 257787520 1% / tmpfs 66009280 0 66009280 0% /dev/shm tmpfs 5120 0 5120 0% /run/lock tmpfs 66009280 0 66009280 0% /sys/fs/cgroup none 64 0 64 0% /etc/network/interfaces.dynamic.d /dev/sdb1 264091588 60508 250592980 1% /mnt tmpfs 13201856 0 13201856 0% /run/user/1000 ubuntu@oom-auto-deploy:~$ free total used free shared buff/cache available Mem: 132018560 392336 131242164 8876 384060 131012328 |
in review under OOM-715
https://jira.onap.org/secure/attachment/11206/oom_entrypoint.sh
If using amsterdam - swap out the onap-parameters.yaml (the curl is hardcoded to a master branch version)
use this method instead of installing az cli directly - for certain corporate oauth configurations
https://azure.microsoft.com/en-us/features/storage-explorer/
Install AZM using the name and access key of a storage account created manually or by enabling the az cli on the browser
See https://docs.microsoft.com/en-us/azure/templates/microsoft.compute/virtualmachines/extensions it looks like Azure has a similar setup to AWS ebextentions
Targetting
type | string | No | Specifies the type of the extension; an example is "CustomScriptExtension". |
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/extensions-customscript
deprecated { "apiVersion": "2015-06-15", "type": "Microsoft.Compute/virtualMachines/extensions", "name": "[concat(parameters('vmName'),'/onap')]", "location": "[resourceGroup().location]", "dependsOn": ["[concat('Microsoft.Compute/virtualMachines/', parameters('vmName'))]"], "properties": { "publisher": "Microsoft.Azure.Extensions", "type": "CustomScript", "typeHandlerVersion": "1.9", "autoUpgradeMinorVersion": true, "settings": { "fileUris": [ "https://jira.onap.org/secure/attachment/11263/oom_entrypoint.sh" ], "commandToExecute": "[concat('./' , parameters('scriptName'), ' -b master -s dns/pub/pri-ip -e onap' )]" } } } use { "apiVersion": "2017-12-01", "type": "Microsoft.Compute/virtualMachines/extensions", "name": "[concat(parameters('vmName'),'/onap')]", "location": "[resourceGroup().location]", "dependsOn": ["[concat('Microsoft.Compute/virtualMachines/', parameters('vmName'))]"], "properties": { "publisher": "Microsoft.Azure.Extensions", "type": "CustomScript", "typeHandlerVersion": "2.0", "autoUpgradeMinorVersion": true, "settings": { "fileUris": [ "https://jira.onap.org/secure/attachment/11281/oom_entrypoint.sh" ], "commandToExecute": "[concat('./' , parameters('scriptName'), ' -b master ', ' -s ', 'ons-auto-201803181110z', ' -e onap' )]" } } } |
ubuntu@ons-dmz:~$ ./oom_deployment.sh
Deployment template validation failed: 'The template resource 'entrypoint' for type 'Microsoft.Compute/virtualMachines/extensions' at line '1' and column '6182' has incorrect segment lengths. A nested resource type must have identical number of segments as its resource name. A root resource type must have segment length one greater than its resource name. Please see https://aka.ms/arm-template/#resources for usage details.'.
ubuntu@ons-dmz:~$ ./oom_deployment.sh
Deployment failed. Correlation ID: 532b9a9b-e0e8-4184-9e46-6c2e7c15e7c7. {
"error": {
"code": "ParentResourceNotFound",
"message": "Can not perform requested operation on nested resource. Parent resource '[concat(parameters('vmName'),'' not found."
}
}
fixed 20180318:1600
Install runs - but I need visibility - checking /var/lib/waagent/custom-script/download/0/
progress
./oom_deployment.sh # 7 min to delete old deployment ubuntu@ons-dmz:~$ az vm extension list -g a_ONAP_auto_201803181110z --vm-name ons-auto-201803181110z .. "provisioningState": "Creating", "settings": { "commandToExecute": "./oom_entrypoint.sh -b master -s ons-auto-201803181110zons-auto-201803181110z.eastus.cloudapp.azure.com -e onap", "fileUris": [ "https://jira.onap.org/secure/attachment/11263/oom_entrypoint.sh" ubuntu@ons-auto-201803181110z:~$ sudo su - root@ons-auto-201803181110z:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 83458596d7a6 rancher/server:v1.6.14 "/usr/bin/entry /u..." 3 minutes ago Up 3 minutes 3306/tcp, 0.0.0.0:8880->8080/tcp rancher_server root@ons-auto-201803181110z:~# tail -f /var/log/azure/custom-script/handler.log time=2018-03-18T22:51:59Z version=v2.0.6/git@1008306-clean operation=enable seq=0 file=0 event="download complete" output=/var/lib/waagent/custom-script/download/0 time=2018-03-18T22:51:59Z version=v2.0.6/git@1008306-clean operation=enable seq=0 event="executing command" output=/var/lib/waagent/custom-script/download/0 time=2018-03-18T22:51:59Z version=v2.0.6/git@1008306-clean operation=enable seq=0 event="executing public commandToExecute" output=/var/lib/waagent/custom-script/download/0 root@ons-auto-201803181110z:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 539733f24c01 rancher/agent:v1.2.9 "/run.sh run" 13 seconds ago Up 13 seconds rancher-agent 83458596d7a6 rancher/server:v1.6.14 "/usr/bin/entry /u..." 5 minutes ago Up 5 minutes 3306/tcp, 0.0.0.0:8880->8080/tcp rancher_server root@ons-auto-201803181110z:~# ls -la /var/lib/waagent/custom-script/download/0/ total 31616 -rw-r--r-- 1 root root 16325186 Aug 31 2017 helm-v2.6.1-linux-amd64.tar.gz -rw-r--r-- 1 root root 4 Mar 18 22:55 kube_env_id.json drwxrwxr-x 2 ubuntu ubuntu 4096 Mar 18 22:53 linux-amd64 -r-x------ 1 root root 2822 Mar 18 22:51 oom_entrypoint.sh -rwxrwxrwx 1 root root 7288 Mar 18 22:52 oom_rancher_setup.sh -rwxr-xr-x 1 root root 12213376 Mar 18 22:53 rancher -rw-r--r-- 1 root root 3736787 Dec 20 19:41 rancher-linux-amd64-v0.6.7.tar.gz drwxr-xr-x 2 root root 4096 Dec 20 19:39 rancher-v0.6.7 |
testing via http://jenkins.onap.cloud/job/oom_azure_deployment/
Need the ip address and not the domain name - via linked template
or
https://docs.microsoft.com/en-us/azure/templates/microsoft.network/publicipaddresses
https://github.com/Azure/azure-quickstart-templates/issues/583
Arm templates cannot specify a static ip - without a private subnet
reference(variables('publicIPAddressName')).ipAddress
for
reference(variables('nicName')).ipConfigurations[0].properties.privateIPAddress
Using the hostname instead of the private/public ip works (verify /etc/hosts though)
obrienbiometrics:oom michaelobrien$ ssh ubuntu@13.90.207.60 ubuntu@ons-auto-201803181110z:~$ sudo su - root@ons-auto-201803181110z:/var/lib/waagent/custom-script/download/0# cat stdout INFO: Running Agent Registration Process, CATTLE_URL=http://ons-auto-201803181110z:8880/v1 INFO: Attempting to connect to: http://ons-auto-201803181110z:8880/v1 INFO: http://ons-auto-201803181110z:8880/v1 is accessible INFO: Inspecting host capabilities INFO: Boot2Docker: false INFO: Host writable: true INFO: Token: xxxxxxxx INFO: Running registration INFO: Printing Environment INFO: ENV: CATTLE_ACCESS_KEY=9B0FA1695A3E3CFD07DB INFO: ENV: CATTLE_HOME=/var/lib/cattle INFO: ENV: CATTLE_REGISTRATION_ACCESS_KEY=registrationToken INFO: ENV: CATTLE_REGISTRATION_SECRET_KEY=xxxxxxx INFO: ENV: CATTLE_SECRET_KEY=xxxxxxx INFO: ENV: CATTLE_URL=http://ons-auto-201803181110z:8880/v1 INFO: ENV: DETECTED_CATTLE_AGENT_IP=172.17.0.1 INFO: ENV: RANCHER_AGENT_IMAGE=rancher/agent:v1.2.9 INFO: Launched Rancher Agent: b44bd62fd21c961f32f642f7c3b24438fc4129eabbd1f91e1cf58b0ed30b5876 waiting 7 min for host registration to finish 1 more min KUBECTL_TOKEN base64 encoded: QmFzaWMgUWpBNE5EWkdRVE5HTmpKRVFVRkZOa1k1UlRNNlMzZzBWRTVUZVc5VGEycFhaVE5HTm5rNWNGWlhVRWg0TVVkV1ZEbDVTRmQxUkc1d2MwWTJRZz09 run the following if you installed a higher kubectl version than the server helm init --upgrade Verify all pods up on the kubernetes system - will return localhost:8080 until a host is added kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system heapster-76b8cd7b5-v5jrd 1/1 Running 0 5m kube-system kube-dns-5d7b4487c9-9bwk5 3/3 Running 0 5m kube-system kubernetes-dashboard-f9577fffd-cpwv7 1/1 Running 0 5m kube-system monitoring-grafana-997796fcf-s4sjm 1/1 Running 0 5m kube-system monitoring-influxdb-56fdcd96b-2mn6r 1/1 Running 0 5m kube-system tiller-deploy-cc96d4f6b-fll4t 1/1 Running 0 5m |
In AWS we can select the "no reboot" option and create an image from a running VM as-is with no effect on the running system.
Having issues with the Azure image creator - it is looking for the ubuntu pw even though I only use key based access
aka: travellers guide
If you run into issues doing a make all - your helm server is not running
# rerun helm serve & helm repo add local http://127.0.0.1:8879 |
(from Microsoft - thank you)
General Azure Documentation
Azure Site http://azure.microsoft.com
Azure Documentation Site https://docs.microsoft.com/en-us/azure/
Azure Training Courses https://azure.microsoft.com/en-us/training/free-online-courses/
Azure Portal http://portal.azure.com
Developer Documentation
Azure AD Authentication Libraries https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-authentication-libraries
Java Overview on Azure https://azure.microsoft.com/en-us/develop/java/
Java Docs for Azure https://docs.microsoft.com/en-us/java/azure/
Java SDK on GitHub https://github.com/Azure/azure-sdk-for-java
Python Overview on Azure https://azure.microsoft.com/en-us/develop/python/
Python Docs for Azure https://docs.microsoft.com/en-us/python/azure/
Python SDK on GitHub https://github.com/Azure/azure-sdk-for-python
REST Api and CLI Documentation
REST API Documentation https://docs.microsoft.com/en-us/rest/api/
CLI Documentation https://docs.microsoft.com/en-us/cli/azure/index
Other Documentation
Using Automation for VM shutdown & startup https://docs.microsoft.com/en-us/azure/automation/automation-solution-vm-management
Azure Resource Manager (ARM) QuickStart Templates https://github.com/Azure/azure-quickstart-templates
The code in this github repo has 2 month old copies of cd.sh and oom_rancher_install.sh
https://github.com/taranki/onap-azure
Use the official ONAP code in review under
https://jira.onap.org/browse/OOM-710
https://jira.onap.org/browse/OOM-715
https://gerrit.onap.org/r/32019
https://jira.onap.org/browse/OOM-716
https://gerrit.onap.org/r/32653
The original seed source from 2017 below is deprecated - use onap links above
https://github.com/obrienlabs/onap-root
https://azure.microsoft.com/en-us/services/container-service/
https://docs.microsoft.com/en-us/azure/templates/microsoft.compute/virtualmachines
https://kubernetes.io/docs/concepts/containers/images/#using-azure-container-registry-acr
https://azure.microsoft.com/en-us/features/storage-explorer/
https://docs.microsoft.com/en-ca/azure/virtual-machines/linux/capture-image
Account Provider: Michael O'Brien of Amdocs