...
How to create a new daily chain...
1) Declare the chain in chained-ci inventory
All the chains must be declared in : https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/blob/master/pod_inventory/inventory
...
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
[orange_terrahouat]
oom-offline
openacumos
acumos_sandbox
onap_oom_gating_k8s_pod4_4
onap_oom_gating_k8s_pod4_3
onap_oom_gating_k8s_pod4_2
onap_oom_gating_k8s_pod4_1
onap_oom_gating_vnfs_pod4_4
onap_oom_gating_vnfs_pod4_3
onap_oom_gating_vnfs_pod4_2
onap_oom_gating_vnfs_pod4_1
onap_oom_gating_pod4_4
onap_oom_gating_pod4_3
onap_oom_gating_pod4_2
onap_oom_gating_pod4_1
onap_daily_pod4_k8s_master
onap_daily_pod4_master
onap_weekly_pod4_k8s_master
onap_weekly_pod4_master
onap_daily_pod4_k8s_ingress_master
onap_daily_pod4_ingress_master
onap_xtesting_k8s
onap_pod4_k8s_service_mesh_master
rke_daily_pod4
rke2_daily_pod4
kubespray_daily_pod4
harbor_server
onap_oom_pod4_sm_master
oronap_oom_gating_k8s_pod4_1
hardening_centos_pod4
onap_daily_pod4_k8s_test
onap_daily_pod4_test
onap_weekly_pod4_k8s_honolulu
onap_weekly_pod4_honolulu
onap_daily_pod4_k8s_honolulu
onap_daily_pod4_honolulu
onap_daily_pod4_k8s_istanbul
onap_daily_pod4_istanbul
onap_weekly_pod4_k8s_istanbul
onap_weekly_pod4_istanbul
new_k8s_daily
new_onap_daily
[azure]
onap_oom_gating_k8s_azure_3
onap_oom_gating_k8s_azure_4
onap_oom_staging_k8s_azure_1
onap_oom_gating_azure_3
onap_oom_gating_azure_4
onap_oom_staging_azure_1
|
In the example above I declared 2 new chains: new_k8s_daily and new_onap_daily.
Note that you can declare only one but for the daily we are used to redeploy the k8s prior to the ONAP deployment - we always restart from scratch. it allows to see any regression on OOM/k8s.
2) Create the chains
Once declare you must create several files
Let's start with the host vars https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/tree/master/pod_inventory/host_vars.
You must create the files: new_k8s_daily.yml and new_onap_daily.yml
These files describe the chain you want to setup
Let's consider new_k8s_daily.yml, it could look like
Code Block | ||||||||
---|---|---|---|---|---|---|---|---|
| ||||||||
--- jumphost: server: rebond.opnfv.fr user: !vault | $ANSIBLE_VAULT;1.1;AES256 3434613036643437336466623463383762363761656564383535373037353066363563666634 33633330623732333364666266316630363162333532663666380a3030386264383034643239 3837626339336334313034623561396365626635656565353061666437393330633464333130 3035366639373863643130346133620a32646230636637656338623835306330663036636235 66623232333464643766 environment: orange_pod4/k8s_master scenario_steps: config: project: config get_artifacts: - name: orange_vim_pod4 static_src: true infra: k8s18-new-daily ssh_access: orange.eyml infra_deploy: project: os_infra_manager get_artifacts: config extra_parameters: ADMIN: true CLEAN: true TENANT_NAME: new-daily USER_NAME: new-daily-ci IDENTIFIER: -new-daily USE_PRIVATE_IP: True ADD_FLOATING_IP: True DNS_NAME: "{{ lookup('env','DNS_NAME') | default('master', true) }}" k8s_deploy: get_artifacts: infra_deploy project: kubespray branch: helm_3 ssh_access: orange.eyml extra_parameters: kubespray_version: release-2.18 helm_release: v3.6.4 kube_network_plugin: cilium kubernetes_release: v1.22.4 ENABLE_MONITORING: false DOCKER_HUB_PROXY: docker.nexus.azure.onap.eu GCR_PROXY: docker.nexus.azure.onap.eu K8S_GCR_PROXY: docker.nexus.azure.onap.eu QUAY_PROXY: docker.nexus.azure.onap.eu trigger: project: trigger k8s_test: project: functest_k8s get_artifacts: - name: infra_deploy limit_to: - inventory/infra: inventory/infra - name: config limit_to: - vars/pdf.yml: vars/pdf.yml - vars/ssh_gateways.yml: vars/ssh_gateways.yml - vars/vaulted_ssh_credentials.yml: vars/vaulted_ssh_credentials.yml - name: k8s_deploy limit_to: - vars/kube-config: vars/kube-config extra_parameters: DEPLOYMENT: kubespray TEST_RESULT_DB_URL: http://testresults.opnfv.org/test/api/v1/results |
There are lots of information in this file, it describes the different stages of the kubernetes installation: config (init), infra_deploy (creation of the VMs on Orange OpenStack), k8s_deploy (deployement of the kubernetes), k8s_test (kubernetes testing).
For each stage we may change some parameters..
- on the config part: the full configration of the VM is indicated through the parameter infra: k8s18-new-daily (see next section)
- on the VM creation for instance we can change the name of the tenants/vms/users/...
- on the kubernetes installation we can change the versions of kubespray, helm, the network plugin,the kubernetes release...This section must be in line with the OOM recommendations
- We also do precise the docker repository (here we use our internal mirror)
Same for the ONAP installation (chained once the kubernetes is installed)
Code Block | ||||||
---|---|---|---|---|---|---|
| ||||||
--- jumphost: server: rebond.opnfv.fr user: !vault | $ANSIBLE_VAULT;1.1;AES256 3434613036643437336466623463383762363761656564383535373037353066363563666634 33633330623732333364666266316630363162333532663666380a3030386264383034643239 3837626339336334313034623561396365626635656565353061666437393330633464333130 3035366639373863643130346133620a32646230636637656338623835306330663036636235 66623232333464643766 environment: orange_pod4/k8s_master/onap_daily inpod: onap_daily_pod4_k8s_master scenario_steps: config: project: config get_artifacts: - name: orange_vim_pod4 static_src: true infra: onap-vnfs ssh_access: orange.eyml vnf_project_deploy: project: os_infra_manager get_artifacts: config extra_parameters: ADMIN: true CLEAN: true TENANT_NAME: onap-master-daily-vnfs USER_NAME: onap-master-daily-vnfs-ci IDENTIFIER: -onap NETWORK_IDENTIFIER: NONE onap_deploy: branch: master extra_parameters: OOM_BRANCH: master ONAP_REPOSITORY: nexus3.onap.org:10001 ONAP_FLAVOR: small DOCKER_HUB_PROXY: docker.nexus.azure.onap.eu ELASTIC_PROXY: docker.nexus.azure.onap.eu K8S_GCR_PROXY: docker.nexus.azure.onap.eu get_artifacts: - name: vnf_project_deploy limit_to: - vars/user_cloud.yml: vars/user_cloud.yml - name: infra_deploy:onap_daily_pod4_k8s_master in_pipeline: false limit_to: - inventory/infra: inventory/infra - name: config:onap_daily_pod4_k8s_master in_pipeline: false limit_to: - vars/pdf.yml: vars/pdf.yml - vars/idf.yml: vars/idf.yml #- vars/ddf.yml: vars/ddf.yml - name: config limit_to: - vars/vim.yml: vars/vim.yml - vars/ssh_gateways.yml: vars/ssh_gateways.yml - vars/vaulted_ssh_credentials.yml: vars/vaulted_ssh_credentials.yml project: oom onap_test: project: xtesting-onap branch: master get_artifacts: - name: infra_deploy:onap_daily_pod4_k8s_master in_pipeline: false limit_to: - inventory/infra: inventory/infra - name: config:onap_daily_pod4_k8s_master in_pipeline: false limit_to: - vars/pdf.yml: vars/pdf.yml - name: k8s_deploy:onap_daily_pod4_k8s_master in_pipeline: false limit_to: - vars/kube-config: vars/kube-config - name: onap_deploy limit_to: - vars/cluster.yml: vars/cluster.yml - name: config limit_to: - vars/vim.yml: vars/vim.yml - vars/ssh_gateways.yml: vars/ssh_gateways.yml - vars/vaulted_ssh_credentials.yml: vars/vaulted_ssh_credentials.yml extra_parameters: DEPLOYMENT: oom INFRA_DEPLOYMENT: kubespray DEPLOYMENT_TYPE: full DEPLOY_SCENARIO: onap-ftw RANDOM_WAIT: True TEST_RESULT_DB_URL: http://testresults.opnfv.org/onap/api/v1/results |
We find back the notion of stages: config (retrieve information from VMs and k8s installation), vnf_project_deploy (create a tenant for the test), onap_deploy (ONAP deployment), onap_test (tests to be executed on ONAP linked to the gitlab project onap_test)
Here again you can modify some parameters. in the onap_deploy stage you can specify the branch of the oom_installer (here master) and the branch of OOM
3) Resources definition
In the k8s chain definition, a parameter infra was set to precise the resources. These resources are defined in the associated pdf and idf files that can be found in https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/tree/master/pod_config/config
If we consider for instances k8s-onap-master, the 2 files are:
- Infrastructure Description File (IDF): https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/blob/master/pod_config/config/idf-k8s18-onap-master.yaml
- Platform Description File (PDF): https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/blob/master/pod_config/config/k8s18-onap-master.yaml
the parameters defined here are used by the collection infra managers to create the VM and or deploying infrastructure and platform components.
4) Summary
If you want to create a new daily
- clone chained-ci repo
- git clone https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci.git
- add onap-daily-k8s-x and onap-daily-x in the inventory https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci/-/blob/master/pod_inventory/inventory
- copy/paste master host_vars
- cd chained-ci/pod_inventory/host_vars
- cp onap_daily_pod4_k8s_master.yml onap_daily_x_k8s.yml
- cp onap_daily_pod4_master.yml onap_daily_x.yml
- Edit and adapt the chains
- e.g. change the OOM branch to x in onap_daily_x.yml, update the versions of kubernetes, helm,..according to OOM recommentations
- if resources changes, create an idf and pdf file in chained-ci/pod_config/config/ (note if resources are unchanged you may just reuse existing ones or copy paste existing ones for clarity)
- cp chained-ci/pod_config/config/idf-k8s18-onap-master.yaml chained-ci/pod_config/config/idf-k8s18-onap-x.yaml
- cp chained-ci/pod_config/config/k8s18-onap-master.yaml chained-ci/pod_config/config/k8s18-onap-x.yaml
- and then reference infra: k8s18-onap-x in onap_daily_x_k8s.yml
- add a schedule in gitlab vi with TARGET=onap-daily-x