...
install python-pip and other python modules (see the comment section)
apt install python-pip
pip install ipaddress
pip install pyyaml
pip install mysql-connector-python
pip install progressbar2
pip install python-novaclient
pip install python-openstackclient
pip install kubernetes
Run automation program to deploy services
Sign into SDC as designer and download five csar files for infra, vbng, vgmux, vbrg, and rescust. Copy all the csar files to directory csar.
If robot has done the model onboardin for you the CSARs may also be inside the robot container in the /tmp/csar directory.
Now you can simply run 'vcpe.py' to see the instructions.
...
service_instance_id: Take from __var/svc_instance_uuid file. Copy the value for gmux without letter 'V'.
| Code Block |
|---|
#demo-k8s.sh <namespace> heatbridge <stack_name> <service_instance_id> <service> <oam-ip-address>
root@oom-rancher:~/integration/test/vcpe# ~/oom/kubernetes/robot/demo-k8s.sh onap heatbridge vcpe_vfmodule_e2744f48729e4072b20b_201811262136 d8914ef3-3fdb-4401-adfe-823ee75dc604 vCPEvGMUX 10.0.101.21
|
...
10. Run `vcpe.py infra`
11. Make sure sniro configuration is run as part of the above step.
12. Install curl command inside sdnc-sdnc-0 container
13. Run `healthcheck-k8s.py onap` to check connectivity from sdnc to brg and gmux. If healthcheck-k8s.sh fails, check /opt/config/sdnc_ip.txt to see it has the SDNC host ip correctly. If you need to change SDNC host ip, you need to clean up and rerun `vcpe.py infra`. Also verify tap interfaces tap-0 and tap-1 are up by running vppctl with show int command. If tap interfaces are not up, use vppctl tap delete tap-0 and tap-1 and then run `/opt/bind_nic.sh` followed by `/opt/set_nat.sh`.
.
If you have changed the SDNC_IP after instantiation of the vBNG and vBRGEMU:
- you need to also update the /opt/sdnc_ip in the vBNG and run v_bng_install.sh to get the vBNG route tables updated.
- you need to change sdnc_ip.txt and ip.txt on the vBRGEMU
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@zdcpe1cpe01brgemu01-201812261515:~# vppctl tap delete tap-0 Deleted. root@zdcpe1cpe01brgemu01-201812261515:~# vppctl tap delete tap-1 Deleted. root@zdcpe1cpe01brgemu01-201812261515:~# /opt/bind_nic.sh root@zdcpe1cpe01brgemu01-201812261515:~# /opt/set_nat.sh root@zdcpe1cpe01brgemu01-201812261515:~# vppctl [WAIT A FEW SECONDS BEFORE DOING NEXT STEPS or you may get an error since vppctl lstack returns error.] root@zdcpe1cpe01brgemu01-201812261515:~# /opt/bind_nic.sh root@zdcpe1cpe01brgemu01-201812261515:~# /opt/set_nat.sh root@zdcpe1cpe01brgemu01-201812261515:~# vppctl show int Name Idx State Counter Count GigabitEthernet0/4/0 1 up tx packets 12 tx bytes 3912 local0 0 down tap-0 2 up rx packets 5 rx bytes 410 drops 7 ip6 1 tap-1 3 up rx packets 1 rx bytes 70 drops 7 ip6 1 |
...
23. To repeat create infra step, you can delete infra vf-module stacks first and the network stacks from Openstack Horizon Orchestration->Stack page, then clean up the record in sdnc DHCP_MAC table before rerun `vcpe.py infra`
24. To repeat create customer step, you can delete customer stack, then clear up tunnles by running `cleanGMUX.py gmux_public_ip` and `cleanGMUX.py brg_public_ip`. After that you can rerun create customer command
Checklist for Dublin Release
25. If SDNC needs to be redeployed, you need again to distribute service model from SDC UI, create ip pool, install curl, and set SDNC VM cluster node routing table. Then you should reinstantiate infra VNFs, otherwise you would need to change sdnc ip address in VNFs for snat config.
Checklist for Dublin and El Alto Releases
- Model distribution by `demo-k8s.sh onap init`. this will onboard VNFs and 4 services, i.e. infrastructure, Model distribution by `demo-k8s.sh onap init`. this will onboard VNFs and 4 services, i.e. infrastructure, brg, bng and gmux
- Run Robot `ete-k8s.sh onap distributevCPEResCust`. This step assumes step 1 successfully distributed the 4 models
- Add customer SDN-ETHERNET-INTERNET (need to put into vcpe init)
- Add identity-url to RegionOne
- Add route on sdnc cluster node `ip route add 10.3.0.0/24 via 10.0.101.10 dev ens3`
- Initialize SDNC ip pool by running from Rancher node `kubectl -n onap exec -it dev-sdnc-sdnc-0 -- /opt/sdnc/bin/addIpAddresses.sh VGW 10.5.0 22 250`
- Install python and other python libraries
- In El Alto this can be done via ~integration/test/vcpe/bin/setup.sh
- Change the openstack env parameters and the customer service related parameter in vcpecommon.py
- Run `vcpe.py init`
- Make sure to Change vgw_VfModuleModelInvariantUuid in vcpecommon.py based on the CSAR - it changes for every CSAR
- Run `vcpe.py init`
- Insert the custom Insert a service workflow entry in SO catalogdb
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@sb04-rancher:~# kubectl exec dev-mariadb-galera-mariadb-galera-0 -- mysql -uroot -psecretpassword -e "INSERT INTO catalogdb.service_recipe (ACTION, VERSION_STR, DESCRIPTION, ORCHESTRATION_URI, SERVICE_PARAM_XSD, RECIPE_TIMEOUT, SERVICE_TIMEOUT_INTERIM, CREATION_TIMESTAMP, SERVICE_MODEL_UUID) VALUES ('createInstance','1','vCPEResCust 2019-06-03 _04ba','/mso/async/services/CreateVcpeResCustService',NULL,181,NULL, NOW(),'6c4a469d-ca2c-4b02-8cf1-bd02e9c5a7ce')"
|
Typical Errors and Solutions
SDNC DG error
If you run vcpe.py customer and see an error similar to the following:
"requestId": "c772d346-ae34-4e88-b419-0af510c4526a",
"requestScope": "service",
"requestStatus": {
"finishTime": "Wed, 22 Aug 2018 18:46:09 GMT",
"percentProgress": 100,
"requestState": "FAILED",
"statusMessage": "Received error from SDN-C: Not Found"
10. Run `vcpe.py infra`
11. Install curl command inside sdnc-sdnc-0 container
12. From Rancher node run `healthcheck-k8s.py onap` to check connectivity from sdnc to brg and gmux
13. Update libevel.so in vGMUX
14. Run heatbridge
15. Push new Policy. Follow Jorge's steps in It is most likely due to an error in vnf-topology-assign DG. This happends in R2 and should have been fixed in R3 (refer
| Jira | ||||||
|---|---|---|---|---|---|---|
|
|
| Code Block | ||||
|---|---|---|---|---|
| ||||
root@dev-robot-robot-66c9dbc759-8j7lr:/# curl -k --silent --user 'healthcheck:zb!XztG34' -X POST "https://policy-api:6969/policy/api/v1/policytypes/onap.policies.controlloop.Operational/versions/1.0.0/policies" -H "Accept: application/json" -H "Content-Type: application/json" -d @operational.vcpe.json.txt
{"policy-id":"operational.vcpe","policy-version":"1","content":"controlLoop%3A%0D%0A++version%3A+2.0.0%0D%0A++controlLoopName%3A+ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e%0D%0A++trigger_policy%3A+unique-policy-id-1-restart%0D%0A++timeout%3A+3600%0D%0A++abatement%3A+true%0D%0A+%0D%0Apolicies%3A%0D%0A++-+id%3A+unique-policy-id-1-restart%0D%0A++++name%3A+Restart+the+VM%0D%0A++++description%3A%0D%0A++++actor%3A+APPC%0D%0A++++recipe%3A+Restart%0D%0A++++target%3A%0D%0A++++++type%3A+VM%0D%0A++++retry%3A+3%0D%0A++++timeout%3A+1200%0D%0A++++success%3A+final_success%0D%0A++++failure%3A+final_failure%0D%0A++++failure_timeout%3A+final_failure_timeout%0D%0A++++failure_retries%3A+final_failure_retries%0D%0A++++failure_exception%3A+final_failure_exception%0D%0A++++failure_guard%3A+final_failure_guard"}
root@dev-robot-robot-66c9dbc759-8j7lr:/# curl --silent -k --user 'healthcheck:zb!XztG34' -X POST "https://policy-pap:6969/policy/pap/v1/pdps/policies" -H "Accept: application/json" -H "Content-Type: application/json" -d @operational.vcpe.pap.json.txt
{
"policies": [
{
"policy-id": "operational.vcpe",
"policy-version": 1
}
]
}
|
16. Start closeloop by `./vcpe.py loop` to trigger packet drop VES event. You may need to run the command twice if the first run fails
[Note you may need to comment out the set_closed_loop in vcpe.py line 165 if
| Jira | ||||||
|---|---|---|---|---|---|---|
|
#vcpecommon.set_closed_loop_policy(policy_template_file)
| Code Block | ||||
|---|---|---|---|---|
| ||||
[2019-06-04 11:03:49,822][INFO ][pool-5-thread-20][org.onap.dcae.common.EventProcessor] - QueueSize:0 EventProcessor Removing element: {"VESversion":"v5","VESuniqueId":"88f3548c-1a93-4f1d-8a2a-001f8d4a2aea","event":{"commonEventHeader":{"startEpochMicrosec":1559646219672586,"sourceId":"d92444f5-1985-4e15-807e-b8de2d96e489","eventId":"Generic_traffic","reportingEntityId":"No UUID available","eventType":"HTTP request rate","priority":"Normal","version":1.2,"reportingEntityName":"zdcpe1cpe01mux01-201906032354","sequence":9,"domain":"measurementsForVfScaling","lastEpochMicrosec":1559646229672586,"eventName":"Measurement_vGMUX","sourceName":"vcpe_vnf_vcpe_vgmux_201906032354"},"measurementsForVfScalingFields":{"cpuUsageArray":[{"percentUsage":0,"cpuIdentifier":"cpu1","cpuIdle":100,"cpuUsageSystem":0,"cpuUsageUser":0}],"measurementInterval":10,"requestRate":492,"vNicUsageArray":[{"transmittedOctetsDelta":0,"receivedTotalPacketsDelta":0,"vNicIdentifier":"eth0","valuesAreSuspect":"true","transmittedTotalPacketsDelta":0,"receivedOctetsDelta":0}],"measurementsForVfScalingVersion":2.1,"additionalMeasurements":[{"name":"ONAP-DCAE","arrayOfFields":[{"name":"Packet-Loss-Rate","value":"22.0"}]}]}}} |
17. Stop cloed loop for testing with ./vcpe.py noloss
Frankfurt vCPE.py Log for creating networks:
View file name vcpe.20200625.log height 250
Typical Errors and Solutions
SDNC DG error
If you run vcpe.py customer and see an error similar to the following:
"requestId": "c772d346-ae34-4e88-b419-0af510c4526a",
"requestScope": "service",
"requestStatus": {
"finishTime": "Wed, 22 Aug 2018 18:46:09 GMT",
"percentProgress": 100,
"requestState": "FAILED",
"statusMessage": "Received error from SDN-C: Not Found"
It is most likely due to an error in vnf-topology-assign DG. This happends in R2 and should have been fixed in R3 (refer
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Enter the SDNC docker
1. make a copy of GENERIC-RESOURCE-API_vnf-topology-operation-assign.xml in the sdnc_controller_container under /opt/sdnc/svclogic/graphs/generic-resource-api
2. edit GENERIC-RESOURCE-API_vnf-topology-operation-assign.xml to replace "<break> </break>" with "<break/>" or "<break></break>"
Enter the SDNC docker
1. make a copy of GENERIC-RESOURCE-API_vnf-topology-operation-assign.xml in the sdnc_controller_container under /opt/sdnc/svclogic/graphs/generic-resource-api
2. edit GENERIC-RESOURCE-API_vnf-topology-operation-assign.xml to replace "<break> </break>" with "<break/>" or "<break></break>"
a. optionally you can change the version to something like 1.3.3-SNAPSHOT-FIX and update graph.versions to match but that is not needed if the xml failed to load .
3. run /opt/sdnc/svclogic/bin/install.sh
this will install the edited DG and make it active as long as the version in the xml and the version in graph.versions match
4. re-run /opt/sdnc/svclogic/bin/showActiveGraphs.sh and you should see the active DG
...
When running "vcpe.py infra" command, if you see error message about subnet can't be found. It may be because your python-openstackclient is not the latest version and don't support "openstack subnet set --name" command option. Upgrade the module with "pip install --upgrade python-openstackclient".subnet can't be found. It may be because your python-openstackclient is not the latest version and don't support "openstack subnet set --name" command option. Upgrade the module with "pip install --upgrade python-openstackclient".
Unable to generate VM name error from SDNC
Received error from SDN-C: Unable to generate VM name: naming-policy-generate-name: input.policy-instance-name is not set and input.policy is ASSIGN.
To resolve this: Check the vgw_VfModuleModelInvariantUuid parameter in the vcpecommon.py script is updated with your ResCust_svc VF_ModuleModelInvariantUuid or not. For every new customer don't forget to update this.
Add this to CheckList:
# CHANGEME: vgw_VfModuleModelInvariantUuid is in rescust service csar, look in service-VcpesvcRescust1118-template.yml for groups vgw module metadata. TODO: read this value automcatically
self.vgw_VfModuleModelInvariantUuid = '26d6a718-17b2-4ba8-8691-c44343b2ecd2'