oy
The intent of the 72 hour stability test is not to exhaustively test all functions but to run a steady load against the system and look for issues like memory leaks that aren't found in the short duration install and functional testing during the development cycle.
...
| Test # | Comment | Message |
|---|
| k8 utilization Wed May 20 18:45:15 UTC 2020 | Memory:
root@long-nfs:~/oom/kubernetes/robot# kubectl -n onap top pods | sort -rn -k 3 | head -25
dev-appc-0 7m 2901Mi
dev-portal-cassandra-59f5cb4cf5-9phmg 159m 2777Mi
dev-appc-2 10m 2705Mi
dev-appc-1 19m 2681Mi
dev-cassandra-0 73m 2417Mi
dev-cassandra-2 48m 2394Mi
dev-cassandra-1 70m 2391Mi
dev-sdnc-2 71m 1868Mi
dev-policy-59f48bd84b-q2fp8 7m 1820Mi
dev-sdnc-0 139m 1627Mi
dev-sdnc-1 26m 1574Mi
dev-vid-5b7558dcdc-rx2d7 9m 1510Mi
dev-clamp-dash-es-6cb85979b5-cvrcs 32m 1480Mi
dev-awx-0 244m 1434Mi
dev-aai-elasticsearch-55b56f855c-f5pp5 2m 1422Mi
dev-sdc-be-77d55774f5-zkfrt 6m 1381Mi
dev-dcae-cloudify-manager-6f854859f9-ctdcv 90m 1312Mi
dep-dcae-tca-analytics-55dbd5cd9d-fsm89 511m 1262Mi
dev-aaf-cass-7d55bfc874-sqcdq 6m 1244Mi
dev-aai-traversal-847c4c6994-qbpst 3m 956Mi
dev-so-bpmn-infra-7b58b75b76-n59sf 5m 953Mi
dev-message-router-zookeeper-2 2m 946Mi
dev-aai-resources-74dd6994d4-nh24m 5m 869Mi
dev-aai-graphadmin-65db8cfc67-svvkd 2m 836Mi
dev-music-cassandra-2 147m 801Mi |
| #37 | vG and vFW failed on VID screen errors looking for data items. Investigation shows that aai-traversal pod restarted. Looks like slow networking caused the pod to be redeployed but not conclusive. Initially so, vid failed healtch check until aai traversal was up then both passed healthcheck. |
|
| Thu May 21 12:33:45 UTC 2020 | Memory: root@long-nfs:/home/ubuntu# kubectl -n onap top pod | sort -rn -k3 | head -20
dev-appc-0 7m 2834Mi
dev-portal-cassandra-59f5cb4cf5-9phmg 152m 2780Mi
dev-appc-1 19m 2700Mi
dev-appc-2 10m 2694Mi
dev-cassandra-2 15m 2449Mi
dev-cassandra-1 21m 2434Mi
dev-vid-5b7558dcdc-rx2d7 16m 1786Mi
dev-sdnc-2 64m 1664Mi
dev-sdnc-0 131m 1631Mi
dev-sdc-be-77d55774f5-zkfrt 9m 1578Mi
dev-sdnc-1 29m 1566Mi
dev-awx-0 291m 1524Mi
dev-clamp-dash-es-6cb85979b5-cvrcs 37m 1496Mi
dep-dcae-tca-analytics-55dbd5cd9d-fsm89 664m 1318Mi
dev-dcae-cloudify-manager-6f854859f9-ctdcv 76m 1302Mi
dev-aaf-cass-7d55bfc874-sqcdq 5m 1250Mi
dev-cds-blueprints-processor-7fd988d584-mvdkz 40m 1228Mi
dev-message-router-zookeeper-1 5m 1127Mi
dev-message-router-zookeeper-0 6m 1023Mi
dev-so-bpmn-infra-7b58b75b76-n59sf 8m 941Mi
|
...
