Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The goal of this page is to provide an E2E infrastructure for testing an hourly or triggered master/tagged build for the purposes of declaring it ready in terms of health check and use case functionality.  CD functionality includes providing real-time and historical analytics of build health via stored/indexed logs from the deployment jobs in our ELK stack that sits outside of ONAP.

ServerURLNotes

Live Amsterdam server


http://amsterdam.onap.info:8880

Login to Rancher/Kubernetes only in the last 45 min of the hour

Use the system only in the last 10 min of the hour

Jenkins serverhttp://jenkins.onap.info/job/oom-cd/view deployment status, deployment (pod up status)

Kibana server

http://kibana.onap.info:5601query "message" logs or view the dashboard

Kibana Dashboard of CD system diagnosing health check issues in an Hourly ONAP OOM Deploy

In the combined ELK and Kibana CD system below we can see that SDC is failing healthcheck on average about 35% of the time - this may be due to a gap between healthcheck using a 200 HTTP return, the SDC rest call timing out when Spring is still coming up on the servlet container or a dependency check in SDC itself on another component where a particular startup order or timing of calls exposes an issue - anyway the ELK system that consumes logs from the hourly build can identify issues like this or the 1 hour healthcheck failure in MSB below that for 14 components that was transient.

Jira
serverONAP JIRA
serverId425b2b0a-557c-3c0c-b515-579789cceedb
keySDC-739

Shane Daniel has created a dashboard that can be used to diagnose the health of the current hourly build based on logs generated by the health check running in robot off an hourly deploy of ONAP OOM (CI triggers are pending)

...