...
1. Upgrade of ELK & Potential Feature Development (AAI search-data-service)
ELK Upgrade
Bath team (in charge of search-data-service, @Colin Burns) is planning to upgrade elasticsearch to 6.1.2 (based on AT&T approved versions).
- Current ELK versions: elasticsearch 2.4, kibana 4.6 (no logstash is being used)
- To better create the dashboards with enhanced Kibana features and look, upgrading to version 5.6 for all ELK stack is desired. (note: Logging project is using 5.5)
- Upgrade from 2.x to 5.x needs requires "Full Cluster-restart Upgrade".
- search-data-service should reflect this upgrade
- deploy/configure the right versions
- potentially update relevant API methods for the elasticsearch data management.
Specifically for POMBA use, Groundhog could provide:
- automatic deployment of kibana (version 6.1.2) through oom (currently, it is manually installed), configure/install all POMBA dashboards
- if necessary for any audit results parsing, automatic deployment of logstash (version 6.1.2) through oom
Feature Enhancements
- Any change of the validation/violation data being pushed to elasticsearch?
- violationDetails (which tells what is really differentwould tell the exact discrepancies; see the sample event below in the '?violations') need to be parsed (using logstash) or sent/stored by searchdata-service (see the sample event below)? Kibana cannot use such nested info in the router or parsed (using logstash)? Such nested info cannot be used in the kibana visualizations.
- Any other meta-data that would be useful? e.g., who invoked the validation (user, dept)
- "(audit) Time elapsed from orchestration timeElapsed time after orchestration" would be useful? to get an idea when the instance content would drift from the intended info if it can happen.
(Note) Below are the sample validation and violation events currently stored in ES that will be the data source for the Kibana dashboards.
...