- LOG-395Getting issue details... STATUS
1. Upgrade of ELK & Potential Feature Development (AAI search-data-service)
ELK Upgrade
- Current ELK versions: elasticsearch 2.4, kibana 4.6 (no logstash is being used)
- To better create the dashboards with enhanced features and look, upgrading to version 5.6 is desired. (note: Logging project is using 5.5)
- Upgrade from 2.x to 5.x needs "Full Cluster-restart Upgrade".
Feature Enhancements
- Any change of the validation/violation data being pushed to elasticsearch?
- violationDetails (what is really different) need to be parsed (using logstash) or sent/stored by search-service? Kibana cannot use the nested info in the visualizations.
- Any other meta-data useful? e.g., who (user, dept) invoked the validation
(Note) Below are the current validation and violation events stored in ES
2. Dashboard Ideas
The visualizations and dashboards will need to be designed and created according to the current and any potential use cases of the POMBA services - what the users want/need to check, how the system could help improve the whole platform integrity. We want the POMBA reporting to be informative, insightful, and intuitive from the user's perspective.
Challenges
- We have created a few sample validation rules but do not know all the rules to be created by the users in the production. That means most likely we can create and provide some high-level dashboards - for the specific rules and details, we could only provide some sample dashboards to give an idea so that how the end users could create their own dashboards customized for their use cases.
- For the Network Discovery, what specific audits will be executed and what kinds of audit results are expected
Dashboard List
(Note) One dashboard type could need multiple dashboard pages depending on the amount of visualizations.
Dashboard Type | Description (What To Want to See) | Required Information To Show (Visualizations) | |
---|---|---|---|
1 | Overall Audit Monitor | As a general admin, want to see the whole platform integrity |
|
2 | Individual Audit Analysis | Given a validation job, the user wants to see all related violations found by POMBA |
|
3 | |||
4 | Violation Analysis | what kind of violations mostly occur in which components |
|
5 | Violation Analysis for Network Discovery |
Supportable Features
- Provide links to move back and forth across the dashboards: e.g., from the violation page to the page displaying its validation info
3. Data Generation based on Audit Use Cases
Generally two approaches to execute the audits and generate audit results:
- Event-Driven Auditing: e.g., Post-orchestration audit trigger by system or user with a specific service instance
- Continuous or Scheduled Auditing: targeting a pre-configured set of service instances (existing and/or new). It could be against all the deployed artifacts.
Configurations
- Audit targets selection: which microservices should be included and cross-checked
- Audit rules selection: which rules should be validated for the target services
- Scheduling parameters: when, which rules will be applied