Versions Compared

Old Version 6

changes.mady.by.user Ray Chang

Saved on

compared with

New Version 7

changes.mady.by.user Ray Chang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • We have created a few sample validation rules but do not know all the rules to be created by the users in the production. That means most likely we can create and provide some high-level dashboards - for the specific rules and details, we could only provide some sample dashboards to give an idea so that how the end users could create their own dashboards customized for their use cases. 
  • For the Network Discovery, what specific audits will be executed and what kinds of audit results are expected 


Dashboard List

(Note) One dashboard type could need multiple dashboard pages depending on the amount of visualizations.


Dashboard TypeDescription (What To Want to See)Required Information To Show (Visualizations)
1Overall Audit MonitorAs a general admin, I want to see the whole platform integrity - health status in terms of all validation rules configured
  • validation total count for the specified time period
  • violation total count for the specified time period
  • validation count over time (trend)
  • violation count over time (trend)
  • violation count by component involved
  • violation count by validation rule
  • violation count by rule severity
  • audit KPI trend: daily violation metric against total validation count (weighted measure considering the severity level)
  • validation list
  • violation list
2Overall Audit Analysis

What kind of validations mostly executed against which models

What kind of violations mostly occur in which components

  • validation stats (e.g., validationId or serviceInstanceId count) by modelInvariantId and modelVersionId
  • which models made how many violations of which type
  • violation stats by validation model, rule, etc.
  • provide the views from the perspective of each component
3Individual Audit AnalysisGiven a validation job, the user wants to see and quickly recognize all relevant violations detected by POMBA
  • validation details and related violation details on the same page
  • (stretch) given the same type of validation job, can we retrieve the historical violation results to give an idea if this is really unusual case or it used to happen?
4Violation Analysis for Network Discovery

For the specific use cases of Network Discovery, the user wants to see the audit stats

  • TBD
5Violation Summary ReportProvide a list of summary list for the validation and violation cases for any potential fixes
  • list of current violations that contain useful up-to-date violations with detail info helping to raise/fix any issues
6Cure History (stretch)For the same validation category (e.g., with the same rule and model id, component set?), the user wants to be ensured that the violation has been fixed and gone now, and wants to get an insight on how much the POMBA helps improve the overall system integrity
  • violation stats over time for the same validation category (e..g, 10 violations reduced to 0 now)
  • KPIs showing the violation resolution trend (e.g., how many violations have been fixed on daily basis)





Supportable Features

  • Provide Where necessary, provide links to move switch the dashboards back and forth across the dashboards: e.g., from the violation page to the page displaying its validation info
  • Color coding for the critical violations


3. Data Generation based on Audit Use Cases

Generally two The user could take some possible approaches to execute the audits and generate audit results:

  • Event-Driven Individual Auditing: e.g., Postpost-orchestration audit trigger triggered by system or user for a single service instance
  • Combined Audit Now: audit the service types and rules selected with a specific service instanceone-click command. This requires to collect and keep a list of service instance info in the entire platform.
  • Continuous Scheduled Auditing: automated auditing of "Combined Audit" for the selected set of services Continuous or Scheduled Auditing: targeting a pre-configured set of service instances (existing and/or new). It could be against all the deployed artifactsFor some services that need a special care or interest, we could customize different schedule (e.g., more frequent validation). 

Configurations 

  • Audit targets selection: which microservices should be included and cross-checked
  • Audit rules selection: which rules should be validated for the target services
  • Scheduling parameters: when, which rules will be applied

...