- A design idea
- Why not Camunda Cockpit as is: current Camunda Cockpit was designed from a BPMN process management perspective (note: need to study for TOSCA cases).
- It does not meet service-level orchestration monitoring.
- It is designed for BPMN definition/execution monitoring; require process knowledge for monitoring.
- We need higher-level monitoring abstraction for both BPMN and TOSCA.
- Associate Service Instance Id (or other keys) to the top-level process instance id. For the association,
- Could use a process variable holding the Service Instance id (or other keys), or
- Could use a database holding the association
- Allow VID, UUI or external apps monitor process workflow process (graphically and text-based) based on extensible search keys.
- We need a platform level runtime and history process activity report capabilties out of the box.
- Regardless use of Camunda Enterprise Edition or Community Edition.
- Query to Camunda/ARIA database to extract activities.
- The following diagram depicts the high-level concept.
- Why not Camunda Cockpit as is: current Camunda Cockpit was designed from a BPMN process management perspective (note: need to study for TOSCA cases).
- Rationale
- Search is an Camunda enterprise feature, but we need to provide searching capabilitiy for non-enterprise edition.
- Finding right process instance(s) for a NS/VNF service request is tedious and hassle.
- To facilitate monitoring, we need more than what Camunda Community/Enterprise edition supports.
- provides the process monitoring (instance-search) hyperlink to the SO clients for launching process monitoring.
- Automates tedious manual steps for finding target process instance(s), bypassing tedious search starting from the process defintions.
- Access customized SO Service List / Camunda Cockpit widgets from VID, UUI and external APIs.
- If a service provider uses Camunda Enterprise edition, they can still utilize this SO monitoring on top of Camunda enterprise edition features.
- Many of Camunda enterprise features such as CRUDV and version control of process definitions would be part of SDC.
- ONAP separated workflow design and runtime.
- Several enterprise features are not part of SO monitoring features and are not applicable.
- What are current TOSCA orchestrator monitoring capabilities?
- SO monitoring should cover both imperative and declarative orchestration.
- Question: can we have a kind of uniform way of monitoring?
- Search is an Camunda enterprise feature, but we need to provide searching capabilitiy for non-enterprise edition.
- High-Level Requirements
- Dashboard views of Service lists
- Filtering capabilities based on search criteria
- Configurable search criteria
- Dashboard views of statistics (donuts, pie charts, etc.) for filtered service instances
- Service Instance Rendering and detail panel views
- with sub-service instance drill-drown and drill-up capabilities
- A service instance could be realized by multiple process instances
- with process / task detail
- Topology (workflow) views during/after orchestration
- with sub-service instance drill-drown and drill-up capabilities
- Input/output data views for process/task/service task (messages, parameters)
- Display on service / task detail panel
- Provide message log views (could be on a pop-up widget)
- Color coding/visual indication of statistic and service type and status
- Troubleshooting capabilities by manipulating the workflow during orchestration for troubleshooting and retry from the current location (stretch goal)
- TOSCA orchestration monitoring (stretch goal)
- Dashboard views of Service lists
- Widget Requirements and Design
- Service List Widget
- Capabilities
- Provides monitoring capabilities for processed services based on search criteria
- Configurable Search Criteria filtering: Service ID, Operation Type, Status, User Id, Date/Time range
- Actual filtering criteria fields could be changed based on configuration
- The Service List panel will display the filtered Service list.
- This widget can be triggered from other UIs (VID, UUI, external applications).
- Provides monitoring capabilities for processed services based on search criteria
- Design
- The search criteria fields will be defined in a search criteria field configuration file (e.g., so-monitoring-config.yaml). So, they can be customized and support internationalization.
- Based on the fields and buttons configuration, the Service List widget will render the corresponding fields and buttons.
- The search criteria fields support the field validation based on the field type; e.g., length, date format, time format.
- The wild characters are supported for the String field.
- Example of so-monitoring-config.yaml
- Capabilities
- Service List Widget
service-list-search-criteria-fields:
user-id:
name: User Id
type: String
max-length: 30
validation: alpha-numeric
service-id:
name: Service Id
type: String
max-length: 30
validation: alpha-numeric
operation-type:
name: Operation Type
type: String
max-length: 30
validation: alpha-numeric
status:
name: Status
type: combo-list
max-length: 30
default-value:
- status-item: finished
name: Finished
- status-item: processing
name: Processing
- status-item: error
name: Error
date-from:
name: Date From
type: date
validation: date-format
time-from:
name: Time From
type: time
validation: time-format
date-to:
name: Date To
type: date
validation: date-format
time-to:
name: Time To
type: time
valiation: time-format
filtering-expression:
name: Filtering Expression
type: string
validation: regular-expression
action-buttions:
search:
name: Search
statistic:
name: Statistic
monitor:
name: Monitor
previous:
name: Previous # or <--
next:
name: Next # or -->
service-instance-rendering-details:
max-page-num: 10
- The Service List panel supports pagination, which will be controlled by the Navigation buttons.
- Queries a service list from the SO Request DB through the Monitoring REST APIs.
- Action buttons are enabled/disabled based on data population and/or context.
- Capabilities
- After the user fills up the desired search crtieria fields, and click the Search button.
- Capabilities
- when a user clicks on a row, highlight the selected row and enable the Monitor button.
- Once the Monitor button is clicked, go to the Service Instance Rendering and Deail widget, passing key fields such as service instance id, which will be mapped into the process instance id to find a matched process instance id in Camunda database in the Service Instnce Rendering and Detail widget.
- Design
- Query for service list data for a given search criteria.
- Count the total number per status (Finished, Processing, Error)
- Status Total / total number
- Query for service list data for a given search criteria.
- use Camunda REST API, GET /process-defintion/{id}/xml
- Get the state of a process instance from the activity-instances
- Use Camunda REST API, get/process-instance/{id}/activity-instances
- Render the BPMN XML with bpmn.io and places markers on top of it, and provide Service instance detail views.
- Design
- This widget will be on the secondary/pop-up browser page.
- The Service List page will be the primiary page, so the user can open multiple Service Instance Rendering and Detail pages.
- The number of pages can be limited to the default 10, but it can be configured.
- The search criteria fields are for display only, and they cannot be changed.
- The Detail panel is scrollable and display contents with the tag: value formt.
- Query a process instance with the matched 'serviceInstanceId' process variable.
- get a process instande id: query bases on the process variable,
- GET /history/variable-instance/{variableValue}, where {variableValue} = 'serviceIntanceId' ;
- query BPML XML based on a process instance id
- GET /process-definition/{id}/xml
- query the process instance detail such as process variables
- GET /process-instance/{id}/variables
- Display the process variables on the Service Instance Detail panel
- query and activity instance (tree) for the process instance id
- GET /process-instance/{id}/activity-instances
- The response provides child activity instance info. By using this we can find the parent and child process instnce id and definition id.
- If the current process instance id has its parent, enable the Drill-Up button.
- When a user clicks on a task, display the task varables
- GET /task/{id}variables
- When the task activity type is call activity, it enables the Drill-Down button.
- get a process instande id: query bases on the process variable,
- Render the process instance diagram with the retrieved XML.
use bpmn.io javascript library for the XML rendering and event attachments for process instance tasks.
- This widget will be on the secondary/pop-up browser page.
bpmnViewer.importXML (xmlDiagram, function(err) {
if (err) { ... }
var canvas = bpmnViewer.get('canvas');
canvas.zoom ('fit-viewport');
var eventBus = bpmnViewer.get('eventBus');
// you may hook into any of the following events
var events = [
'element.hover',
'element.out',
'element.click',
'element.dblclick',
'element.mousedown',
'element.mouseup'
];
events.forEach(function(event) {
eventBus.on(event, function(e) {
log(event, 'on', e.element.id);
- The Close button is always enabled.
- Task Drill-Down/Drill-Up and Detail View
- Capabilities
- Support Task (CAll Activity) drill-down/drill-up capabilties.
- When a call activity task is selected, the Drill-Down button will be enabled.
- Provide detail views for the selected Task.
- When a task is selected including the call activity, the Detail view panel will display details for the selected task.
- Design
- When a selected/clicked task activity type is the Call Activity, the Drill-Down button is enabled.
if (activityType.equals('CallActivity') {enableDrillDown();}
- If the current process instance has its parent process instance (from the activity tree), the Drill-Up button is enabled.
- if (parentActivityInstanceId != null) {enableDrillUp();}
- When a selected/clicked task activity type is the Call Activity, the Drill-Down button is enabled.
- Capabilities
- Task Drill-Down/Drill-Up and Detail View
- Sub-Service Instance Rendering and Detail View
- Capabilities
- Get a sub process defintion XML through the Service Id and Process Instance association.
- Use Camunda REST API, get/process-definition/{sub-id}/xml
- Get the state of a sub-process instance from the activity-instances.
- Use Camunda REST API, get/process-instance/{sub-id}/activity-instances
- Render the BPMN XML with bpmn.io and places markers on top of it, and provides service instance detail views.
- This widget uses the same code of the Service Instance Rendering and Detail View with the sub-process instance id.
- The Drill-Up button will be enabled.
- Design
- This widget code is the same as the Service Instance Rendering and Detail.
- The difference is the passed process id; the child process instance id will be passed.
- Capabilities
- Sub-Service Instance Rendering and Detail View
- REST APIs for providing data to UIs
- Capabilities
- Provides REST services, by utilzing 1) Camunda REST APIs, such as BPMN XML string, process activity data, process variable, statistic, and 2) SO Request DB APIs for a service list.
- Consolidate data responses from multpile Camunda calls and feed them to UIs.
- Use of HistoryService APIs, example, processEngine.getHistoryService().createHistoricProcessVariableQuery().xyz
- Set a History level to ACTIVITY as a minimum; AUDIT (default) level for process variable tracing
- Provides workflow tracing (between parent-child workflows, interaction with other services; service task in and out); example,
- processEngine.getRuntimeService().createExecutionQuery().processVariableValueEquals("serviceInstanceId", serviceInstanceId).singleResult();
- Custom Query
- Custom Query against History ACT_HI_DETAIL database table, as needed
- Create a REST API, getServiceList with search criteria
- invoke getInfraRequest(...) to collect service list data based on search criteria.
- Design
- Write REST APIs
- getServiceList(...) invokes new getInfraRequest(...)
- getProcessInstanceId(String serviceInstanceId);
- getprocessDiagram(String processInstanceId); getProcessDiagram(String serviceInstanceId);
- getprocessInstanceDetail(String processInstanceId); getProcessInstanceDetail(String serviceInstanceId);
- getProcessInstanceActivity(String processInstanceId); getProcessInstanceActivity(String serviceInstanceId);
- getProcessTaskDetail(String taskId);
- getProcessTaskType(String taskid);
- Write REST APIs
- Capabilities
- Custom History Event Producer
- Populate additional data in history with extensibility
- Component Build Dependencies
- The following diagram depicts the component build dependencies.
- Components can be built in parallel with a certain level of sequential integration.
- Each estimate includes code development and unit testing plus some component integration and integration testing (since there are API dependencies).
...
- Note:
- It is a rough estimate - to be refined
- Estimate includes development and unit testing time
- As mentioned above section, the components can be built in parallel.
- 5 hours / day
...
- populate serach filter criteria
- Displays a service list panel
- control action buttons
...
- Displays statistic for the selected service scope
...
Service Instance Rendering
and Detail Panel
...
- Renders service instance workflow graphically
- provides the service instance details and selected task details
- Provides drill-down and drill-up capabilities
...
- REST facade for
- collecting and conslidating various Camunda process instance and history data to simplify GUI data collection interaction.
- collecting SO Request DB data
- REST facade for
...
NOTE: Moved it to Casablanca, See SO Monitoring Feature Design for further development/implementation.