...
The Validation Scripts have historically produced a number of reports geared toward human consumption, but no report has existed that provided a full accounting of results in a machine readable format. A file named failures was produced in the output directory, but it contained only a subset of information about the execution run. The following proposal is to create a new file ( report.json ) that contains a full account of all available information. It would be produced on every run of the validation scripts, and would be suitable for consumption for tools such as VNFSDK or Dovetail as part of the certification program.
File Specification
File Header/Top Level
The top level will include a summary of available execution metadata.
...
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
{
"version": "dublin",
"template_directory": "C:\\Users\\tl2972\\Desktop\\stark_template2",
"timestamp": "2019-01-21T02:11:07.305000",
"checksum": "6296aa211870634f9b4a23477c5eab28",
"profile": "",
"outcome": "FAIL",
"tests": [],
"requirements": [],
} |
| Field Name | Required/ Optional/ Conditional | Datatype | Valid Values | Description |
|---|---|---|---|---|
version | Required | string | Version of the validation scripts that produced the report. For now, this is set to the ONAP release name, but it may change to a semantic version number in the future. | |
template_directory | Required | string | Absolute path of the directory containing the Heat templates that were validated | |
timestamp | Required | string | ISO 8601 Timestamp in UTC | |
checksum | Required | string | MD5 hash of all file contents in the template_directory | |
profile | Required | string | Validation profile used to validate the templates. In the default ONAP code base, this is always the empty string | |
outcome | Required | string |
| One of the valid values.
|
tests | Required | list of Test Result | List of Test Result |
Test Result
For each test result a JSON object will be provided that informs the consumer what tests was run, its result, and the requirements it validated.
| Code Block | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
{
"files": [
"/Users/username/Desktop/stark_template2/STARKDB-nested-1.yaml",
"/Users/username/Desktop/stark_template2/base_starkdb.yaml",
],
"test_module": "test_resource_indices",
"test_case": "test_indices_start_at_0_increment",
"result": "FAIL",
"error": " Index values associated with resource ID prefix STARKDB_server_ do not start at 0\n",
"requirements": [
{
"id": "R-11690",
"text": "When a VNF's Heat Orchestration Template's Resource ID contains an\n``{index}``, the ``{index}`` is a numeric value that **MUST** start at\nzero and **MUST** increment by one.\n\nAs stated in R-16447,\n*a VNF's <resource ID> MUST be unique across all Heat\nOrchestration Templates and all HEAT Orchestration Template\nNested YAML files that are used to create the VNF*. While the ``{index}``\nwill start at zero in the VNF, the ``{index}`` may not start at zero\nin a given Heat Orchestration Template or HEAT Orchestration Template\nNested YAML file.",
"keyword": "MUST"
}
]
} |