| View file | ||||
|---|---|---|---|---|
|
Notation:
This has been approved as a best practice for Jakarta.
Meetings
- 2/4/2022 - Recording not available
- Attendees: Toine, Amy Z, Muddasar, Bob H, Byung
- 2/11/2022
- Attendees:
- Recording: video1168149182.mp4, audio1168149182.m4a
- 2/18/2022
- Meeting Canceled
- 2/25/2022
- Recordings: video1165099769.mp4, audio1165099769.m4a
- 3/4/202
- Recordings: video1007532139.mp4, audio1007532139.m4a
- 4/8/2022
- Robert Heinemann Sourabh Sourabh Toine Siebelink Muddasar Ahmed Met to discuss progress on Security Logging Fields POC in CPS. The meeting was not recorded.
- Add ordering to fields. Added Order Column to tables
- 4/27/2022
- Robert Heinemann Sourabh Sourabh Toine Siebelink Ajay Deep Singh Met and discussed log output type and integration with logging collection architecture. The meeting was not recorded. Group agreed that a JSON format would be good to use. This was easier to implement with the logging library being used in CPS. The change was just adding an appender to logback,xml. In addition, the fluentid software used in the logging collection architecture handles the JSON format. Group decided that this was a good approach to continue with the POC. Ajay Deep Singh has committed code regarding the logging architecture POC. A demo of the POC will be given in 1 - 2 weeks from today.
General feedback is that the first iteration of these fields were focused on transactional events. However there are other logging situations that are not transactional such as debug / informational logging. So, some of the tweaks below are to make the fields fit a more general case.
NOTE: The fields below should be outputted in the order indicate if the output format is positional like CSV.
CPS Implementation:
| Jira | ||||||
|---|---|---|---|---|---|---|
|
Key Points
- There are many different types of logging formats that have been proposed and informally adopted across ONAP
- 2017 OpenECOMP Logging Specifications; ONAP Logging Specifications v1,1, v1,2 and v1.3.
- Many different types of logging libraries EELF, PyLog, Log4J, logback, FLog, dropwizard-logging, log4js, logkit, clojure logging, jboss-logging, UnderscoreLog. Probably others. (See Nexus Report Here).
- It is not the place of SECCOM to define a logging format for ONAP Projects to follow.
- We should stay focused on proposing Security focused logging requirements and recommend to TSC to adopt as a Best Practice and subsequently adopting as a Global Requirement.
- When proposing requirements we should not be dictating implementation details BUT we should be cognizant of existing implementations and how our proposed requirements will impact those existing implementations.
- We should strive to reduce impact on existing implementations as much as possible.
Security Log Structure
...
Timestamp
...
Log Type
...
Log Level
...
Transaction ID
...
Status Code
...
Severity
...
Container Data
...
Protocol
...
Service / Program Name
...
Log Message
...
Image Tag / Name
...
Image Digest
...
ID
...
Name
...
Principal ID
...
Role / Attribute ID
NOTE:
...
Example:
From Fabian:
2021-09-10T14:50:37.929Z|d855a2c6-c58f-4d8d-b199-3382d11504d2|http-nio-8083-exec-5|/manage/health|kube-probe/1.19|||DEBUG|500||Headers : X-Content-Type-Options:nos
ISO 8601 TIMESTAMP: 2021-09-10T22:41:40+0000
Log Level: INFO
Transaction ID: 15a28073-3cce-495b-abb4-00771fa011b7
Status Code: COMPLETE
Severity: NONE
Container Image Name:
Container Image Digest:
Container ID:
Container Name:
Principal ID
...
Security Log Field Definitions
Type Synonyms:
REQUIRED: SHALL OR MUST
RECOMMENDED: SHOULD
OPTIONAL: MAY
CON-SEC-LOG-01
CON-LOG-REQ-7
| Order | Field Name | Property Name | Description | EELF Field | Log Spec Field | Reference | CPS Logging Field POC |
|---|---|---|---|---|---|---|---|
| GREEN FIELDS: Best Practice for Jakarta: Existing Fields Recommended | |||||||
| 1 | Timestamp | logTimeStamp | |||||
The container and container application MUST log the field “date/time” in the security audit logs. The value should be represented in UTC and formatted per ISO 8601, such as “2015-06-03T13:21:58+00:00”. The time should be shown with the maximum resolution available to the logging component (e.g., milliseconds, microseconds) by including the appropriate number of decimal digits. For example, when millisecond precision is available, the date-time value would be presented as, as “2015-06-03T13:21:58.340+00:00”. | BeginTimestamp Timestamp | LogTimeStamp |  |
| 2 | Log Type Name | logTypeName | The container and container application |
MAY log the field "Log type" in security audit |
logs |
. This field will adhere to the following ENUM ::= "AUDIT" | "METRICS" | "ERROR" | "DEBUG" | |
"" This is here for legacy purposes. Older projects used to generate 4 separate log files. However since projects now will output all logs to STDOUT and STDERR this field is here to give projects adhering to the old standard a way to specify those log file types. NOTE: This field is optional but a placeholder is still required to be outputted. That is why the "" is included in the ENUM. | N/A | p_marker | (4) | |
| 3 | Log Level | logLevel |
The container and container application MUST use an appropriately configured logging level that can be changed dynamically. The intention of this field is to not cause performance degradation via excessive logging. This field will adhere to the following ENUM ::= "FATAL" | "ERROR" | "WARN" | "INFO" | "DEBUG" | "TRACE" The verbosity of the logging increases from left to right. |
| Category Log Level | Level | (4) |
Transaction ID
| |||
| 4 | Trace ID | traceId | The container and container application MUST log |
Trace ID A |
trace ID is a universally unique value that identifies a single transaction request or a series of related log events within the ONAP platform. Its value is conformant to RFC4122 UUID. This value is readily and easily obtained in most programming environments. | Request ID | Transaction ID | (4) |
CON-LOG-REQ-10
|
| 5 |
| Status Code | statusCode | The container and container application MUST log a "status code" in the security audit logs. This field indicates the high level status for transactional, APIs calls, or sub operational events. This field will adhere to the following ENUM ::= " |
SUCCESS" | "INPROGRESS" |"FAIL_WARN" | "FAIL_ERROR" | " |
FAIL_FATAL" |
|
|
From an end user perspective these categories should not be considered strict due to the absence of contextual information of holistic operations. There may be some circumstances where FAIL_WARN may be more serious than FAIL_ERROR. Regardless, from a developer view, FAIL_WARN, FAIL_ERROR, and FAIL_FATAL should be viewed as increasing importance and understand that the end user will need to provide additional context for their comprehension and execute and potential action from the particular failure. | Status Code | Status Code | (4) |
| ||
| 6 | Principal ID | principalId |
The container and container application MUST log the |
Principal identity of a |
requestor in the security audit logs. This field should contain the identification of the entity (user agent, client id, user, user id, login ID, non-person entity (NPE), Token, etc.) that made the request of the service or API indicated in the Service/Program Name field. For a serving API that is authenticating the request, this should be the authenticated username or equivalent. NOTE: The CPS project uses a framework that provides this field. | N/A | User | R-89474 | | |||
| 7 | Service / Program Name | serviceName | The container and container application MUST log the field “service or program used for access” in the security audit logs. This intention is to capture the service name endpoint or an externally advertised API invoked, e.g., where are you connecting to. This is represented as a URI or URL. NOTE: The CPS project uses a framework that provides this field. | ServiceName | ServiceName | (4) | |
| 8 | Log Message | message | The free text payload of a log event. | detailMessage | p_message | |
| Order | Field Name | Description | EELF Field | Log Spec Field | Reference |
|---|---|---|---|---|---|
| YELLOW FIELDS: This fields are not part of the best practice. They are a work in progress and are intended to be added to generated logs via a logging architecture. | |||||
This is to be used for error reporting in internal processing in conjunction with the status code field.
This field will adhere to the following ENUM ::= "NONE" | "WARN" | "ERROR" | "FATAL"
| Container Image Name / Tag | The container and container application MUST log the Container Image Name/Tag. The image name/tag is as returned by the docker images command. NOTE: Images are not required to have tags | ||||
| N/A | N/A | ||||
Container Image Digest | The container and container application MUST log the container image digest. The digest is a cryptographic digest as returned by the docker images --digests command. | N/A | N/A | T1036, T1525 |
| Container ID | The container and container application MUST log the container ID. The container ID is the same that is returned by the docker ps -q command. NOTE: The container ID is unique for life time of the the container instance. Once the container is killed, this ID goes away. |
The container and container application MUST log the container name.
This is the unique name of the image ( webserver, FW, DCAE01). This is returned by the docker ps command.
The container and container application MUST log the Principal identity of a requestor in the security audit logs.
This field should contain the identification name of the client application (user agent, client id, user, user id, login ID, non-person entity (NPE), Token, etc.) of the entity accessing or invoking the service or API (Service / Program Name).
This field should contain the identification of the entity (user agent, client id, user, user id, login ID, non-person entity (NPE), Token, etc.) that made the request of the service or API indicated in the Service/Program Name field. For a serving API that is authenticating the request, this should be the authenticated username or equivalent.
There are not a concrete set of values for this field. The developer should keep the following set of guidelines when determining what value to use or set for this field.
- Use the short name of your component, e.g. xyzdriver.
- Values should be human-readable.
- Values should be fine-grained enough to disambiguate subcomponents where it's likely to matter. This is subjective.
- Be consistent: your component should ALWAYS report same value.
REF: See PartnerName in v1.3 and (4).
| N/A | N/A | ||||
| THE BELOW FIELDS ARE OUT OF SCOPE FOR THE YELLOW FIELDS POC | |||||
Role / Attribute ID | The container and container application MUST log the Role or Attribute ID of the Principal identity of the entity accessing the requested service or API. Note: The group ID is in reference to a Role or Attribute as part of a RBAC or ABAC scheme. 3/4/2022: Team decided this field is out of scope for yellow fields for now. Potentially revisit for a later phase.
| N/A |
CON-LOG-REQ-8
| N/A | |||||
| Protocol | The container and container application MUST log the field “protocol” in the security audit logs. This refers to the communication mechanism for a request. The value of this field should be representative of the OSI application layer protocol. This is represented as a decimal formatted TCP/IP port number. | N/A | N/A |
CON-LOG-REQ-9
The container and container application MUST log the field “service or program used for access” in the security audit logs.
This intention is to capture the service name endpoint or an externally advertised API invoked, e.g., where are you connecting to. This is represented as a URI or URL.
(4)
Green Fields: Fields or data that is within the scope of a single ONAP sub-project
Yellow Fields:
Fields or data that is broader in scope than a single project. Maybe contain data that is only available at the container or orchestration level. Also may contain data that involves multiple ONAP sub-projects.