CPS-251 - Getting issue details... STATUS
Introduction
Assumptions
- Temporal database stores full information in the payload ( CPS-192: Design data store for Temporal Service).
- It is possible that data stored for the different timestamps is the same, especially if the payload is used to filter data.
- Data in the response body will always contain the entire payload data.
Query APIs
Filtering Data
In CPS temporal, the data can be filtered based on three criteria
Datetime
- Data in last X hours ( Not decided )
- Data after a particular DateTime
- parameter name - "after"
- type - "DateTime"
- Format - 'YYYY-mm-ddTHH:MM:SS.SS-TZ'
- Example
- Data before a particular DateTime
- parameter name - "before"
- type - "DateTime"
- Format - 'YYYY-mm-ddTHH:MM:SS.SS-TZ'
- Example
- Last X network data ( Not decided )
- DataTypes in CPS Core
Dataspace & schema-set
- Dataspace & anchor
- Dataspace & multiple anchors - To improve performance, if there is a need to fetch data for multiple anchors.
- Dataspace
- Payload
- Based on a subsection or field in the payload. These criteria do need schema-set to be fixed to search in the same set of anchors
Adding payload filtering with DateTime criteria can make the query APIs complicated. Hence, we will implement the basic APIs first and will add complex ones when required.
Others Parameters
| No | Name | Type | Default | Purpose | Example |
|---|---|---|---|---|---|
| 1 | index | int | The query output can have many rows so it is important to limit the fetched data. To limit the number of records and to provide pagination, these two parameters can be used. "index" parameter represents the page number, starts with 0, and maxSize is the page size. | ||
| 2 | maxSize | int | 1000 | ||
| 3 | point-in-time | DateTime | CurrentDateTime | Pagination does not work well if new data gets added which fulfils the search criteria. The user must provide this value to avoid this issue. |
Proposed APIs - Approach 1 - GET
In the GET APIs, all the filtering parameters are passed as query parameter.
1.1 Get anchor data which is added after
| No | API endpoint | Description | Example |
|---|---|---|---|
| 1. | GET /dataspaces/{dataspace-name}/anchors/{anchor-name}?after=<epoch-time>&maxSize=1000&?payload={"abcd" : "abcd"} | Return all the data entries for an anchor after the specified epoch in nanoseconds | |
| 2. | GET /dataspaces/{dataspace-name}/schema-sets/{schema-set}?after=<epochtime> | Return all the data entries based on provided schema-set after the specified epoch in nanoseconds. | |
Response Body
It contains three field
| name | type | |
|---|---|---|
| nextRecordsLink | string | added only if there are remaining records to be fetched for the query. |
| previousRecordsLink | string | added only if it is not the first set of records. |
| records | list | contains one record for each timestamp that meets filtering criteria. It contains header information along with data. |
{
"nextRecordsLink": "cps-temporal/api/v1/dataspaces/{dataspace-name}/anchors/{anchor-name}?after=<epoch-time>&maxSize=1000&before=<epoch-time>&index=2&point-in-time=DATE",
"previousRecordsLink": "cps-temporal/api/v1/dataspaces/{dataspace-name}/anchors/{anchor-name}?after=<epoch-time>&maxSize=1000&before=<epoch-time>&index=0&point-in-time=DATE",
"records": [
{
"timestamp": "1234567788889",
"dataspace": "my-dataspace",
"schemaSet": "my-schema-set",
"anchor": "my-anchor",
"data": {
"status" : "UP"
}
}
]
}
Open Items
- Datetime format - YYYY-MM-DDTHH:MM:SS.microseconds-timezone