Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Chameleon -  Subsystem that processes spike events from the real-time graph updates and formats the events, entry point to gallifrey/time machine, and enforces/formats the requests into the format needed by Gallifrey/Time Machine

Gallifrey/Time Machine - Subsystem that makes & retrieves historical assertions

...

Resources > Gizmo > Synapse > Chameleon > Gallifrey/Time Machine > Historical Champ > Historical DB Cluster

I feel Chameleon could be bypassed for this flow, treating chameleon's functionality as just the dmaap history processor, and synapse would call gallifrey/tm directly. This way if chameleon goes down, gallifrey/tm could still service historical GET requests.

Resources > Gizmo > Synapse > Chameleon > Gallifrey/Time Machine > Historical Champ > Historical DB Cluster

...

If the timestamp was sent on a non-singular node call then we would return a message stating that this functionality is not supported.

Gallifrey/Time Machine API Spec:


Type

URI

Query Params

Description

Champ Interaction

GET

relationship/<ID>

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network


Retrieve a relationship by ID


Champ needs to handle accepting a relationship id and the timestamp to run a historical query on the graph db

A subgraph strategy will be used in champ to filter on the relative timestamp provided.

meta=[true/false] if true, payload retrieved will hold the metaproperties at t-k or t-t

GET

relationship/<ID>/lifespan

NOT in scope for Casablanca - decided on call 5/23/2018

None

Retrieve all the timestamps for create, update, delete operations against this relationship

Champ would be called to retrieve the lifespan on the relationship with meta=true and t-k=lifespan to retrieve all the metaproperties on the relationship

GET

entity/<ID>

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network


Retrieve an entity by ID

Champ needs to handle accepting an entity id and the timestamp to run a historical query on the graph db retrieving the asserted state. Default not sending back metaproperties, if metaproperties are needed a parameter would need to be sent to champ.

A subgraph strategy will be used in champ to filter on the relative timestamp provided.

meta=[true/false] if true, payload retrieved will hold the metaproperties at t-k

GET

entity/<ID>/lifespan

NOT in scope for Casablanca - decided on call 5/23/2018

None

Retrieve all the timestamps for create, update, delete operations against this entity


Champ would be called to retrieve the lifespan on the entity with meta=true and t-k=lifespan to retrieve all the metaproperties on the entity

PUT

relationship/<ID>

actor=name of the system making the assertion

changes-only=[true|false] if true, gallifrey/tm will actually determine what has changed between the PUT payload and the most recent set of assertions for the relationship. If false, the entire PUT body will be considered as a new set of assertions whether something has changed or not.

create=[true|false] if true, Gallifrey/TM assumes that this is a create request, if false it assumes it is an update

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network


Asserts that a relationship is to be created or updated (depending on the query parameters that are passed in). This API appends new assertions against the specified relationship.

Generated in Gallifrey/TM

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM (why are we generating this here? if there is a maintenance issue our timings would be out of sync with when these took place in the real-time db)

create = false (changes-only true (execute diff)/false(assume everything changed))

When an assertion is being made without a network timestamp gallifrey/tm will call champ requesting the relationship with it's most current metaproperties. Gallifrey/TM would then adjust the metaproperties (of the updated properties) and would send the payload back to champ with a new current state and an updated previous state's metaproperties. Champ would override it's current metaproperty (for the updated/deleted properties) with the old and current metaproperties sent from Gallifrey/TM. For added properties, they would be added directly with the metaproperties sent from Gallifrey/TM.

When an assertion is being made with a network timestamp gallifrey/tm will call champ requesting the relationship with all of its metaproperties. Gallifrey/TM would then insert the new assertion where appropriate (and adjust neighboring metaproperties) and send the modified payload back to champ for a replace.

create = true

POST - this would be a new create and Gallifrey/TM would pass the metaproperties on each of it's property values and on the relationship itself

PUT

entity/<ID>

actor=name of the system making the assertion

changes-only=[true|false] if true, gallifrey/tm will actually determine what has changed between the PUT payload and the most recent set of assertions for the entity. If false, the entire PUT body will be considered as a new set of assertions whether something has changed or not.

create=[true|false] if true, Gallifrey/TM assumes that this is a create request, if false it assumes it is an update

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network

Asserts that an entity is to be created or updated (depending on the query parameters that are passed in). This API appends new assertions against the specified entity.

Generated in Gallifrey/TM

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM

create = false (changes-only true (execute diff)/false(assume everything changed))

When an assertion is being made without a network timestamp gallifrey/tm will call champ requesting the entity with it's most current metaproperties. Gallifrey/TM would then adjust the metaproperties (of the updated properties) and would send the payload back to champ with a new current state and an updated previous state's metaproperties. Champ would override it's current metaproperty (for the updated/deleted properties) with the old and current metaproperties sent from Gallifrey/TM. For added properties, they would be added directly with the metaproperties sent from Gallifrey/TM.

When an assertion is being made with a network timestamp gallifrey/tm will call champ requesting the entity with all of its metaproperties. Gallifrey/TM would then insert the new assertion where appropriate (and adjust neighboring metaproperties) and send the modified payload back to champ for a replace.

create = true

POST - this would be a new create and Gallifrey/TM would pass the metaproperties on each of it's property values and on the entity itself

DELETE

relationship/<ID>

actor=name of the system making the assertion

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network

Asserts that a relationship has been deleted.

Generated in Gallifrey/TM

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM

Gallifrey/TM would request the latest relationship from champ and would set all of it's properties dbEndTimes to t-k along with the dbEndTime on the relationship itself.

DELETE

entity/<ID>

actor=name of the system making the assertion

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network

Asserts that an entity has been deleted.

Generated in Gallifrey/TM

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM


Gallifrey/TM would request the latest entity from champ and would set all of it's properties dbEndTimes to t-k along with the dbEndTime on the entity itself.

...

Type

URI

Query Params

Description

GET

relationship/<ID>

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network

Retrieve a relationship by ID

GET

entity/<ID>

t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM

t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network

Retrieve an entity by ID

...

Between Chameleon->Gallifrey/TM, the following calls are made:

...

Open question: what happens in the event that db edge rules change etc. (migrations)

When To Record T-K value (Live DB vs Historical DB)

Below is an explanation of the thoughts around when to record the t-k value, examples explaining outcome of storing at live vs historical DB and their pros/cons.

View file
nameHistoricalData - WritingTK Value.pdf
height250

GUI Mocks

New integrated functionality (updates for history)

...