...
Solution 5: True Participant Replicas
The requirements are:
- Participants can be replicated, each participant can have an arbitrary number of replicas
- Composition definitions, instances, element instances and all their data including properties is identical in all participant replicas
- When anything is changed in one replica, the change is propagated to all the replicas of a participant
- An operation on a composition element can be sent to any replica of a participant, which means that for a given element, the deploy could be on replica 1, the update could be on replica 2 and the delete could be on replica 3, as one would expect in any HA solution
- A single REST operation called on ACM-R will select a participant replica (probably using round robin initially but we could support other algorithms in the future), and use that replica for that operation.
- The ACM runtime will be made HA (more than one replica of ACM-R will be supported), it will run on a HA postgres.
- The implementation of change propagation is transparent to participant API users
- Replicas are "eventually consistent", with consistency typically occurring in 100s of milliseconds
This solution uses a similar approach to that used by Kubernetes when using etcd to implement CRDs and CRs. It implements replication in participants by introducing
- The concept of participant replicas into ACM-R and the Participant Intermediary
- A lightweight mechanism for replica update between replicas and ACM-R. Every time a replica changes its data, the change is reported to ACM-R and ACM-R updates all other replicas for that participant
In this solution:
- Participant Design backward compatibility is preserved, there is no change to the participant intermediary interface for participant implementations
- Participant Configuration Backward compatibility is preserved, apart from a new "replicas" parameter (optional, default is 1), there is no change to participant configuration
- ACM-R introduces a new REST API for replica management
- ACM-R is made HA so that it itself can scale
Optimal Solution:
After analysis, it is clear that the best solution to use is number 3.
...