References
CPS-2146 - Getting issue details... STATUS
Assumptions
| # | Assumption | Notes |
|---|---|---|
| 1 | Proposed solution is not a quick fix, but allows for future scaling of NCMP. |
Issues & Decisions
| # | Issue | Notes | Decision |
|---|---|---|---|
| 1 | This is an open issue | ||
| 2 | Do we need a analysis template? | is convention for (new) developers to guide them | Luke Gleeson and Toine Siebelink agreed we do to have consistent study pages |
| 3 | This is a very important (blocking issue) |
<Note. use green for closed issues, yellow for important ones if needed>
Background
CPS and NCMP have much higher memory consumption than required. Regarding NCMP specifically, it has some in-memory data structures that grow linearly with the number of CM-handles. Regarding CPS-core, there is a more fundamental problem in that CPS path queries could return any amount data, it will be unknown to the application until a query is executed (even if pagination on DataNodes were used, since CPS DataNodes represent tree-structures, it would not be known how deep the tree is). This study and implementation proposal will target simple and concrete steps to reduce memory consumption.
Analysis
A number of issues leading to high memory usage have been identified.
The use of Hazelcast (an In-Memory Data Grid) has been identified as a particular source of high memory usage.
The following is an overview of Hazelcast structures in CPS and NCMP.
| Component | Hazelcast Structure | Type | Purpose | Recommendation | Implementation Proposal | Notes |
|---|---|---|---|---|---|---|
| CPS | anchorDataCache | Map<String, AnchorDataCacheEntry> | Needs further analysis | |||
| NCMP | moduleSyncWorkQueue | BlockingQueue<DataNode> | Remove immediately | TBC | Entire CM handles are stored in work queue for module sync. This creates very high memory usage during CM handle registration. | |
| NCMP | moduleSyncStartedOnCmHandles | Map<String, Object> | Remove immediately | TBC | ||
| NCMP | dataSyncSemaphores | Map<String, Boolean> | Remove later | Low priority - this map is only populated if data sync is enabled for a CM handle. | ||
| NCMP | trustLevelPerCmHandle | Map<String, TrustLevel> | Remove immediately | TBC | One entry is stored in memory per CM handle. This is directly implicated in logs supplied in investigation of out-of-memory errors in CPS-2146 | |
| NCMP | trustLevelPerDmiPlugin | Map<String, TrustLevel> | See notes | Low priority - there are only small number of DMIs, so this structure will not grow so large. However, if trustLevelPerCmHandle is being removed, this structure may be removed as part of the same solution. | ||
| NCMP | cmNotificationSubscriptionCache | Map<String, Map<String, DmiCmNotificationSubscriptionDetails>> | Remove later | This is low priority as the feature is not yet implemented, thus it is not in use. It is unclear how much data will be stored in the structure. It is presumed to be low, as this structure will only hold pending subscriptions. |