Test Steps & Test Data

  1. Create Dataspace: curl -u cpsuser:cpsr0cks! -X POST "http://localhost:8883/cps/api/v1/dataspaces?dataspace-name=openroadm" -H "accept: text/plain" -d ''
  2. Create Schemaset:  curl -u cpsuser:cpsr0cks! -X POST "http://localhost:8883/cps/api/v1/dataspaces/openroadm/schema-sets?schema-set-name=owb-msa221-schema" -H "accept: text/plain" -H "Content-Type: multipart/form-data" -F 'file=@owb-msa221.zip;type=application/zip'   Schemaset: owb-msa221.zip
  3. Create Anchor: curl -u cpsuser:cpsr0cks! -X POST "http://localhost:8883/cps/api/v1/dataspaces/openroadm/anchors?schema-set-name=owb-msa221-schema&anchor-name=owb-msa221-anchor" -H "accept: text/plain" -d '' 
  4. Post the initial Node: 
    curl -g -H "Authorization: Basic Y3BzdXNlcjpjcHNyMGNrcyE=" -H "Content-Type: application/json" --request POST 'http://localhost:8883/cps/api/v1/dataspaces/openroadm/anchors/owb-msa221-anchor/nodes' -d @owb-msa22-first-node-payload.json -i  Test data: owb-msa22-first-node-payload.json
  5. Use Test client to post multiple data concurrently. Test data:  msa221-data.zip

Write Test Cases

Test Iteration 1 - 4 GB RAM, 2 vCPU - Kubernetes Cluster

Test Environment
CPUs2 vCPU
Memory4 Gi (~4.2 GB)
Data Size269 KB/DataNode
VM/K8s cluster

K8s cluster

application (CPS) instance - 1

postgres instance - 2

Test Results

#

#Nodes

ThreadTime taken to post single nodeTime taken to post all data nodesComments
15010.8 sec42.6 sec
250010.9 sec9 m 36 sec
Connections drops for a while and resumes

Reason For Connection drops: Application restarts 

due Out OF Memory Error 

350010~ 1.2 sec8 m 42 secNo improvements with increased threads count

Test Iteration 2 - 8 GB RAM, 4 vCPU - Kubernetes Cluster

Test Environment
CPUs4 vCPU
Memory8 Gi (~48.5 GB)
Data Size269 KB/DataNode
VM/K8s cluster

K8s cluster

application (CPS) instance - 1

postgres instance - 2

Test Results

#

#Nodes

ThreadTime taken to post single nodeTime taken to post all data nodesComments
150010.8 - 0.9 sec7m 53 sec

Not able to post more than 500 nodes which lead to

application pod restart with the the reason code 137 (OOMKilled)

Test Iteration 3 - 17 GB RAM, 8 vCPU - Kubernetes Cluster

Test Environment
CPUs8 vCPU
Memory16 Gi (~17 GB)
Data Size269 KB/DataNode
VM/K8s cluster

K8s cluster

application (CPS) instance - 1

postgres instance - 2

Test Results

#

#Nodes

ThreadTime taken to post single nodeTime taken to post all data nodesComments
15001~0.9 sec7m 44 sec

Not able to post more than 500 nodes which lead to

application pod restart with the the reason code 137 (OOMKilled)

Test Iteration 4 - 16 GB RAM, 12 vCPU - Kubernetes Cluster

Test Environment
CPUs12 vCPU
Memory16 Gi (~17 GB)
Data Size269 KB/DataNode
VM/K8s cluster

K8s cluster

application (CPS) instance - 1

postgres instance - 2

Test Results

#

#Nodes

ThreadTime taken to post single nodeTime taken to post all data nodesComments
150010.8 - 0.9 sec7m 23 sec

Not able to post more than 500 nodes which lead to

application pod restart with the the reason code 137 (OOMKilled)

Test Iteration 5 - 128 GB RAM, 64 vCPU, Single VM - CPS temporal notifications enabled

Test Environment
CPUs64 vCPU
Memory128 GB
Data Size269 KB/DataNode
VM/K8s cluster

Single VM

Test Results

#

#Nodes

ThreadTime taken to post single nodeTime taken to post all data nodesComments
15001~ 2 sec16 m 6 sec
210001~ 2 sec
  1. Very slow after 752 POST transactions. 25 min 29 sec for 752 transactions. Got OOM error (Heap Size 40 GB)
  2.  Introduced 2s sleep time between the post calls. 759 txns completed in 25 min. Then got stuck due to OOM error (Heap Size 40 GB)
  3. Increased heap size to 60 GB
    Not much improvement. Got stuck at 762 nodes due to OOM error
  4. Increased heap size to 100 GB
    - Posted 1000 nodes successfully in 53 min 24 sec)


31000602 - 6 sec1h 19 min 2 secNo improvements with increased threads count
420001
1h 34 min for 1131 nodes. Got stuck due to postgres connection leak

Test Iteration 6 - 128 GB RAM, 64 vCPU, Single VM - CPS temporal notifications disabled

Test Environment
CPUs64 vCPU
Memory128 GB
Data Size269 KB/DataNode
VM/K8s cluster

Single VM

Test Results

#

#Nodes

ThreadTime taken to post single nodeTime taken to post all data nodesComments
1100011.8 - 2 sec30 m 15 sec
210002 1.8 - 2 sec29 m 49 secNo improvements in performance with increased threads count

310005 1.8 - 2 sec30m 44 sec
4100010 1.8 - 2 sec30m 21 sec
520001 1.8 - 2 sec59 m 26 sec
630001 1.8 - 2 sec1h 30 min 29 sec

Read Test Cases

See Reading Large Data Instrumentation Comparisons

Test Results (After xpath query performance improvements)

No of nodesthreads countTime to read single txnTime to read allComments
5001~0.5 sec4 min 10 secDrastic improvement in performance than before. It took 2 hours to retrieve the data before the fix.
5002~0.5 sec4 min 12 secNo improvements in performance with increased threads count
  • No labels