Summary
The Performance test was executed by making request against the Policy RESTful APIs residing on the XACML PDP installed in the windriver lab (PFPP) to create, push and get policy decisions. This was running on a kubernetes host, using the m2.xlarge spec, having the following configuration:
- 16GB RAM
- 8 VCPU
- 160GB Disk
The performance test runs 10 simultaneous threads calling Policy Engine RESTful APIs to create and push policies to the PDP-X for Guard, Guard Min Max, and Config Base policy types, each thread repeats this process 10 times. Once the first thread group is complete, the performance test runs 10 simultaneous threads calling Policy Engine RESTful APIs to get decisions for Guard and Guard Min/Max policies on the PDP-X, each thread repeats this process 1000 times.
The test execution resulted in 20000 getDecision PERMIT requests resulting in 99.87% returning the expected response and 20000 getDecision DENY requests resulting in 99.79% returning the expected response with throughput averaging about 2.3 transactions per second. CPU and memory usage are provided below with JMeter results.
Result
CPU Utilization
Total CPU used by the PDP was measured before and after the test, using "ps -l".
| Initial CPU time | Final CPU time | Total CPU used during test | Average CPU per request | Initial CPU% | Initial Memory% | Final CPU% | Final Memory% |
|---|---|---|---|---|---|---|---|
| 00:02:33 | 01:56:08 | 6815 ms | 9.8 ms | 9.4% | 4.4% | 92.3% | 8.9% |
Memory Utilization
Number of young garbage collections used during the test: 1506 Avg. Young garbage collection time: ~5.8 ms per collection Total number of Full garbage collection: 5 Avg. Full garbage collection time: ~112 ms per collection S0C S1C S0U S1U EC EU OC OU MC MU CCSC CCSU YGC YGCT FGC FGCT GCT 52736.0 49664.0 17713.7 0.0 246784.0 72845.7 699392.0 61080.8 78336.0 76078.3 9472.0 9004.4 24 1.559 3 0.584 2.143 27648.0 27648.0 24017.9 0.0 293888.0 62859.4 699392.0 168780.0 82944.0 80562.5 9472.0 9027.6 1530 53.013 5 1.076 54.088
Performance Metrics
No. | Metric | Description | Result | Comments |
|---|---|---|---|---|
| 4 | Maximum Simultaneous Executions | Measure the maximum number of simultaneous policy executions that can be achieved whilst maintaining system stability and resource utilization | 10 | DMaaP connection limitations prevented the test from running more than 10 simultaneous threads/ONSETs |
| 5 | Multi-Threaded Response Time | Measure the execution time for onset and abatement in each use case when multiple threads are injecting ONSET events simultaneously | vCPE - 18 ms vFirewall - 55 ms vDNS - 9 ms VOLTE - 9 ms | |
| 6 | Multi Threaded CPU Usage | CPU Usage for each use case when multiple threads are injecting ONSET events simultaneously | 9.8 ms | |
| 7 | Multi Threaded Memory Usage | Memory Usage for each use case when multiple threads are injecting ONSET events simultaneously |