...

1. First we have to generate certificates in ~/tools/ssl folder by using gen_certs. This step only needs to be performed during the first test setup (or if the generated files have been deleted).

   Code Block
   title Generating certificates
   ./cloud-based-performance-test.sh gen_certs

   
   

2. Then we call setup in order to send certificates to HV-VES, and deploy Consumers, Prometheus, Grafana and create their ConfigMaps.

   Code Block
   title Setting up the test environment
   ./cloud-based-performance-test.sh setup

   
   

3. After that we have to change HV-VES configuration in Consul KEY/VALUE tab (typically we can access Consul at port 30270 of any Controller node, i.e. http://slave1:30270/ui/#/dc1/kv/dcae-hv-ves-collector/edit).

   Code Block
   title HV-VES Consul configuration collapse true
   {"security.sslDisable": false, "logLevel": "INFO", "server.listenPort": 6061, "server.idleTimeoutSec": 300, "cbs.requestIntervalSec": 5, "streams_publishes": { "perf3gpp": { "type": "kafka", "aaf_credentials": { "username": "admin", "password": "admin_secret" }, "kafka_info": { "bootstrap_servers": "message-router-kafka:9092", "topic_name": "HV_VES_PERF3GPP" } } }, "security.keys.trustStoreFile": "/etc/ves-hv/ssl/custom/trust.p12", "security.keys.keyStoreFile": "/etc/ves-hv/ssl/custom/server.p12", "security.keys.trustStorePasswordFile":"/etc/ves-hv/ssl/custom/trust.pass", "security.keys.keyStorePasswordFile": "/etc/ves-hv/ssl/custom/server.pass"}

   
   

   Info
   title Hint
   How to access Consul UI? Consul's address: http://<worker external IP>:<Consul External Port> To check Consul External Port, execute: ubuntu@onap-5422-rke-node:~$ kubectl -n onap get svc | grep consul consul-server-ui                   NodePort       10.43.132.178   <none>                                 8500:31190/TCP                                           6d20h ---------------------------------------------------------------------------------------------------------------------------------------------- If service "consul-server-ui" is not exposed to external port (NodePort) and is configured as Cluster IP, please follow steps below. ubuntu@onap-5422-rke-node:~$ kubectl -n onap get svc | grep consul consul-server-ui                                       ClusterIP      10.43.132.178   <none>                                 8500/TCP                                                 25h ubuntu@onap-5422-rke-node:~$ kubectl -n onap edit svc consul-server-ui ---- ... apiVersion: v1 kind: Service metadata: ... spec: ...   ports: ...   type: ClusterIP --> NodePort     ### change value to NodePort status:  ... --------------------------------------- service/consul-server-ui edited ubuntu@onap-5422-rke-node:~$ kubectl -n onap get svc | grep consul consul-server-ui                                       NodePort       10.43.132.178   <none>                                 8500:31190/TCP                                           25h

   
   

4. After completing previous steps we can run call the start function, which provides Producers and starts the test. The table below contains the parameters that can be passed to cloud

   Code Block
   title Performing the test
   ./cloud-based-performance-test.sh

   script.

   start

   For the start function we can use optional arguments:

   --load	should the test keep defined number of running producers until script interruption (false)
   --containers	number of producer containers to create (1)
   --properties-file	path to file with benchmark properties (./test.properties

   gen_certsgenerate certs in ../../ssl directorysetupset up ConfigMaps and consumerssetup_allset up ConfigMaps consumers and producerssend_configsend producers configuration (message interval and payload), located in producers-config/producer-config.json to each producerstart_intervalstart interval mode, config file is located in producers-config/interval-config.json
   Optional parameters:
   --producers : number of producers in deployment (10

   )

   --retention-time-minutes

   : messages

   retention time

   on

   of messages in kafka in minutes (60)

   start_instantstart_instant : start instant mode, config file is located in producers-config/instant-config.json
   Optional parameters:
   --producers : number of producers in deployment (10)
   --retention-time-minutes : messages retention time on kafka in minutes (60)scale_producersscale producer deployment to number provide in argumentstopstop all producersreset_producersreset all metrics on each producercleanremove ConfigMap, HV-VES consumersclean_allremove ConfigMap, HV-VES consumers and producershelpprint usage

   Example invocations of test start:

   Code Block
   title Starting performance test with single producers creation
   ./cloud-based-performance-test.sh start --containers 10

   The command above starts the test that creates 10 producers which send the amount of messages defined in test.properties once.

   Code Block
   title Starting performance test with constant messages load
   Code Block
   title Performing the test
   ./cloud-based-performance-test.sh start_interval --load true --producerscontainers 10

   The command above starts the test that creates 10 producers which send messages in the interval mode. The parameters can be changed in configuration files located in producers-config folder.

   Code Block
   language js title interval-config.json collapse true
   { "intervalConfigs": [ { "duration": 1, "interval": 100 }, { "duration": 2, "interval": 100 } ] }

   Above request body queues a task to create new connections for 1 second with 100 milliseconds interval and then for 2 seconds with 100 milliseconds interval, meaning that overall of 30 connections should be setup over 3 seconds.

   Code Block
   language js title producer-config.json collapse true
   { "payloadSize": 8192, "messageIntervalMs": 100 }

   Above request updates configuration of producers to send messages with 8192 bytes with 100 milliseconds interval. 

   It is also possible to run the test in instant mode.

   Code Block
   language js title instant-config.json collapse true
   { "connections": 500 }

   Above request queues a task to create 500 connections without intervals between them.

The test.properties file contains Producers and Consumers configurations and it allows setting following properties:

...

1. --retention-time-minutes 30

   This invocation starts load test, meaning the script will try to keep the amount of running containers at 10 with kafka message retention of 30 minutes.

   
   

   The test.properties file contains Producers and Consumers configurations and it allows setting following properties:

   Producer
   hvVesAddress	HV-VES address (dcae-hv-ves-collector.onap:6061)
   client.count	Number of clients per pod (1)
   message.size	Size of a single message in bytes (16384)
   message.count	Amount of messages to be send by each client (1000)
   message.interval	Interval between messages in miliseconds (1)
   Certificates paths
   client.cert.path	Path to cert file (/ssl/client.p12)
   client.cert.pass.path	Path to cert's pass file (/ssl/client.pass)
   Consumer
   kafka.bootstrapServers	Adress of Kafka service to consume from (message-router-kafka:9092)
   kafka.topics	Kafka topics to subscribe to (HV_VES_PERF3GPP)

...

To remove created ConfigMaps, Consumers, Producers, Grafana and Prometheus from Kubernetes cluster we call clean function. Note: clean doesn't remove certificates from HV-VES.

./cloud-based-performance-test.sh clean

In order to restart the test environment, which means redeploying hv-ves pod, resetting kafka topic and performing setup, we use reboot-test-environment.sh.

...

...

1. 
   Results can be accessed under following links:

   • Prometheus: http://slave1:30000/graph?g0.range_input=1h&g0.expr=hv_kafka_consumer_travel_time_seconds_count&g0.tab=1
   • Grafana: http://slave1:30001/d/V94Kjlwmz/hv-ves-processing?orgId=1&refresh=5s

HV-VES Performance test results

...