Performance

The following performance tests can be found in the repo, https://gerrit.onap.org/r/admin/repos/integration, under "test/mocks/datafilecollector-testharness/auto-test". The "README.md" file explains how to run the tests.

Memory and thread consumption

During maximum load, 3500 new files every poll for 24h, the memory consumption vary between 2 and 5,5 GB during file processing. The threads used va

File handling capacity

Note! The simulators (MR, DR, and PNF) are run on the same hardware as the DFC during all the tests below.

700 nodes (700 events) with 1MB file each, 5 polls → total 3500 files (3.5 GB)
(From polling to all files downloaded and published)

HWTest CaseFTP protDublin version: 1.1.2Dublin version: 1.1.3
Execution time (minutes)Files per second
Execution time (minutes)Files per second
Ubuntu 16, 8 vCPU, 32GBFTC10SFTP8,56,96,59
Ubuntu 16, 8 vCPU, 32GBFTC11FTPS3,516,72,523


700 nodes (3500 event) with 100 1MB files in 100 polls → total 72800 files (72.8 GB)

(From polling to all files downloaded and published)

HWTest CaseFTP protDublin version: 1.1.2Dublin version: 1.1.3
Execution time (minutes)Files per secondExecution time (minutes)Files per second
Ubuntu 16, 8 vCPU, 32GBFTC20SFTP1378,91239,8
Ubuntu 16, 8 vCPU, 32GBFTC21FTPS6817,86020


Max load test on DFC, polling 3500 new files every poll for 24h (from 700 nodes)

HWTest CaseFTP protDublin version: 1.1.2Dublin version: 1.1.3
Polled, downloaded and published filesFiles per secondPolled, downloaded and published filesFiles per second
Ubuntu 16, 8 vCPU, 32GBMaxFiles1MB_SFTP_24hSFTP2150002,53650004,2
Ubuntu 16, 8 vCPU, 32GBMaxFiles1MB_FTPS_24hFTPS99050011,5106374112,3


3 day test. 100 files from 700 nodes in the first event from each node. Then one new file per node every 15 min.

HWTest CaseFTP protDublin version: 1.1.2Dublin version 1.1.3
Polled, downloaded and published files
Ubuntu 16, 8 vCPU, 32GBStability1MB_SFTP_72hSFTP270900 (expected number of files)270900 (expected number of files)
Ubuntu 16, 8 vCPU, 32GBStability1MB_FTPS_72hFTPS270060 (840 files less than target. Not all 72h polls where made probably due to delays in the simulators or too agressive test criterias. However all polled files downloaded and published successfully. )269780 (1120 files less than target. Not all 72h polls where made probably due to delays in the simulators or too agressive test criterias. However all polled files downloaded and published successfully. )