You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 13 Next »

General information

Purpose

The goal of the collector is to support high volume data. It uses plain TCP connections tunneled in SSL/TLS. Connections are stream-based (as opposed to request-based) and long running. Payload is binary-encoded (currently we are using Google Protocol Buffers). HV-VES uses direct connection to DMaaP's Kafka. All these decisions were made in order to support high-volume data with minimal latency.

For more details on the rationale, please read a high-level feature description.

Background

VES-HV collector has been proposed, based on a need to process high-volumes of data generated by a large number of NFs. The driving use-case is the 5G RAN, where it is expected that up to 10k NF instances report the data. It is expected as well, that the frequency of data transmission will be pretty high (per PNF instance). The network traffic generated in simulations has shown, that GPB serialization is 2-3 times more effective, than JSON serialization existing with VES collector.

Results have been published within ONAP presentation in Casablanca Release Developer Forum:  Google Protocol Buffers versus JSON - 5G RAN use-case - comparison

Implementation details

Technology stack

  • Project Reactor is used as a backbone of the internal architecture.
  • Netty is used by means of reactor-netty library.
  • We are using Kotlin so we can write very concise code with great interoperability with existing Java libraries.
  • Types defined in Λrrow library are also used when it improves readability or general cleanness of the code.

Rules

  • Do not block. Use non-blocking libraries. Do not use block* Reactor calls inside the core of the application.
  • Pay attention to memory usage.
  • Do not decode the payload - it can be of a considerable size. The goal is to direct the event into a proper Kafka topic. The routing logic should be based only on VES Common Header.
  • All application logic should be defined in hv-collector-core module and tested on a component level by tests defined in hv-collector-ct. The core module should have a clean interface (defined in boundary package: api and adapters).
  • Use Either functional data type when designing fail-cases inside the main Flux. Using exceptions is a bit like using goto + it adds some performance penalty: collecting stack trace might be costly but we do not usually need it in such cases. RuntimeExceptions should be treated as application bugs and fixed.
  • No labels