...
- VES-HV has been designed as a high-volume variant of the existing VES(JSON) collector, and not a completely new collector
- VES-HV follows the VES-JSON schema - as much as possible
- It uses a PROTO representation of the VES Common Header
- The PROTO files tend to use most encoding effective types defined by GPB to cover Common Header fields.
- It makes routing decisions based mostly on the content of the "Domain" parameter
- It allows to embed Payload of different types (by default hvMeas domain is included)
- VES-HV publishes events on DMaaP-Kafka bus, using native Kafka Interfaces
- An analytics application operating on high-volume data needs to be prepared to read directly from Kafka
- An analytics application need to operate on GPB encoded data in order to benefit from GPB encoding efficiencies
...
Extendability
VES-HV was designed to allow for extensibility extendability - by adding new domain-specific PROTO files. The PROTO file, which contains the VES CommonHeader, comes with a binary-type Payload parameter, where domain-specific data shall be placed. Domain-specific data are encoded as well with GPB, and they do require a domain-specific PROTO file to decode the data. This domain-specific PROTO needs to be shared with analytics applications - VES-HV is not analyzing domain-specific data.
...