As DataLake infrastructure itself will be considered outside of ONAP scope, the POC will focus on delivering one of key component of DL-Feeder as service under DCAE project.
Architecture
...
POC Scope
- DataLake Feeder delivery as service under DCAE (dcaegen2/services/dl-handler/dl-feeder)
- DLFeeder will support pluggable sources and datastore support
- DMAAP/Kafka as primary source (option to poll kafka broker for new topic?)
- Druid/Couchbase/Elasticsearch/Hadoop datastore support
- ConfigBingingService integration to fetch deployment/run-time configuration
- Alternate option will be to leverage k8s Configmap
- DLAdmin delivery as service under DCAE (dcaegen2/services/dl-handler/dl-admin). Will integrate the DL Admin UI into DCAE dashboard, which is built on portal SDK and serves as UAM for DCAE users/operation.
- Integrate DLAdmin into DCAE-Dashboard (See note below)
Other extension cases (stretch goal)
- RDMS datastore support
- Support other input data stream beside Dmaap/kafka
- Integrate with PNDA/datastore
- DCAE Platform integration for dynamic topic configuration
- Logging project integration
Deliverable
- DL Feeder container
- DL Admin GUI container
- Documentation (wiki and readme files) to setup/configure above
- Instruction for Test storage and UI setup and DL configuration management to demo data flow and visualization
...
- Karl Chiang
- Kate Hsuan
- Ekko Chang
IBM
Driptaroop Das
...