References
https://jira.onap.org/browse/CPS-1601
Assumptions
# | Issue | Notes | Decisions |
---|---|---|---|
1 | kafka container port issue | The exposed port of kafka container changed from "19092" to "9092" in docker-compose.yml | As per the technical discussion, there is no hard-coded port given in any java classes. |
2 | KAFKA_ADVERTISED_LISTENERS hostname resolution issue | The hostname changed from "kafka" to "localhost" in docker-compose.yml | As per the technical discussion, there is no hard-coded port given in any java classes. |
Issues & Decisions
# | Issue | Notes | Decisions |
---|---|---|---|
1 | kafka container port issue | Open Questions: 1- Can we change the port to "9092"? | Since docker-compose.yml is being used for local env. setup and CSIT, it is possible to change to 9092. |
2 | KAFKA_ADVERTISED_LISTENERS hostname resolution issue | Open Questions: 1- Can we change the "kafka" to "localhost" in the listener? 2- Is it possible to update /etc/hosts in ONAP Jenkins? | As per local tests run by the team 2, and ONAP build, the required change has no negative effect. |
Overview
Can Robot Framework test Kafka clusters and events?
- In the latest version of Robot Framework (6.0.2), there is no support for Kafka Event Streaming.
Is there any third-party library that would aid in this?
- ConfluentKafkaLibrary is a Robot Framework library to verify Kafka Event Streaming.
- Robotframework-ConfluentKafkaLibrary
- API for robotframework-ConfluentKafkaLibrary
Issues
1-In the current implementation of ConfluentKafkaLibrary, the consumer module connects to Kafka cluster from the port "9092".
def __init__( self, server='127.0.0.1', port='9092', topics='', group_id=None, only_value=True, **kwargs ):
Whereas, CPS's kafka container expose its port from "19092"
kafka: image: confluentinc/cp-kafka:6.2.1 container_name: kafka ports: - "19092:19092" depends_on: - zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,CONNECTIONS_FROM_HOST://localhost:19092 KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
In order to solve this conflict, the current port has been changed to 9092 in docker-compose.yml
2-The ConfluentKafkaLibrary also try to access the KAFKA_ADVERTISED_LISTENERS (PLAINTEXT://kafka:9092) natively.
However, it gets fail because of hostname to IP address resolution issue. There are two possible solution for this problem.
a- Add an entry into /etc/hosts like "127.0.0.1 kafka"
b- Change the KAFKA_ADVERTISED_LISTENERS into PLAINTEXT://localhost:9092,CONNECTIONS_FROM_HOST://localhost:19092
The second solution is being applied in the latest patch set.