Versions Compared

Old Version 7

changes.mady.by.user Halil Cakal

Saved on

compared with

New Version Current

changes.mady.by.user Halil Cakal

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents


References


CPS-1613 Explore How to Test Kafka in the CSITshttps://jira.onap.org/browse/CPS-1601

Assumptions

from "19092" "" in The possible affected classes must be updated hostname changed from "kafka" to "localhost" in docker-compose.ymlThe possible affected classes must be updated
#IssueNotesDecisions
1kafka container port issueThe exposed port of kafka container changed to 9092 in docker-compose.ymlAs per the technical discussion, there is no hard-coded port given in any java classes.
2KAFKA_ADVERTISED_LISTENERS hostname resolution issueThe port of KAFKA_ADVERTISED_LISTENERS has been changed to 29092.As per the technical discussion, there is no hard-coded port given in any java classes.

Issues & Decisions

to "9092"? "kafka" to "localhost" in the listener?2- Is it possible to update /etc/hosts in ONAP Jenkins?
#IssueNotesDecisions
1kafka container port issue

Open Questions:

1- Can we change the port

to 9092

Since docker-compose.yml is being used for local env. setup and CSIT, it is possible to change to 9092.
2KAFKA_ADVERTISED_LISTENERS hostname resolution issue

Open Questions:

1- Can we change the

port to 29092

As per local tests run by the Team 2, and ONAP build, the required change has no negative effect.

Overview

Can Robot Framework test Kafka clusters and events?

...

 Issues

1-In the current implementation of ConfluentKafkaLibrary, the consumer module connects to Kafka cluster from the port "9092".

...

Whereas, CPS's kafka container expose its port from "19092"


Code Block
languagexmlyml
titleCPS's kafka containerThe previous Kafka container config
   kafka:
    image: confluentinc/cp-kafka:6.2.1
    container_name: kafka
    ports:
      - "19092:19092"
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,CONNECTIONS_FROM_HOST://localhost:19092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

...

In order to solve this conflict, the current port has been changed to 9092 in docker-compose.yml


Code Block
languageyml
titleThe new Kafka container config
  kafka:
    image: confluentinc/cp-kafka:6.2.1
    container_name: kafka
    ports:
      - '9092:9092'
    depends_on:
      - zookeeper
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,CONNECTIONS_FROM_HOST:PLAINTEXT
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,CONNECTIONS_FROM_HOST://localhost:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1



2-The ConfluentKafkaLibrary also try to access the KAFKA_ADVERTISED_LISTENERS (PLAINTEXT://kafka:9092) natively.

However, it gets fail because of hostname to IP address resolution issue. There are two is a possible solution for this problem.

a- Add an entry into /etc/hosts like "127.0.0.1 kafka"

...

A change in the KAFKA_ADVERTISED_LISTENERS port to 29092

KAFKA_ADVERTISED_LISTENERS

...

: PLAINTEXT://

...

kafka:

...

29092,CONNECTIONS_FROM_HOST://localhost:

...

909