1 answer
- 10-1
your Subject says "...But some pods are there"
Please show that list of pods and any logs for sick pods.
- Vaibhav Surwade
Hi there,
This is the pod list of so and dmaap. Despite both so and dmaap are failed in helm deploy.
ubuntu@rke:~/oom/kubernetes$ kubectl get pods --namespace onap | grep -e "dmaap" -e "so"
dev-aai-aai-resources-5d9bcff86c-bpsdn 2/2 Running 0 4h1m
dev-dmaap-dbc-pg-0 1/1 Running 0 4h1m
dev-dmaap-dbc-pg-1 1/1 Running 0 3h51m
dev-dmaap-dbc-pgpool-77b47cd74c-g6zcq 1/1 Running 0 4h1m
dev-dmaap-dbc-pgpool-77b47cd74c-n6hwq 1/1 Running 0 4h1m
dev-dmaap-dmaap-bc-56b98c56d7-gvnf4 1/1 Running 0 4h1m
dev-dmaap-dmaap-bc-post-install-wbxh9 0/1 Completed 0 4h1m
dev-dmaap-dmaap-dr-db-0 1/1 Running 0 4h1m
dev-dmaap-dmaap-dr-db-1 1/1 Running 0 3h57m
dev-dmaap-dmaap-dr-node-0 2/2 Running 0 4h1m
dev-dmaap-dmaap-dr-prov-68bf6677b-lqb2w 2/2 Running 0 4h1m
dev-dmaap-message-router-0 0/1 Init:0/1 22 4h1m
dev-dmaap-message-router-kafka-0 0/1 CrashLoopBackOff 4 3m42s
dev-dmaap-message-router-kafka-1 0/1 CrashLoopBackOff 4 3m38s
dev-dmaap-message-router-kafka-2 0/1 CrashLoopBackOff 4 3m26s
dev-dmaap-message-router-mirrormaker-57df4965fb-rx4dx 0/1 Init:1/2 21 4h1m
dev-dmaap-message-router-post-install-h4vgx 1/1 Running 0 3h50m
dev-dmaap-message-router-zookeeper-0 1/1 Running 0 4h1m
dev-dmaap-message-router-zookeeper-1 1/1 Running 0 4h1m
dev-dmaap-message-router-zookeeper-2 1/1 Running 0 4h1m
dev-oof-cmso-db-0 1/1 Running 0 3h35m
dev-oof-oof-cmso-optimizer-85dd97547f-nbnl4 1/1 Running 0 3h35m
dev-oof-oof-cmso-service-688c74d7c-rds6j 3/3 Running 0 3h35m
dev-oof-oof-cmso-ticketmgt-7d9cbbdcdb-z9qtk 1/1 Running 0 3h35m
dev-oof-oof-cmso-topology-5bdb8c5758-wdtn9 1/1 Running 0 3h35m
dev-oof-oof-has-solver-54f57575dc-gdnrn 1/1 Running 0 3h35m
dev-sdnc-cds-blueprints-processor-64cb456b8c-wm9pr 1/1 Running 0 3h34m
dev-sdnc-sdnc-dmaap-listener-79b9578894-ncjc6 0/1 Init:0/1 19 3h34m
dev-so-so-69c9cb9d89-px8rm 0/1 Init:0/1 20 3h34m
dev-so-so-bpmn-infra-5c8cb95779-vd85z 0/1 Init:0/1 19 3h34m
dev-so-so-catalog-db-adapter-579f89b46b-tlv5b 0/1 Init:0/1 19 3h34m
dev-so-so-mariadb-config-job-wx9xj 0/1 Init:0/2 0 6m
dev-so-so-monitoring-6f996bd4c9-qtfcv 1/1 Running 0 3h34m
dev-so-so-openstack-adapter-7fbd9498-zl64l 0/1 Init:0/1 19 3h34m
dev-so-so-request-db-adapter-7ddd8ddb4b-k5c9l 0/1 Init:0/1 20 3h34m
dev-so-so-sdc-controller-c96d945d-4cg9m 0/1 Init:0/1 20 3h34m
dev-so-so-sdnc-adapter-76f9c6c87c-ctgdz 1/1 Running 0 3h34m
dev-so-so-vfc-adapter-88d46f8d7-t7znz 0/1 Init:0/1 19 3h34m
dev-so-so-vnfm-adapter-6f468d94b-dkt9s 1/1 Running 0 3h34m - Dominic Lunanuova
The helm FAILED status is likely due to the message-router-kafka pods not completing, which also prevents some of the post-install scripts from completing (within the timeout). So, would next look into the details of the message-router-kafka logs.
- Vaibhav Surwade
Hello,
Below is the logs of that message-router-kafka pod please suggest how to solve this issue ?
+ export 'KAFKA_BROKER_ID=0'
+ export 'ENDPOINT_PORT=30490'
+ export 'KAFKA_ADVERTISED_LISTENERS=EXTERNAL_SASL_PLAINTEXT://10.0.0.12:30490,INTERNAL_SASL_PLAINTEXT://:9092'
+ exec start-kafka.sh
waiting for kafka to be ready
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/kafka_2.12-1.1.1/libs/kafka11aaf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/kafka_2.12-1.1.1/libs/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
[main] INFO kafka.utils.Log4jControllerRegistration$ - Registered kafka:type=kafka.Log4jController MBean
[main] INFO kafka.server.KafkaServer - starting
[main] INFO kafka.server.KafkaServer - Connecting to zookeeper on message-router-zookeeper:2181
[main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Initializing a new session to message-router-zookeeper:2181.
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=dev-dmaap-message-router-kafka-0.message-router-kafka.onap.svc.cluster.local
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.8.0_202
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=/opt/jdk1.8.0_202/jre
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=/opt/kafka/bin/../libs/activation-1.1.1.jar:/opt/kafka/bin/../libs/aopalliance-repackaged-2.5.0-b32.jar:/opt/kafka/bin/../libs/argparse4j-0.7.0.jar:/opt/kafka/bin/../libs/commons-lang3-3.5.jar:/opt/kafka/bin/../libs/connect-api-1.1.1.jar:/opt/kafka/bin/../libs/connect-file-1.1.1.jar:/opt/kafka/bin/../libs/connect-json-1.1.1.jar:/opt/kafka/bin/../libs/connect-runtime-1.1.1.jar:/opt/kafka/bin/../libs/connect-transforms-1.1.1.jar:/opt/kafka/bin/../libs/guava-20.0.jar:/opt/kafka/bin/../libs/hk2-api-2.5.0-b32.jar:/opt/kafka/bin/../libs/hk2-locator-2.5.0-b32.jar:/opt/kafka/bin/../libs/hk2-utils-2.5.0-b32.jar:/opt/kafka/bin/../libs/jackson-annotations-2.9.6.jar:/opt/kafka/bin/../libs/jackson-core-2.9.6.jar:/opt/kafka/bin/../libs/jackson-databind-2.9.6.jar:/opt/kafka/bin/../libs/jackson-jaxrs-base-2.9.6.jar:/opt/kafka/bin/../libs/jackson-jaxrs-json-provider-2.9.6.jar:/opt/kafka/bin/../libs/jackson-module-jaxb-annotations-2.9.6.jar:/opt/kafka/bin/../libs/javassist-3.20.0-GA.jar:/opt/kafka/bin/../libs/javassist-3.21.0-GA.jar:/opt/kafka/bin/../libs/javax.annotation-api-1.2.jar:/opt/kafka/bin/../libs/javax.inject-1.jar:/opt/kafka/bin/../libs/javax.inject-2.5.0-b32.jar:/opt/kafka/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/kafka/bin/../libs/javax.ws.rs-api-2.0.1.jar:/opt/kafka/bin/../libs/jaxb-api-2.3.0.jar:/opt/kafka/bin/../libs/jersey-client-2.25.1.jar:/opt/kafka/bin/../libs/jersey-common-2.25.1.jar:/opt/kafka/bin/../libs/jersey-container-servlet-2.25.1.jar:/opt/kafka/bin/../libs/jersey-container-servlet-core-2.25.1.jar:/opt/kafka/bin/../libs/jersey-guava-2.25.1.jar:/opt/kafka/bin/../libs/jersey-media-jaxb-2.25.1.jar:/opt/kafka/bin/../libs/jersey-server-2.25.1.jar:/opt/kafka/bin/../libs/jetty-client-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-continuation-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-http-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-io-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-security-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-server-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-servlet-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-servlets-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jetty-util-9.2.24.v20180105.jar:/opt/kafka/bin/../libs/jopt-simple-5.0.4.jar:/opt/kafka/bin/../libs/kafka-clients-1.1.1.jar:/opt/kafka/bin/../libs/kafka-log4j-appender-1.1.1.jar:/opt/kafka/bin/../libs/kafka-streams-1.1.1.jar:/opt/kafka/bin/../libs/kafka-streams-examples-1.1.1.jar:/opt/kafka/bin/../libs/kafka-streams-test-utils-1.1.1.jar:/opt/kafka/bin/../libs/kafka-tools-1.1.1.jar:/opt/kafka/bin/../libs/kafka11aaf-jar-with-dependencies.jar:/opt/kafka/bin/../libs/kafka_2.12-1.1.1-sources.jar:/opt/kafka/bin/../libs/kafka_2.12-1.1.1-test-sources.jar:/opt/kafka/bin/../libs/kafka_2.12-1.1.1.jar:/opt/kafka/bin/../libs/log4j-1.2.17.jar:/opt/kafka/bin/../libs/lz4-java-1.4.1.jar:/opt/kafka/bin/../libs/maven-artifact-3.5.3.jar:/opt/kafka/bin/../libs/metrics-core-2.2.0.jar:/opt/kafka/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/kafka/bin/../libs/plexus-utils-3.1.0.jar:/opt/kafka/bin/../libs/reflections-0.9.11.jar:/opt/kafka/bin/../libs/rocksdbjni-5.7.3.jar:/opt/kafka/bin/../libs/scala-library-2.12.4.jar:/opt/kafka/bin/../libs/scala-logging_2.12-3.8.0.jar:/opt/kafka/bin/../libs/scala-reflect-2.12.4.jar:/opt/kafka/bin/../libs/slf4j-api-1.7.25.jar:/opt/kafka/bin/../libs/slf4j-log4j12-1.7.25.jar:/opt/kafka/bin/../libs/snappy-java-1.1.7.1.jar:/opt/kafka/bin/../libs/validation-api-1.1.0.Final.jar:/opt/kafka/bin/../libs/zkclient-0.10.jar:/opt/kafka/bin/../libs/zookeeper-3.4.10.jar
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=/tmp
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Linux
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=4.15.0-51-generic
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=mrkafka
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=/home/mrkafka
[main] INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=/
[main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=message-router-zookeeper:2181 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@223f3642
[main] INFO kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Waiting until connected.
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local/10.42.4.106:2181
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local/10.42.4.106:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local/10.42.8.94:2181
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local/10.42.8.94:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local/10.42.5.123:2181
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local/10.42.5.123:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local/10.42.4.106:2181
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local/10.42.4.106:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local/10.42.8.94:2181
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local/10.42.8.94:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local/10.42.5.123:2181
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local/10.42.5.123:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local/10.42.4.106:2181
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local/10.42.4.106:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-1.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local/10.42.8.94:2181
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local/10.42.8.94:2181, initiating session
[main-SendThread(dev-dmaap-message-router-zookeeper-2.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] WARN org.apache.zookeeper.ClientCnxn - SASL configuration failed: javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/opt/kafka/config/kafka_server_jaas.conf'. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local/10.42.5.123:2181
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local/10.42.5.123:2181, initiating session
[main-EventThread] ERROR kafka.zookeeper.ZooKeeperClient - [ZooKeeperClient] Auth failed.
[main-SendThread(dev-dmaap-message-router-zookeeper-0.message-router-zookeeper.onap.svc.cluster.local:2181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect
[main] ERROR kafka.server.KafkaServer - Fatal error during KafkaServer startup. Prepare to shutdown
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:225)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:250)
at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:221)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:95)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1548)
at kafka.server.KafkaServer.createZkClient$1(KafkaServer.scala:348)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:372)
at kafka.server.KafkaServer.startup(KafkaServer.scala:202)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:75)
at kafka.Kafka.main(Kafka.scala)
[main] INFO kafka.server.KafkaServer - shutting down
[main] WARN kafka.utils.CoreUtils$ - null
java.lang.NullPointerException
at kafka.server.KafkaServer.$anonfun$shutdown$6(KafkaServer.scala:572)
at kafka.utils.CoreUtils$.swallow(CoreUtils.scala:85)
at kafka.server.KafkaServer.shutdown(KafkaServer.scala:572)
at kafka.server.KafkaServer.startup(KafkaServer.scala:329)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:38)
at kafka.Kafka$.main(Kafka.scala:75)
at kafka.Kafka.main(Kafka.scala)
[main] INFO kafka.server.KafkaServer - shut down completed
[main] ERROR kafka.server.KafkaServerStartable - Exiting Kafka.
[kafka-shutdown-hook] INFO kafka.server.KafkaServer - shutting down
Add your comment...
I deployed onap dublin release using this command
$helm deploy dev local/onap --namespace onap -f onap/resources/overrides/onap-all.yaml -f onap/resources/overrides/openstack.yaml --timeout 900
fetching local/onap
release "dev" deployed
release "dev-aaf" deployed
release "dev-aai" deployed
release "dev-appc" deployed
release "dev-cassandra" deployed
release "dev-clamp" deployed
release "dev-cli" deployed
release "dev-consul" deployed
release "dev-contrib" deployed
release "dev-dcaegen2" deployed
release "dev-dmaap" deployed
release "dev-esr" deployed
release "dev-log" deployed
release "dev-mariadb-galera" deployed
release "dev-msb" deployed
release "dev-multicloud" deployed
release "dev-nbi" deployed
release "dev-oof" deployed
release "dev-policy" deployed
release "dev-pomba" deployed
release "dev-portal" deployed
release "dev-robot" deployed
release "dev-sdc" deployed
release "dev-sdnc" deployed
release "dev-sniro-emulator" deployed
release "dev-so" deployed
release "dev-uui" deployed
release "dev-vfc" deployed
release "dev-vid" deployed
release "dev-vnfsdk" deployed
dev-dmaap 1 Wed Jun 19 03:38:31 2019 FAILED dmaap-4.0.1 onap
dev-so 1 Wed Jun 19 03:54:30 2019 FAILED so-4.0.0 onap
In this dmaap and so are showing failed. But when I check pod list it shows some pods of dmaap and so.
I also tried to redeploy these individual components, but those also failed.
$ helm deploy dev-so local/so --namespace onap
fetching local/so
Error: UPGRADE FAILED: "dev-so" has no deployed releases
release "dev-so" deployed
dev-so 1 Wed Jun 19 03:54:30 2019 FAILED so-4.0.0 onap
$ helm deploy dev-dmaap local/dmaap --namespace onap
fetching local/dmaap
Error: UPGRADE FAILED: "dev-dmaap" has no deployed releases
release "dev-dmaap" deployed
dev-dmaap 1 Wed Jun 19 03:38:31 2019 FAILED dmaap-4.0.1 onap
How to solve this issue?