Skip to content

About

kafka cluster in docker with three nodes

Resources

Stars

1 star

Watchers

1 watching

Forks

Latest commit

 

History

11 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

title kafka 3.1.1 cluster with three nodes in docker

ENV

bitnami/kafka 3.1.1

Docker version 19.03.5, build 633a0ea838

jdk 14.0.2

Fedora 29 x86_64 workstation

zookeeper setting

see zookeeper cluster in docker

      - KAFKA_CFG_ZOOKEEPER_CONNECT=zoo1:2181,zoo2:2181,zoo3:2181

listener setting

      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=BROKERCLIENT:PLAINTEXT,HOSTCLIENT:PLAINTEXT
      - KAFKA_CFG_LISTENERS=BROKERCLIENT://0.0.0.0:9092,HOSTCLIENT://0.0.0.0:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=BROKERCLIENT://kafka1:9092,HOSTCLIENT://localhost:19093
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=BROKERCLIENT

advertised listener setting

  • connections among borkers
  1. initial connect, e.g. kafka2:9092
  2. kafka2 response with metadata. "kafka2:9092" should be used for the client to produce events or consume events.
  3. The client connects the broker for produce operation or consume operation etc.
  • connections among borkers and host
  1. initial connect, e.g. localhost:19093
  2. kafka1 response with metadata. "localhost:19093" should be used for the client on host to produce events or consume events.
  3. The client on host connects the broker for produce operation or consume operation etc.

Client

[hostuser@host-machine kafka$]docker exec --interactive --tty kafka-cluster-3 bash
I have no name!@kafka3:/opt/bitnami/kafka$ bin/kafka-console-producer.sh --broker-list kafka3:9092 --topic test
>first event
>second event
>
[hostuser@host-machine kafka$]docker exec --interactive --tty kafka-cluster-1 bash
I have no name!@kafka1:/opt/bitnami/kafka$ bin/kafka-console-consumer.sh --bootstrap-server kafka1:9092 --topic test --from-beginning
first event
second event
[hostuser@host-machine kafka$]bin/kafka-console-consumer.sh --bootstrap-server localhost:29093 --topic test --from-beginning
first event
second event

see more scripts in folder scripts/

Java Api

see code in this project

About

kafka cluster in docker with three nodes

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages