Kafka Cheat Sheet

How to Create Topic

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions] 

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1 

How to Delete Topic

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic

How to List Kafka Topics

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --list

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic

How to Describe Kafka Topics

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --describe --topic [topic-name]

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --describe --topic TestDevOps1Topic

How to Produce Message to Topic

Syntac :

$ echo "[Message Content]" | kafka-console-producer.sh --broker-list [broker-node]:9092 --topic [topic-name]

Eg :

$ echo "Test Message 5" | kafka-console-producer.sh --broker-list de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic

How to Consume Message from Topic

Syntac :

$ kafka-console-consumer.sh --bootstrap-server [bootstrap-node]:9092 --topic [topic-name] --from-beginning

Eg :

$ kafka-console-consumer.sh --bootstrap-server de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic --from-beginning

How to Check Topics Offset Count

Syntac :

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1

Eg :

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1

How to Check Kafka Cluster Broker Status

Syntac :

$ zookeeper-shell.sh [zookeeper-node]:2181 ls /brokers/ids

Eg :

$ zookeeper-shell.sh de-cicd5qa1au-kafka-vm-0:2181 ls /brokers/ids
$ zookeeper-shell.sh de-cicd5qa1au-kafka-vm-1:2181 ls /brokers/ids