Difference between revisions of "Kafka Cheat Sheet"

From Gejoreuy
Jump to navigation Jump to search
(Created page with "'''How to List Kafka Topics''' Syntac : $ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --list Eg : $ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --li...")
 
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
'''How to Create Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1
 +
 +
 +
'''How to Delete Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic
 +
 +
 
'''How to List Kafka Topics'''
 
'''How to List Kafka Topics'''
  
Line 8: Line 30:
  
 
  $ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic
 
  $ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic
 +
 +
'''How to Describe Kafka Topics'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --describe --topic [topic-name]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --describe --topic TestDevOps1Topic
 +
 +
 +
'''How to Produce Message to Topic'''
 +
 +
Syntac :
 +
 +
$ echo "[Message Content]" | kafka-console-producer.sh --broker-list [broker-node]:9092 --topic [topic-name]
 +
 +
Eg :
 +
 +
$ echo "Test Message 5" | kafka-console-producer.sh --broker-list de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic
 +
 +
 +
'''How to Consume Message from Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-console-consumer.sh --bootstrap-server [bootstrap-node]:9092 --topic [topic-name] --from-beginning
 +
 +
Eg :
 +
 +
$ kafka-console-consumer.sh --bootstrap-server de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic --from-beginning
 +
 +
 +
'''How to Check Topics Offset Count'''
 +
 +
Syntac :
 +
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1
 +
 +
Eg :
 +
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1

Latest revision as of 14:27, 1 April 2022

How to Create Topic

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions] 

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1 


How to Delete Topic

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic


How to List Kafka Topics

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --list

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic

How to Describe Kafka Topics

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --describe --topic [topic-name]

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --describe --topic TestDevOps1Topic


How to Produce Message to Topic

Syntac :

$ echo "[Message Content]" | kafka-console-producer.sh --broker-list [broker-node]:9092 --topic [topic-name]

Eg :

$ echo "Test Message 5" | kafka-console-producer.sh --broker-list de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic


How to Consume Message from Topic

Syntac :

$ kafka-console-consumer.sh --bootstrap-server [bootstrap-node]:9092 --topic [topic-name] --from-beginning

Eg :

$ kafka-console-consumer.sh --bootstrap-server de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic --from-beginning


How to Check Topics Offset Count

Syntac :

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1

Eg :

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1