Difference between revisions of "Kafka Cheat Sheet"

From Gejoreuy
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
 +
'''How to Create Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1
 +
 +
 +
'''How to Delete Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic
 +
 +
 
'''How to List Kafka Topics'''
 
'''How to List Kafka Topics'''
  
Line 42: Line 64:
  
  
'''How to Check Topics Offset'''
+
'''How to Check Topics Offset Count'''
  
 
Syntac :
 
Syntac :
  
  $ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --time -1 --topic [topic-name]
+
  $ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1
  
 
Eg :
 
Eg :
  
  $ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --time -1 --topic rndtest1.datafabric.storage.egestion
+
  $ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
 
+
  $ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1
 
 
'''How to Delete Topic'''
 
 
 
Syntac :
 
 
 
  $ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]
 
 
 
Eg :
 
 
 
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic
 

Latest revision as of 14:27, 1 April 2022

How to Create Topic

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions] 

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1 


How to Delete Topic

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic


How to List Kafka Topics

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --list

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic

How to Describe Kafka Topics

Syntac :

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --describe --topic [topic-name]

Eg :

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --describe --topic TestDevOps1Topic


How to Produce Message to Topic

Syntac :

$ echo "[Message Content]" | kafka-console-producer.sh --broker-list [broker-node]:9092 --topic [topic-name]

Eg :

$ echo "Test Message 5" | kafka-console-producer.sh --broker-list de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic


How to Consume Message from Topic

Syntac :

$ kafka-console-consumer.sh --bootstrap-server [bootstrap-node]:9092 --topic [topic-name] --from-beginning

Eg :

$ kafka-console-consumer.sh --bootstrap-server de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic --from-beginning


How to Check Topics Offset Count

Syntac :

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1

Eg :

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1