Difference between revisions of "Kafka Cheat Sheet"

From Gejoreuy
Jump to navigation Jump to search
(Created page with "'''How to List Kafka Topics''' Syntac : $ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --list Eg : $ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --li...")
 
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
 +
'''How to Create Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1
 +
 +
 +
'''How to Delete Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic
 +
 +
 
'''How to List Kafka Topics'''
 
'''How to List Kafka Topics'''
  
Line 8: Line 30:
  
 
  $ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic
 
  $ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic
 +
 +
'''How to Describe Kafka Topics'''
 +
 +
Syntac :
 +
 +
$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --describe --topic [topic-name]
 +
 +
Eg :
 +
 +
$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --describe --topic TestDevOps1Topic
 +
 +
 +
'''How to Produce Message to Topic'''
 +
 +
Syntac :
 +
 +
$ echo "[Message Content]" | kafka-console-producer.sh --broker-list [broker-node]:9092 --topic [topic-name]
 +
 +
Eg :
 +
 +
$ echo "Test Message 5" | kafka-console-producer.sh --broker-list de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic
 +
 +
 +
'''How to Consume Message from Topic'''
 +
 +
Syntac :
 +
 +
$ kafka-console-consumer.sh --bootstrap-server [bootstrap-node]:9092 --topic [topic-name] --from-beginning
 +
 +
Eg :
 +
 +
$ kafka-console-consumer.sh --bootstrap-server de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic --from-beginning
 +
 +
 +
'''How to Check Topics Offset Count'''
 +
 +
Syntac :
 +
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1
 +
 +
Eg :
 +
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
 +
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1
 +
 +
'''How to Check Kafka Cluster Broker Status'''
 +
 +
Syntac :
 +
 +
$ zookeeper-shell.sh [zookeeper-node]:2181 ls /brokers/ids
 +
 +
Eg :
 +
 +
$ zookeeper-shell.sh de-cicd5qa1au-kafka-vm-0:2181 ls /brokers/ids
 +
$ zookeeper-shell.sh de-cicd5qa1au-kafka-vm-1:2181 ls /brokers/ids

Latest revision as of 13:55, 22 August 2022

How to Create Topic

Syntac : 

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --create --topic [topic-name] --replication-factor [number-of-replication-factor] --partitions [number-of-partitions]

Eg : 

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --create --topic TestDevOps1Topic --replication-factor 1 --partitions 1


How to Delete Topic

Syntac : 

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --delete --topic [topic-name]

Eg : 

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --delete --topic TestDevOps1Topic


How to List Kafka Topics

Syntac : 

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --list

Eg : 

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --list | grep TestTopic

How to Describe Kafka Topics

Syntac : 

$ kafka-topics.sh --zookeeper [zookeeper-node]:2181 --describe --topic [topic-name]

Eg : 

$ kafka-topics.sh --zookeeper de-devcluster2-kafka-vm-0:2181 --describe --topic TestDevOps1Topic


How to Produce Message to Topic

Syntac : 

$ echo "[Message Content]" | kafka-console-producer.sh --broker-list [broker-node]:9092 --topic [topic-name]

Eg : 

$ echo "Test Message 5" | kafka-console-producer.sh --broker-list de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic


How to Consume Message from Topic

Syntac : 

$ kafka-console-consumer.sh --bootstrap-server [bootstrap-node]:9092 --topic [topic-name] --from-beginning

Eg : 

$ kafka-console-consumer.sh --bootstrap-server de-devcluster2-kafka-vm-0:9092 --topic TestDevOps1Topic --from-beginning


How to Check Topics Offset Count

Syntac : 

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name]
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list [broker-node]:9092 --topic [topic-name] --time -1

Eg : 

$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion
$ kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list de-devcluster2-kafka-vm-0:9092 --topic rndtest1.datafabric.storage.egestion --time -1

How to Check Kafka Cluster Broker Status

Syntac : 

$ zookeeper-shell.sh [zookeeper-node]:2181 ls /brokers/ids

Eg : 

$ zookeeper-shell.sh de-cicd5qa1au-kafka-vm-0:2181 ls /brokers/ids
$ zookeeper-shell.sh de-cicd5qa1au-kafka-vm-1:2181 ls /brokers/ids
Retrieved from "https://gejoreuy.com/index.php?title=Kafka_Cheat_Sheet&oldid=2086"