...
Kafka Broker Details (Bitnami)
Folders
Kafka Home Folder: /opt/bitnami/kafka
Folders:
| Code Block |
|---|
$ ls -l total 48 -rw-rw-r-- 1 root root 14520 Sep 21 09:58 LICENSE -rw-rw-r-- 1 root root 953 Sep 21 09:58 NOTICE drwxrwxr-x 1 root root 4096 Sep 21 10:06 bin drwxrwxr-x 1 root root 4096 Oct 5 03:34 config drwxrwxr-x 1 root root 4096 Sep 21 10:06 libs drwxrwxr-x 1 root root 4096 Sep 21 10:06 licenses drwxrwxr-x 1 root root 4096 Oct 6 17:14 logs drwxrwxr-x 1 root root 4096 Sep 21 10:06 site-docs |
Commands
| Code Block |
|---|
## Creating new Topics
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--create \
--topic kafka.learning.tweets \
--partitions 1 \
--replication-factor 1
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--create \
--topic kafka.learning.alerts \
--partitions 1 \
--replication-factor 1
## Listing Topics
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--list
## Getting details about a Topic
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--describe
## Publishing Messages to Topics
./kafka-console-producer.sh \
--bootstrap-server localhost:29092 \
--topic kafka.learning.tweets
## Consuming Messages from Topics
./kafka-console-consumer.sh \
--bootstrap-server localhost:29092 \
--topic kafka.learning.tweets \
--from-beginning
## Deleting Topics
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--delete \
--topic kafka.learning.alerts
#Create a Topic with multiple partitions
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--create \
--topic kafka.learning.orders \
--partitions 3 \
--replication-factor 1
#Check topic partitioning
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--topic kafka.learning.orders \
--describe
## Publishing Messages to Topics with keys
./kafka-console-producer.sh \
--bootstrap-server localhost:29092 \
--property "parse.key=true" \
--property "key.separator=:" \
--topic kafka.learning.orders
## Consume messages using a consumer group
./kafka-console-consumer.sh \
--bootstrap-server localhost:29092 \
--topic kafka.learning.orders \
--group test-consumer-group \
--property print.key=true \
--property key.separator=" = " \
--from-beginning
## Check current status of offsets
./kafka-consumer-groups.sh \
--bootstrap-server localhost:29092 \
--describe \
--all-groups
## Creating the Topic
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--create \
--topic kafka.usecase.students \
--partitions 2 \
--replication-factor 1
## Describe the Topic
./kafka-topics.sh \
--zookeeper zookeeper:2181 \
--topic kafka.usecase.students \
--describe
## Publish to the Topic
./kafka-console-producer.sh \
--bootstrap-server localhost:29092 \
--property "parse.key=true" \
--property "key.separator=:" \
--topic kafka.usecase.students
## Consume Message from the Topic
./kafka-console-consumer.sh \
--bootstrap-server localhost:29092 \
--topic kafka.usecase.students \
--group usecase-consumer-group \
--property print.key=true \
--property key.separator=" = " \
--from-beginning |
References
| Reference | URL |
|---|---|
| Apache Kafka in 5 minutes | https://www.youtube.com/watch?v=PzPXRmVHMxI |
| Nokia Learning - Kafka | https://nokialearn.csod.com/ui/lms-learning-details/app/course/dc425d19-5642-535f-916c-211768f90a00 |
| Kafka Topics, Partitions and Offsets Explained | https://www.youtube.com/watch?v=_q1IjK5jjyU |
| Kafka Helm Charts | https://github.com/confluentinc/cp-helm-charts |
Confluent for Kubernetes | https://docs.confluent.io/operator/current/overview.html#operator-about-intro |
...