[Kafka] Kafka Console Producer CLI.

Producers

• Producers write data to topics (which is made of partitions)
• Producers automatically know to which broker and partition to write to
• In case of Broker failures, Producers will automatically recover

• Producers can choose to receive acknowledgment of data writes:
• acks=0: Producer won’t wait for acknowledgment (possible data loss)
• acks=1: Producer will wait for leader acknowledgment (limited data loss)
• acks=all: Leader + replicas acknowledgment (no data loss)

Producers: Message keys

• Producers can choose to send a key with the message (string, number, etc..)
• If key=null, data is sent round robin (broker 101 then 102 then 103…)
• If a key is sent, then all messages for that key will always go to the same partition
• A key is basically sent if you need message ordering for a specific field (ex: truck_id)

kafka-console-producer.sh

chúng ta list ra xem chúng ta đã tạo topics nào trước đó?

kafka-topics.sh  --bootstrap-server localhost:9092 --list

kafka-console-producer.sh  --bootstrap-server localhost:9092 --topic first-topic

Chọn receive acknowledgment of data writes

kafka-console-producer.sh  --bootstrap-server localhost:9092 --topic first-topic --producer-property acks=all

Nếu topic chưa được tạo trước đó thì sao?

kafka-console-producer.sh  --bootstrap-server localhost:9092 --topic new-topic

Nếu topic chưa được tạo trước đó mà producer vẫn tạo lệnh write vào new topic đó. thì kafka sẽ văng warning: WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {new-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) và thực create topic đó

kafka-topics.sh --bootstrap-server localhost:9092 --list

chúng show thông tin của topic “new-topic” sem có j cool

kafka-topics.sh  --bootstrap-server localhost:9092 --topic new-topic --describe

Nếu bạn muốn thay đôi số partition đối với hành vi tạo tự động các topic thì

grep -v "^#" /root/kafka/config/server.properties