Producers
• Producers write data to topics (which is made of partitions)
• Producers automatically know to which broker and partition to write to
• In case of Broker failures, Producers will automatically recover
• Producers can choose to receive acknowledgment of data writes:
• acks=0: Producer won’t wait for acknowledgment (possible data loss)
• acks=1: Producer will wait for leader acknowledgment (limited data loss)
• acks=all: Leader + replicas acknowledgment (no data loss)
Producers: Message keys
- Producers can choose to send a key with the message (string, number, etc..)
- If key=null, data is sent round robin (broker 101 then 102 then 103…)
- If a key is sent, then all messages for that key will always go to the same partition
- A key is basically sent if you need message ordering for a specific field (ex: truck_id)
kafka-console-producer.sh
chúng ta list ra xem chúng ta đã tạo topics nào trước đó?
kafka-topics.sh --bootstrap-server localhost:9092 --list
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first-topic
Chọn receive acknowledgment of data writes
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic first-topic --producer-property acks=all
Nếu topic chưa được tạo trước đó thì sao?
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic new-topicNếu topic chưa được tạo trước đó mà producer vẫn tạo lệnh write vào new topic đó. thì kafka sẽ văng warning: WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {new-topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient) và thực create topic đó
kafka-topics.sh --bootstrap-server localhost:9092 --list
chúng show thông tin của topic “new-topic” sem có j cool
kafka-topics.sh --bootstrap-server localhost:9092 --topic new-topic --describe
Nếu bạn muốn thay đôi số partition đối với hành vi tạo tự động các topic thì
grep -v "^#" /root/kafka/config/server.properties
