Category: Kafka

Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings

Posted on By nim No Comments on Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings

https://blog.developer.adobe.com/wins-from-effective-kafka-monitoring-at-adobe-stability-performance-and-cost-savings-a3ecb701ee5b Co-author: Eugen Nicolae Cojan As the father of quality management W. Edwards Deming, put it: “Without data, you are just another person with an opinion.” This post combines real-world issues faced by our team and our approach to addressing them. It also shares the key metrics to start monitoring Kafka client implementations effectively. Case studies…

Read More “Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings” »

Kafka

[Kafka/MSK] Install kafka or MSK on aws through terraform.

Posted on By nim 2 Comments on [Kafka/MSK] Install kafka or MSK on aws through terraform.

1) Creating VPC Đầu tiên mình sẽ tạo vpc 2) Creating Security Group. 3) Declare variable and Create Kafka by terraform on AWS Tiếp đến là tạo msk hay còn gọi là kafka on aws Nếu bạn bị lỗi bên dưới Bạn phải khai báo:client_authentication_unauthenticated_enabled = true bạn cần chú ý số number_of_nodes nó phải…

Read More “[Kafka/MSK] Install kafka or MSK on aws through terraform.” »

AWS - Amazon Web Service, Kafka

[Kafka] Install kafka and zookeeper cluster on kubernetes.

Posted on By nim No Comments on [Kafka] Install kafka and zookeeper cluster on kubernetes.

Mới đây mình research ra các cái kafka trên k8s khá tiệnhttps://docs.bitnami.com/tutorials/deploy-scalable-kafka-zookeeper-cluster-kubernetes/ Link helm:https://charts.bitnami.com/bitnami Nó gồm 1 phần: – Install zookeeper. – Install Kafka. 1) Install zookeeper values.yaml Nếu bạn có sài lọnghorn thì ko cần quan tâm đến pvc. Khoẻ re. 2) Install kafka on kubernetes. values.yaml Bạn để ý là mình lựu chọn…

Read More “[Kafka] Install kafka and zookeeper cluster on kubernetes.” »

Apache Kafka, Kafka

[Kafka] Console Consumer CLI and Consumers in Group

Posted on By nim No Comments on [Kafka] Console Consumer CLI and Consumers in Group

Giờ chúng ta tìm hiểu về việc read và write vào 1 topic trên kafka bằng CLI 1) Overview 1.1) Consumers Consumers read data from a topic (identified by name) Consumers know which broker to read from In case of broker failures, consumers know how to recover Data is read in order within each partitions…

Read More “[Kafka] Console Consumer CLI and Consumers in Group” »

Apache Kafka, Kafka

[Kafka] Kafka Console Producer CLI.

Posted on By nim No Comments on [Kafka] Kafka Console Producer CLI.

Producers • Producers write data to topics (which is made of partitions) • Producers automatically know to which broker and partition to write to • In case of Broker failures, Producers will automatically recover • Producers can choose to receive acknowledgment of data writes: • acks=0: Producer won’t wait for acknowledgment (possible data loss) •…

Read More “[Kafka] Kafka Console Producer CLI.” »

Apache Kafka, Kafka, Log & Monitor

[Kafka] Kafka Topics CLI

Posted on By nim No Comments on [Kafka] Kafka Topics CLI

Ở bài này mình đi tìm hiểu về Kafka CLI, để chạy được kafka bạn tham khảo bài này nhé: 1) kafka-topics 1.1) what is the topic in Kfaka? Vậy Topic trong kafka là gì? Topics: a particular stream of data Similar to a table in a database (without all the constraints) You can have…

Read More “[Kafka] Kafka Topics CLI” »

Apache Kafka, BareMetal, Kafka

[Kafka-Zookeeper] Starting Kafka and Zookeeper

Posted on By nim No Comments on [Kafka-Zookeeper] Starting Kafka and Zookeeper

Hiện tại mình đang tìm hiểu về Kafka and ZookeeperBài này chỉ đơn thuần là start 2 service này lên thôi! Install Java 11 Giờ chúng ta download kafka. https://kafka.apache.org/downloads Bạn cần add cái này vào cuối fileexport PATH=/root/kafka/bin:$PATH sau đó chúng ta mở một cưa số khác,rồi gõ kafka sau đó nhấn tab, nếu…

Read More “[Kafka-Zookeeper] Starting Kafka and Zookeeper” »

Apache Kafka, BareMetal, Kafka