Category: Kubernetes & Container

[Kafka-connect] Streaming the data of MySQL throughs Kafka-connect and Debezium plugin.

Posted on By nim No Comments on [Kafka-connect] Streaming the data of MySQL throughs Kafka-connect and Debezium plugin.
[Kafka-connect] Streaming the data of MySQL throughs Kafka-connect and Debezium plugin.

1) Install and configurate mysql to use to the kafka-connect Lab. Run docker MySQL: giờ sửa file: mysql.cnf xong rồi thoát ra restart docker In order for Debezium MySQL to track all the changes in bin-log, it needs a user with SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT permissions. Grant them to the user…

Read More “[Kafka-connect] Streaming the data of MySQL throughs Kafka-connect and Debezium plugin.” »

Apache Kafka, Kafka Connect

[Kafka] Install kafka and zookeeper cluster on kubernetes.

Posted on By nim No Comments on [Kafka] Install kafka and zookeeper cluster on kubernetes.
[Kafka] Install kafka and zookeeper cluster on kubernetes.

Mới đây mình research ra các cái kafka trên k8s khá tiệnhttps://docs.bitnami.com/tutorials/deploy-scalable-kafka-zookeeper-cluster-kubernetes/ Link helm:https://charts.bitnami.com/bitnami Nó gồm 1 phần: – Install zookeeper. – Install Kafka. 1) Install zookeeper values.yaml Nếu bạn có sài lọnghorn thì ko cần quan tâm đến pvc. Khoẻ re. 2) Install kafka on kubernetes. values.yaml Bạn để ý là mình lựu chọn…

Read More “[Kafka] Install kafka and zookeeper cluster on kubernetes.” »

Apache Kafka, Kafka

[Kafka/Schema-registry] Installing Schema-registry to use for the Kafka and Kafka-connect model

Posted on By nim No Comments on [Kafka/Schema-registry] Installing Schema-registry to use for the Kafka and Kafka-connect model
[Kafka/Schema-registry] Installing Schema-registry to use for the Kafka and Kafka-connect model

1) what is the Schema-registry Hiện giờ mình mình cũng chưa hiểu rõ Schema-registry.Mình có 1 số link tham khảo:https://docs.confluent.io/platform/current/schema-registry/index.html#sr-overview A few words about Avro By default, Debezium writes the JSON-formatted data. This method is acceptable for sandboxes and small volumes of data but can be a problem in high-load databases. Avro is an excellent alternative to…

Read More “[Kafka/Schema-registry] Installing Schema-registry to use for the Kafka and Kafka-connect model” »

Apache Kafka

[Kafka-connect] Install Kafka-connect on Kubernetes through helm-chart.

Posted on By nim No Comments on [Kafka-connect] Install Kafka-connect on Kubernetes through helm-chart.
[Kafka-connect] Install Kafka-connect on Kubernetes through helm-chart.

1) Overview Mình đã có 1 vài bài về kafka cũng như kafka-connect.Kafka-connect thì có nhiệm vụ là Lấy data từ source nào đó như là: file, database, … ===> rồi ghi vào topic của KafkaKhi mà topic đó đã có data:– thì client có read trực tiếp data từ topic đó.– Hoặc sử dụng…

Read More “[Kafka-connect] Install Kafka-connect on Kubernetes through helm-chart.” »

Apache Kafka, Kafka Connect

[Kafka-connect] Install lenses to manage Kafka.

Posted on By nim No Comments on [Kafka-connect] Install lenses to manage Kafka.
[Kafka-connect] Install lenses to manage Kafka.

Như bạn thấy mình đã có những bài nói về comand trong kafka cũng như kafka-connectChúng sẽ có 1 UI để quản lý đó là lense, giao diện khá đẹp. 1) License Để run được lenses chúng ta cần có file là license.json 1.1) It’s free! chúng ta cần vào trang web này để đăng…

Read More “[Kafka-connect] Install lenses to manage Kafka.” »

Apache Kafka, Kafka Connect, Lenses

[wordpress] Install WordPress so easily.

Posted on By nim No Comments on [wordpress] Install WordPress so easily.
[wordpress] Install WordPress so easily.

đầu tiên cần chạy database trước: Tiêp theo có file docker-compose.yaml Sau đó bạn đăng nhập vào web với port 8084 và làm theo các hướng dẫn của wordpress How to increase Memory Limit of WordPress. Go to file wp-config.php Tăng nó lên 512MB How to Run Third-Party Schedules Mình có sài 1 FluentCRM để…

Read More “[wordpress] Install WordPress so easily.” »

Kubernetes & Container

[Kafka-connect] research on Kafka Connect Source and demo watch the changing file.

Posted on By nim No Comments on [Kafka-connect] research on Kafka Connect Source and demo watch the changing file.
[Kafka-connect] research on Kafka Connect Source and demo watch the changing file.

Bài này chúng ta sẽ tìm hiểu thử Kafka connect là cái quái j?Và có 1 vài demo để dễ hiểu hơn. Đầu tiên:Nếu bạn muốn có người cầm tay chỉ việc vip hơn mình thì down khoá này nhé:https://www.udemy.com/course/kafka-connect/ 1) Kafka Connect Source Hiệu đơn giản thì chúng ta sẽ lấy data từ 1…

Read More “[Kafka-connect] research on Kafka Connect Source and demo watch the changing file.” »

Apache Kafka, Kafka Connect

[Kafka] Console Consumer CLI and Consumers in Group

Posted on By nim No Comments on [Kafka] Console Consumer CLI and Consumers in Group
[Kafka] Console Consumer CLI and Consumers in Group

Giờ chúng ta tìm hiểu về việc read và write vào 1 topic trên kafka bằng CLI 1) Overview 1.1) Consumers 1.3) Consumer Groups What if too many consumers? giờ mình thực hiện mở 2 cửa sổ: – 1 consumer để lắng nghe topic– 1 producer để write data into topic 2) Practice 2.1) Consumer…

Read More “[Kafka] Console Consumer CLI and Consumers in Group” »

Apache Kafka, Kafka

[Kafka] Kafka Console Producer CLI.

Posted on By nim No Comments on [Kafka] Kafka Console Producer CLI.
[Kafka] Kafka Console Producer CLI.

Producers • Producers write data to topics (which is made of partitions) • Producers automatically know to which broker and partition to write to • In case of Broker failures, Producers will automatically recover • Producers can choose to receive acknowledgment of data writes: • acks=0: Producer won’t wait for acknowledgment (possible data loss) •…

Read More “[Kafka] Kafka Console Producer CLI.” »

Apache Kafka, Kafka, Log, Monitor & Tracing