Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Ingress
      • Pod
    • Helm Chart
    • OAuth2 Proxy
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
    • MetalLB
    • Kong Gateway
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Argo Events
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • DevSecOps
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
      • Azure Cloud
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log, Monitor & Tracing
    • DataDog
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
    • NextCloud
  • Toggle search form

Category: Kafka Connect

IBM MQ -> RabbitMQ -> Kafka ->Pulsar: How do message queue architectures evolve?

Posted on October 10, 2023 By nim No Comments on IBM MQ -> RabbitMQ -> Kafka ->Pulsar: How do message queue architectures evolve?
IBM MQ -> RabbitMQ -> Kafka ->Pulsar: How do message queue architectures evolve?

Nguồn: https://blog.bytebytego.com/i/137744340/ibm-mq-rabbitmq-kafka-pulsar-how-do-message-queue-architectures-evolve IBM MQ IBM MQ được ra mắt vào năm 1993. Ban đầu nó được gọi là MQSeries và được đổi tên thành WebSphere MQ vào năm 2002. Nó được đổi tên thành IBM MQ vào năm 2014. IBM MQ là một sản phẩm rất thành công, được sử dụng rộng rãi trong ngành…

Read More “IBM MQ -> RabbitMQ -> Kafka ->Pulsar: How do message queue architectures evolve?” »

Kafka Connect

[Kafka-connect] Solving the error or issues on Kafka-connect

Posted on April 28, 2022April 28, 2022 By nim No Comments on [Kafka-connect] Solving the error or issues on Kafka-connect
[Kafka-connect] Solving the error or issues on Kafka-connect

Với các connector của debezium bạn có thể tham khảo các issues ở đây:https://issues.redhat.com/projects/DBZ/issues/DBZ-5037?filter=allopenissues Và thường chúng ta sẽ lọc theo version. mysqldebezium Nếu bị lỗi trên thì change value: database.history.kafka.topic (thay đổi ký tự chẳng hạn)

Kafka Connect

[Kafka-connect] Reset Connector in Kafka Connect

Posted on April 3, 2022April 3, 2022 By nim No Comments on [Kafka-connect] Reset Connector in Kafka Connect
[Kafka-connect] Reset Connector in Kafka Connect

Hello các bạn sau đâu mình sẽ trình bày về vần để reset 1 connector trên kafka. Why need you reset a connector in Kafka-connect?Vì 1 số case thì kafka-connect sẽ read data từ Database rồi write data vào trong 1 topic trên kafka.Với Mysql thì thì kafka-connect sẽ đọc log bin, với Mongodb thì…

Read More “[Kafka-connect] Reset Connector in Kafka Connect” »

Kafka Connect

[Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb

Posted on April 1, 2022April 7, 2022 By nim No Comments on [Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb
[Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb

Mình đã có 1 bài hướng dẫn các bạn tạo Connector Source CDC từ read data từ 1 database và 1 collection thuộc 1 Mongodb và write vào topic. Ta để ý value của key sẽ thay đổi liên tục. 1st: Create Document A –(read data change)–> Kafka-connect –(write topic)–> Kafka –> 1 message có…

Read More “[Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb” »

Kafka Connect

[Kafka-connect] Single Message Transform: lesson 12 – Community Transformations

Posted on February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 12 – Community Transformations
[Kafka-connect] Single Message Transform: lesson 12 – Community Transformations

Apache Kafka ships with many Single Message Transformations included – but the great thing about it being an open API is that people can, and do, write their own transformations. Many of these are shared with the wider community, and in this final installment of the series I’m going to look at some of the transformations written by Jeremy…

Read More “[Kafka-connect] Single Message Transform: lesson 12 – Community Transformations” »

Kafka Connect

[Kafka-connect] Single Message Transform: lesson 11 – Predicate and Filter – filtering topic name and content of the messages.

Posted on February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 11 – Predicate and Filter – filtering topic name and content of the messages.
[Kafka-connect] Single Message Transform: lesson 11 – Predicate and Filter – filtering topic name and content of the messages.

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. reference link:https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day11.adoc Với connector source-voluble-datagen-day11-00 thì nó sẽ sinh ra 2 topic:– day11-sys01– day11-systemBOK bạn cần nhớ điều này…

Read More “[Kafka-connect] Single Message Transform: lesson 11 – Predicate and Filter – filtering topic name and content of the messages.” »

Kafka Connect

[Kafka-connect] Single Message Transform: lesson 10 – ReplaceField – U can drop, keep and renamed field in kafka message.

Posted on February 27, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 10 – ReplaceField – U can drop, keep and renamed field in kafka message.
[Kafka-connect] Single Message Transform: lesson 10 – ReplaceField – U can drop, keep and renamed field in kafka message.

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. reference link:https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day10.adoc Dropping fields in a sink connector: [Kafka-connect] Single Message Transform: lesson 5 MaskFieldỞ bài 5 chúng ta…

Read More “[Kafka-connect] Single Message Transform: lesson 10 – ReplaceField – U can drop, keep and renamed field in kafka message.” »

Kafka Connect

[Kafka-connect] Single Message Transform: lesson 9 – Cast

Posted on February 27, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 9 – Cast
[Kafka-connect] Single Message Transform: lesson 9 – Cast

Bài này chúng ta sẽ chỉ custom type data khi nào source connector 1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. reference link:https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day9.adoc Bạn thấy kết quả…

Read More “[Kafka-connect] Single Message Transform: lesson 9 – Cast” »

Apache Kafka, Kafka Connect

[Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…

Posted on February 26, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…
[Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. reference link:https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day8.adoc Tạo the data generatorhttps://www.confluent.io/hub/mdrogalis/voluble Giờ chúng ta kiểm tra data trong topic và chính xác là field:…

Read More “[Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…” »

Apache Kafka, Kafka Connect

[Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the process data.

Posted on February 26, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the process data.
[Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the  process data.

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. Bài này kiểu như bạn thêm số field mà trong quá trình kafka-connect stream data.Ví dụ trong stream data…

Read More “[Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the process data.” »

Apache Kafka, Kafka Connect

Posts pagination

1 2 3 Next
Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • [Argo Workflow] Create an access token for Argo Workflows July 14, 2025
  • [Argo Workflow] SSO Authentication for Argo Workflows. July 14, 2025
  • [AWS/EKS] Cache Docker image to accelerate EKS container deployment. July 10, 2025
  • [Laravel] Laravel Helpful June 26, 2025
  • [VScode] Hướng dẫn điều chỉnh font cho terminal June 20, 2025

Archives

  • July 2025
  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • BareMetal
    • NextCloud
  • CI/CD
    • Argo Events
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • DevSecOps
    • Golang
    • Jquery & JavaScript
    • Laravel
    • NextJS 14 & ReactJS & Type Script
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • Azure Cloud
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kong Gateway
    • Kubernetes
      • Ingress
      • Pod
    • Longhorn – Storage
    • MetalLB
    • OAuth2 Proxy
    • Vault
    • VictoriaMetrics
  • Log, Monitor & Tracing
    • DataDog
    • ELK
      • Kibana
      • Logstash
    • Fluent
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2025 NimTechnology.