Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Gateway API
      • Ingress
      • Pod
    • Helm Chart
    • OAuth2 Proxy
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
    • MetalLB
    • Kong Gateway
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Argo Events
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • DevSecOps
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
      • Azure Cloud
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log, Monitor & Tracing
    • DataDog
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
    • NextCloud
  • Toggle search form

Category: Apache Kafka

[Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…

Posted on February 26, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…
[Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. reference link:https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day8.adoc Tạo the data generatorhttps://www.confluent.io/hub/mdrogalis/voluble Giờ chúng ta kiểm tra data trong topic và chính xác là field:…

Read More “[Kafka-connect] Single Message Transform: lesson 8 – TimestampConverter – convert the type of the fields from string to timestamp or date,…” »

Apache Kafka, Kafka Connect

[Kafka-connect] Single Message Transform: lesson 7 – TimeStampRouter and MessageTimestampRouter – Custom format topic name with timestamp

Posted on February 26, 2022February 26, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 7 – TimeStampRouter and MessageTimestampRouter – Custom format topic name with timestamp
[Kafka-connect] Single Message Transform: lesson 7 – TimeStampRouter and MessageTimestampRouter – Custom format topic name with timestamp

Bài này thì anh Tây chỉ cánh chỉnh topic name trong source và table name trong sink bằng cách kêt với timestamp. https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day7.adoc 1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose…

Read More “[Kafka-connect] Single Message Transform: lesson 7 – TimeStampRouter and MessageTimestampRouter – Custom format topic name with timestamp” »

Apache Kafka

[Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the process data.

Posted on February 26, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the process data.
[Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the  process data.

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. Bài này kiểu như bạn thêm số field mà trong quá trình kafka-connect stream data.Ví dụ trong stream data…

Read More “[Kafka-connect] Single Message Transform: lesson 6 InsertField – Insert fields using attributes in the process data.” »

Apache Kafka, Kafka Connect

[Kafka-connect] Single Message Transform: lesson 5 MaskField – Cover the sensitive data.

Posted on February 26, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 5 MaskField – Cover the sensitive data.
[Kafka-connect] Single Message Transform: lesson 5 MaskField – Cover the sensitive data.

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. Tạo the data generatorhttps://www.confluent.io/hub/mdrogalis/voluble Mình đặt ra một trường hợp, Bạn sử dụng kafka-connect để stream table trong database….

Read More “[Kafka-connect] Single Message Transform: lesson 5 MaskField – Cover the sensitive data.” »

Apache Kafka, Kafka Connect

[Kafka-connect] Single Message Transform: lesson 4 – RegexRouter – change topic name.

Posted on February 24, 2022February 26, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 4 – RegexRouter – change topic name.
[Kafka-connect] Single Message Transform: lesson 4 – RegexRouter – change topic name.

1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka Bạn nhớ delele container cũ và run lại docker-compose nhé:docker-compose down -vdocker-compose up -d 2) Practice. Tạo the data generatorhttps://www.confluent.io/hub/mdrogalis/voluble Giờ bạn thấy nó tạo ra 1 topic: day4-transactions Giờ ta tạo 1 sink connector:…

Read More “[Kafka-connect] Single Message Transform: lesson 4 – RegexRouter – change topic name.” »

Apache Kafka

[Lenses/kafka] Fix the problem “Cannot extract connector information from the configuration provided” when creating connector Kafka-connect

Posted on February 23, 2022April 3, 2022 By nim No Comments on [Lenses/kafka] Fix the problem “Cannot extract connector information from the configuration provided” when creating connector Kafka-connect
[Lenses/kafka] Fix the problem “Cannot extract connector information from the configuration provided” when creating connector Kafka-connect

Nếu bạn sửa dụng kafka-connect mà sẽ cần add thêm rất nhiều plugin vào server kafka-connect.Mình đang gặp vấn đề có 1 vài plugin lenses ko thể validate được. Ví dụ như plugin này: MongoDB Connector (Source and Sink) khi mình create 1 connector có hiện tượng sau: Có 1 người anh chỉ mình:https://docs.lenses.io/5.0/configuration/static/options/topology/#configure-lenses-to-load-a-custom-connector chúng…

Read More “[Lenses/kafka] Fix the problem “Cannot extract connector information from the configuration provided” when creating connector Kafka-connect” »

Apache Kafka, Lenses

[Kafka] UI control Kafka, Kafka-connect, … It’s akhq.io

Posted on February 23, 2022November 15, 2024 By nim No Comments on [Kafka] UI control Kafka, Kafka-connect, … It’s akhq.io
[Kafka] UI control Kafka, Kafka-connect, … It’s akhq.io

Trong các bài viết về kafka connect của mình, thì mình hay sử dụng lenses.Nhưng có vấn đề lenses tốn phí. Mình đã được 1 bạn trong group giới thiệu Mình có dò thì conduktor nó tốn phí còn akhq thì kocảm ơn bạn rất nhiều Mình còn thấy 1 ứng cử viên tiềm năng…

Read More “[Kafka] UI control Kafka, Kafka-connect, … It’s akhq.io” »

Apache Kafka, Lenses

[Kafka-connect] Streaming the data of MongoDB through Kafka-connect into topic Kafka.

Posted on February 23, 2022May 22, 2022 By nim No Comments on [Kafka-connect] Streaming the data of MongoDB through Kafka-connect into topic Kafka.
[Kafka-connect] Streaming the data of MongoDB through Kafka-connect into topic Kafka.

1) Install MongoDB Nếu bạn chưa rành về mongoDB có thể sem vài chiêu thức cơ bản ở đây: Reference links:https://gist.github.com/asoorm/7822cc742831639c93affd734e97ce4f Mình cung cấp link docker-compose.yaml như sau: Giờ bạn run:docker-compose up -d thực hiện hiện exec vào container và tạo database, collection và insert data. Hiện tại thì chúng ta chạy mongodb ở mode…

Read More “[Kafka-connect] Streaming the data of MongoDB through Kafka-connect into topic Kafka.” »

Apache Kafka, Kafka Connect

[Kafka-connect]Single Message Transform: lesson 3 – Flatten and LongConverter

Posted on February 21, 2022February 27, 2022 By nim No Comments on [Kafka-connect]Single Message Transform: lesson 3 – Flatten and LongConverter
[Kafka-connect]Single Message Transform: lesson 3 – Flatten and LongConverter

Khi data trong 1 topic ở format Avro,… mà ta muốn sink vào DB thì nó sẽ bị lỗi ở việc tạo field column cho 1 table. 1) Setup the components. Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka 2) Practice. Launch ksqlDB: OK giờ bạn thấy…

Read More “[Kafka-connect]Single Message Transform: lesson 3 – Flatten and LongConverter” »

Apache Kafka, Kafka Connect

[Kafka-connect] Single Message Transform: lesson 2 – ValueToKey and ExtractField in Sink

Posted on February 20, 2022February 27, 2022 By nim No Comments on [Kafka-connect] Single Message Transform: lesson 2 – ValueToKey and ExtractField in Sink
[Kafka-connect] Single Message Transform: lesson 2 – ValueToKey and ExtractField in Sink

1) Setup Phần setup kafka, bạn có thể coi lại link này he:Setup the needing components of Kafka 2) Practice giờ chui vào mysql nghịch: tạo table và insert data. Giờ chúng ta đã có table mới: Create source connector reference links:https://github.com/confluentinc/demo-scene/blob/master/kafka-connect-single-message-transforms/day2.adochttps://docs.confluent.io/current/connect/kafka-connect-jdbc/source-connector/Kafka Connect Deep Dive – JDBC Source Connector Giờ bạn run command để tạo…

Read More “[Kafka-connect] Single Message Transform: lesson 2 – ValueToKey and ExtractField in Sink” »

Apache Kafka, Kafka Connect

Posts pagination

Previous 1 2 3 4 5 Next
Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • [Rancher/EKS] Rancher from v2.12.x can not work on eks cluster. April 15, 2026
  • [Telegram/Openclaw] Configure openclaw bot in a Telegram group. March 31, 2026
  • Tutorial: Gateway API + Traefik + oauth2-proxy (Microsoft Entra ID) March 30, 2026
  • Full + incremental backup: When restoring, do deleted files come back? March 27, 2026
  • [K8S] Create long-lived kubeconfig on k8s March 23, 2026

Archives

  • April 2026
  • March 2026
  • February 2026
  • January 2026
  • December 2025
  • November 2025
  • October 2025
  • September 2025
  • August 2025
  • July 2025
  • June 2025
  • May 2025
  • April 2025
  • March 2025
  • February 2025
  • January 2025
  • December 2024
  • November 2024
  • October 2024
  • September 2024
  • August 2024
  • July 2024
  • June 2024
  • May 2024
  • April 2024
  • March 2024
  • February 2024
  • January 2024
  • December 2023
  • November 2023
  • October 2023
  • September 2023
  • August 2023
  • July 2023
  • June 2023
  • May 2023
  • April 2023
  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • AI
    • OpenClaw
  • BareMetal
    • NextCloud
  • CI/CD
    • Argo Events
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • DevSecOps
    • Golang
    • Jquery & JavaScript
    • Laravel
    • NextJS 14 & ReactJS & Type Script
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • Azure Cloud
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kong Gateway
    • Kubernetes
      • Gateway API
      • Ingress
      • Pod
    • Longhorn – Storage
    • MetalLB
    • OAuth2 Proxy
    • Vault
    • VictoriaMetrics
  • Log, Monitor & Tracing
    • DataDog
    • ELK
      • Kibana
      • Logstash
    • Fluent
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2026 NimTechnology.