Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Ingress
    • Helm Chart
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
    • MetalLB
    • Kong Gateway
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log & Monitor
    • DataDog
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
    • NextCloud
  • Toggle search form

Category: Apache Kafka

Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings

Posted on December 21, 2022 By nim No Comments on Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings
Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings

https://blog.developer.adobe.com/wins-from-effective-kafka-monitoring-at-adobe-stability-performance-and-cost-savings-a3ecb701ee5b Co-author: Eugen Nicolae Cojan As the father of quality management W. Edwards Deming, put it: “Without data, you are just another person with an opinion.” This post combines real-world issues faced by our team and our approach to addressing them. It also shares the key metrics to start monitoring Kafka client implementations effectively. Case studies…

Read More “Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings” »

Kafka

[Kafka/MSK] Install kafka or MSK on aws through terraform.

Posted on November 2, 2022November 3, 2022 By nim 2 Comments on [Kafka/MSK] Install kafka or MSK on aws through terraform.
[Kafka/MSK] Install kafka or MSK on aws through terraform.

1) Creating VPC Đầu tiên mình sẽ tạo vpc 2) Creating Security Group. 3) Declare variable and Create Kafka by terraform on AWS Tiếp đến là tạo msk hay còn gọi là kafka on aws Nếu bạn bị lỗi bên dưới Bạn phải khai báo:client_authentication_unauthenticated_enabled = true bạn cần chú ý số number_of_nodes nó phải…

Read More “[Kafka/MSK] Install kafka or MSK on aws through terraform.” »

AWS - Amazon Web Service, Kafka

[Kafka] How many partitions are enough in a topic on Kafka?

Posted on October 3, 2022October 3, 2022 By nim No Comments on [Kafka] How many partitions are enough in a topic on Kafka?
[Kafka] How many partitions are enough in a topic on Kafka?

Chúng ta cần set bao nhiều partion là phù hợp. Số partition không vượt quá số tổng CPU của các node vật lý.Số partition bằng 2 x nodeSố partitions = số pod trên k8s

Kafka

[Lenses] Insert data or Push messages to Kafka via Lenses.

Posted on June 13, 2022 By nim No Comments on [Lenses] Insert data or Push messages to Kafka via Lenses.
[Lenses] Insert data or Push messages to Kafka via Lenses.

Bên mình sài lenses thì hôm này code 1 tool tự động reset các connector bị lỗi:API reference:https://api.lenses.io/ https://lenses.io/blog/2019/11/four-ways-to-produce-data-kafka-lenses/

Lenses

[Kafka-connect] Solving the error or issues on Kafka-connect

Posted on April 28, 2022April 28, 2022 By nim No Comments on [Kafka-connect] Solving the error or issues on Kafka-connect
[Kafka-connect] Solving the error or issues on Kafka-connect

Với các connector của debezium bạn có thể tham khảo các issues ở đây:https://issues.redhat.com/projects/DBZ/issues/DBZ-5037?filter=allopenissues Và thường chúng ta sẽ lọc theo version. mysqldebezium Nếu bị lỗi trên thì change value: database.history.kafka.topic (thay đổi ký tự chẳng hạn)

Kafka Connect

[Kafka] Thiết kế và Testing Cluster kafka HA trên k8

Posted on April 28, 2022April 28, 2022 By nim No Comments on [Kafka] Thiết kế và Testing Cluster kafka HA trên k8
[Kafka] Thiết kế và Testing Cluster kafka HA trên k8

Lượn được bài hay về kafka của một anh

Kafka

[Kafka-connect] Reset Connector in Kafka Connect

Posted on April 3, 2022April 3, 2022 By nim No Comments on [Kafka-connect] Reset Connector in Kafka Connect
[Kafka-connect] Reset Connector in Kafka Connect

Hello các bạn sau đâu mình sẽ trình bày về vần để reset 1 connector trên kafka. Why need you reset a connector in Kafka-connect?Vì 1 số case thì kafka-connect sẽ read data từ Database rồi write data vào trong 1 topic trên kafka.Với Mysql thì thì kafka-connect sẽ đọc log bin, với Mongodb thì…

Read More “[Kafka-connect] Reset Connector in Kafka Connect” »

Kafka Connect

[Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb

Posted on April 1, 2022April 7, 2022 By nim No Comments on [Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb
[Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb

Mình đã có 1 bài hướng dẫn các bạn tạo Connector Source CDC từ read data từ 1 database và 1 collection thuộc 1 Mongodb và write vào topic. Ta để ý value của key sẽ thay đổi liên tục. 1st: Create Document A –(read data change)–> Kafka-connect –(write topic)–> Kafka –> 1 message có…

Read More “[Kafka-Connect] Overwrite value of Key on Mongo Source Connector – Mongodb” »

Kafka Connect

[Lenses/kafka] Configure authentication and authorization for lenses

Posted on March 11, 2022June 2, 2022 By nim No Comments on [Lenses/kafka] Configure authentication and authorization for lenses
[Lenses/kafka] Configure authentication and authorization for lenses

Mặc định pass là admin/adminGiờ chúng ta muốn change pass admin thì làm sao. 1) Configure authentication Reference links:https://docs.lenses.io/3.0/install_setup/configuration/security.html Lenses Administrator A default account with administrator privileges is always active. The default credentials are admin / admin and if left at default, a notification will show in the user interface, informing that the setup is insecure. The default account username…

Read More “[Lenses/kafka] Configure authentication and authorization for lenses” »

Lenses

[kafka] Topic doesn’t have the record.

Posted on March 3, 2022 By nim No Comments on [kafka] Topic doesn’t have the record.
[kafka] Topic doesn’t have the record.

Vào một ngày đẹp trời!Bạn vào lại topic của mình và thấy. VALUE:“No records were returned for 20 seconds. The max idle timeout has been reached. Use [SET SessionKey(max.idle.time)=’60s’;] to control it. You can specify the timeout in ms(milliseconds), s(seconds), m(minutes), h(hours).”

Kafka

Posts navigation

1 2 … 4 Next
Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • Experiences for IP Addresses Shortage on EKS Clusters March 29, 2023
  • [Talisman] Discover the sensitive information in your code. March 28, 2023
  • [Prometheus/Grafana] Install Prometheus and Grafana on ubuntu. March 27, 2023
  • [Kong Gateway] WebSocket connection failed March 26, 2023
  • [Nextcloud] Can’t download files to have a size bigger than 2Gi on NextCloud – RaspBerry March 24, 2023

Archives

  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • BareMetal
    • NextCloud
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • Golang
    • Jquery & JavaScript
    • Laravel
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kong Gateway
    • Kubernetes
      • Ingress
    • Longhorn – Storage
    • MetalLB
    • Vault
    • VictoriaMetrics
  • Log & Monitor
    • DataDog
    • ELK
      • Kibana
      • Logstash
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2023 NimTechnology.