Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Ingress
    • Helm Chart
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
    • MetalLB
    • Kong Gateway
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log & Monitor
    • DataDog
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
    • NextCloud
  • Toggle search form

Category: Kafka

Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings

Posted on December 21, 2022 By nim No Comments on Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings
Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings

https://blog.developer.adobe.com/wins-from-effective-kafka-monitoring-at-adobe-stability-performance-and-cost-savings-a3ecb701ee5b Co-author: Eugen Nicolae Cojan As the father of quality management W. Edwards Deming, put it: “Without data, you are just another person with an opinion.” This post combines real-world issues faced by our team and our approach to addressing them. It also shares the key metrics to start monitoring Kafka client implementations effectively. Case studies…

Read More “Wins from Effective Kafka Monitoring at Adobe: Stability, Performance, and Cost Savings” »

Kafka

[Kafka/MSK] Install kafka or MSK on aws through terraform.

Posted on November 2, 2022November 3, 2022 By nim 2 Comments on [Kafka/MSK] Install kafka or MSK on aws through terraform.
[Kafka/MSK] Install kafka or MSK on aws through terraform.

1) Creating VPC Đầu tiên mình sẽ tạo vpc 2) Creating Security Group. 3) Declare variable and Create Kafka by terraform on AWS Tiếp đến là tạo msk hay còn gọi là kafka on aws Nếu bạn bị lỗi bên dưới Bạn phải khai báo:client_authentication_unauthenticated_enabled = true bạn cần chú ý số number_of_nodes nó phải…

Read More “[Kafka/MSK] Install kafka or MSK on aws through terraform.” »

AWS - Amazon Web Service, Kafka

[Kafka] How many partitions are enough in a topic on Kafka?

Posted on October 3, 2022October 3, 2022 By nim No Comments on [Kafka] How many partitions are enough in a topic on Kafka?
[Kafka] How many partitions are enough in a topic on Kafka?

Chúng ta cần set bao nhiều partion là phù hợp. Số partition không vượt quá số tổng CPU của các node vật lý.Số partition bằng 2 x nodeSố partitions = số pod trên k8s

Kafka

[Kafka] Thiết kế và Testing Cluster kafka HA trên k8

Posted on April 28, 2022April 28, 2022 By nim No Comments on [Kafka] Thiết kế và Testing Cluster kafka HA trên k8
[Kafka] Thiết kế và Testing Cluster kafka HA trên k8

Lượn được bài hay về kafka của một anh

Kafka

[kafka] Topic doesn’t have the record.

Posted on March 3, 2022 By nim No Comments on [kafka] Topic doesn’t have the record.
[kafka] Topic doesn’t have the record.

Vào một ngày đẹp trời!Bạn vào lại topic của mình và thấy. VALUE:“No records were returned for 20 seconds. The max idle timeout has been reached. Use [SET SessionKey(max.idle.time)=’60s’;] to control it. You can specify the timeout in ms(milliseconds), s(seconds), m(minutes), h(hours).”

Kafka

[Kafka] Install kafka and zookeeper cluster on kubernetes.

Posted on February 13, 2022April 3, 2022 By nim No Comments on [Kafka] Install kafka and zookeeper cluster on kubernetes.
[Kafka] Install kafka and zookeeper cluster on kubernetes.

Mới đây mình research ra các cái kafka trên k8s khá tiệnhttps://docs.bitnami.com/tutorials/deploy-scalable-kafka-zookeeper-cluster-kubernetes/ Link helm:https://charts.bitnami.com/bitnami Nó gồm 1 phần: – Install zookeeper. – Install Kafka. 1) Install zookeeper values.yaml Nếu bạn có sài lọnghorn thì ko cần quan tâm đến pvc. Khoẻ re. 2) Install kafka on kubernetes. values.yaml Bạn để ý là mình lựu chọn…

Read More “[Kafka] Install kafka and zookeeper cluster on kubernetes.” »

Apache Kafka, Kafka

[Kafka] Console Consumer CLI and Consumers in Group

Posted on January 31, 2022February 27, 2022 By nim No Comments on [Kafka] Console Consumer CLI and Consumers in Group
[Kafka] Console Consumer CLI and Consumers in Group

Giờ chúng ta tìm hiểu về việc read và write vào 1 topic trên kafka bằng CLI 1) Overview 1.1) Consumers Consumers read data from a topic (identified by name) Consumers know which broker to read from In case of broker failures, consumers know how to recover Data is read in order within each partitions…

Read More “[Kafka] Console Consumer CLI and Consumers in Group” »

Apache Kafka, Kafka

[Kafka] Kafka Console Producer CLI.

Posted on January 31, 2022February 27, 2022 By nim No Comments on [Kafka] Kafka Console Producer CLI.
[Kafka] Kafka Console Producer CLI.

Producers • Producers write data to topics (which is made of partitions) • Producers automatically know to which broker and partition to write to • In case of Broker failures, Producers will automatically recover • Producers can choose to receive acknowledgment of data writes: • acks=0: Producer won’t wait for acknowledgment (possible data loss) •…

Read More “[Kafka] Kafka Console Producer CLI.” »

Apache Kafka, Kafka, Log & Monitor

[Kafka] Kafka Topics CLI

Posted on January 31, 2022October 7, 2022 By nim No Comments on [Kafka] Kafka Topics CLI
[Kafka] Kafka Topics CLI

Ở bài này mình đi tìm hiểu về Kafka CLI, để chạy được kafka bạn tham khảo bài này nhé: 1) kafka-topics 1.1) what is the topic in Kfaka? Vậy Topic trong kafka là gì? Topics: a particular stream of data Similar to a table in a database (without all the constraints) You can have…

Read More “[Kafka] Kafka Topics CLI” »

Apache Kafka, BareMetal, Kafka

[Kafka-Zookeeper] Starting Kafka and Zookeeper

Posted on January 30, 2022February 27, 2022 By nim No Comments on [Kafka-Zookeeper] Starting Kafka and Zookeeper
[Kafka-Zookeeper] Starting Kafka and Zookeeper

Hiện tại mình đang tìm hiểu về Kafka and ZookeeperBài này chỉ đơn thuần là start 2 service này lên thôi! Install Java 11 Giờ chúng ta download kafka. https://kafka.apache.org/downloads Bạn cần add cái này vào cuối fileexport PATH=/root/kafka/bin:$PATH sau đó chúng ta mở một cưa số khác,rồi gõ kafka sau đó nhấn tab, nếu…

Read More “[Kafka-Zookeeper] Starting Kafka and Zookeeper” »

Apache Kafka, BareMetal, Kafka
Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • [Prometheus/Grafana] Install Prometheus and Grafana on ubuntu. March 27, 2023
  • [Kong Gateway] WebSocket connection failed March 26, 2023
  • [Nextcloud] Can’t download files to have a size bigger than 2Gi on NextCloud – RaspBerry March 24, 2023
  • [Datadog] Using DataDog to monitor all services on kubernetes March 19, 2023
  • [Metrics Server] Failed to make webhook authorizer request: the server could not find the requested resource March 17, 2023

Archives

  • March 2023
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • BareMetal
    • NextCloud
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • Golang
    • Jquery & JavaScript
    • Laravel
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kong Gateway
    • Kubernetes
      • Ingress
    • Longhorn – Storage
    • MetalLB
    • Vault
    • VictoriaMetrics
  • Log & Monitor
    • DataDog
    • ELK
      • Kibana
      • Logstash
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2023 NimTechnology.