Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Ingress
    • Helm Chart
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log & Monitor
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
  • Toggle search form

[Kafka] UI control Kafka, Kafka-connect, … It’s akhq.io

Posted on February 23, 2022December 13, 2022 By nim No Comments on [Kafka] UI control Kafka, Kafka-connect, … It’s akhq.io

Trong các bài viết về kafka connect của mình, thì mình hay sử dụng lenses.
Nhưng có vấn đề lenses tốn phí.

Mình đã được 1 bạn trong group giới thiệu

Mình có dò thì conduktor nó tốn phí còn akhq thì ko
cảm ơn bạn rất nhiều

Contents

  • 1) Deploy akhq on k8s
  • 2) AKHQ integrate MSK or Kafka that has the authentication type “TLS”

1) Deploy akhq on k8s

Video về thanh niên này

Repo github
https://github.com/tchiotludo/akhq

trong bài họ đã hướng dẫn run bằng command docker òi he.

giờ mình deploy lên k8s:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    workload.user.cattle.io/workloadselector: deployment-kafka-kafka-akhq
  name: kafka-akhq
  namespace: kafka
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: deployment-kafka-kafka-akhq
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        workload.user.cattle.io/workloadselector: deployment-kafka-kafka-akhq
    spec:
      containers:
      - env:
        - name: AKHQ_CONFIGURATION
          value: |-
            akhq:
              connections:
                docker-kafka-server:
                  properties:
                    bootstrap.servers: "kafka:9092"
                  schema-registry:
                    url: "http://schema-registry-cp-schema-registry:8081"
                  connect:
                    - name: "kafka-connect"
                      url: "http://kafka-connect-cp-kafka-connect:8083"
        image: tchiotludo/akhq
        imagePullPolicy: Always
        name: kafka-akhq
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP

Docker composer.

docker-compose.yaml
version: '3.6'
services:
  akhq:
    # build:
    #   context: .
    image: tchiotludo/akhq
    environment:
      AKHQ_CONFIGURATION: |
        akhq:
          connections:
            docker-kafka-server:
              properties:
                bootstrap.servers: "kafka.us-west-2.amazonaws.com:9094"
                security.protocol: SSL
              # schema-registry:
              #   url: "http://schema-registry:8085"
              # connect:
              #   - name: "connect"
              #     url: "http://connect:8083"
    ports:
      - 8080:8080

Bạn để ý chỗ ENV AKHQ_CONFIGURATION
Bạn cần điện ip kafka, ip schema-registry, ip kafka-connect

Cái này có 1 điểm hay ở phần tạo connector của kafka-connect. Họ giải thích từng key đó có ý nghĩa gì?


Mình mới thấy dược repo helm chart:
https://staging.artifacthub.io/packages/helm/akhq/akhq

2) AKHQ integrate MSK or Kafka that has the authentication type “TLS”

Bạn thấy là config MSK đang có authentication Type “TLS”
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    workload.user.cattle.io/workloadselector: deployment-kafka-kafka-akhq
  name: kafka-akhq
  namespace: kafka
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      workload.user.cattle.io/workloadselector: deployment-kafka-kafka-akhq
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
    type: RollingUpdate
  template:
    metadata:
      labels:
        workload.user.cattle.io/workloadselector: deployment-kafka-kafka-akhq
    spec:
      containers:
      - env:
        - name: AKHQ_CONFIGURATION
          value: |-
            akhq:
              connections:
                docker-kafka-server:
                  properties:
                    bootstrap.servers: "kafka:9092"
                    security.protocol: SSL ### Look at
                  schema-registry:
                    url: "http://schema-registry-cp-schema-registry:8081"
                  connect:
                    - name: "kafka-connect"
                      url: "http://kafka-connect-cp-kafka-connect:8083"
        image: tchiotludo/akhq
        imagePullPolicy: Always
        name: kafka-akhq
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
      restartPolicy: Always
      nodeSelector:
        kubernetes.io/os: linux
Apache Kafka, Lenses

Post navigation

Previous Post: [Kafka-connect] Streaming the data of MongoDB through Kafka-connect into topic Kafka.
Next Post: [Lenses/kafka] Fix the problem “Cannot extract connector information from the configuration provided” when creating connector Kafka-connect

More Related Articles

[Kafka-connect] Install lenses to manage Kafka. Apache Kafka
[Lenses/kafka] Configure authentication and authorization for lenses Lenses
[Kafka-connect] Streaming the data of MongoDB through Kafka-connect into topic Kafka. Apache Kafka
[Kafka-connect] Single Message Transform: lesson 5 MaskField – Cover the sensitive data. Apache Kafka
[Kafka-connect] Single Message Transform: lesson 7 – TimeStampRouter and MessageTimestampRouter – Custom format topic name with timestamp Apache Kafka
[Kafka] Kafka Topics CLI Apache Kafka

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • [Smartctl] Instruction check the health disk of Raspberry. January 16, 2023
  • [kubectl/Argocd] How to create a kubectl config file for serviceaccount or from the cluster secret of Argocd January 12, 2023
  • [Helm/Github] Create a public Helm chart repository with GitHub Pages January 8, 2023
  • [AWS] How to increase the disk size of a Windows EC2 machine? January 4, 2023
  • [Redis] ElastiCache-Redis Cross-Region Replication|Global DataStore January 3, 2023

Archives

  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • BareMetal
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • Golang
    • Jquery & JavaScript
    • Laravel
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kubernetes
      • Ingress
    • Longhorn – Storage
    • Vault
    • VictoriaMetrics
  • Log & Monitor
    • ELK
      • Kibana
      • Logstash
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2023 NimTechnology.