When K8s pods are stuck mounting large volumes

Posted on By nim No Comments on When K8s pods are stuck mounting large volumes
When K8s pods are stuck mounting large volumes

refer:https://blog.devgenius.io/when-k8s-pods-are-stuck-mounting-large-volumes-2915e6656cb8 Recently we ran into the following problem with our Loki deployment on AWS/EKS. On every deployment or restart of a Loki Pod, mounting the persistent volume took longer and longer. It started with a few minutes delay and ended up with nearly 25 minutes on our production cluster. Having no solution for this we…

Read More “When K8s pods are stuck mounting large volumes” »

Pod

[Argo-Workflows] Lesson6: Output Parameter File

Posted on By nim No Comments on [Argo-Workflows] Lesson6: Output Parameter File
[Argo-Workflows] Lesson6: Output Parameter File

Chà chà đã quá lâu chúng ta trờ lại học Argo Workflows Ở bài 5 chúng ta chuyền value từ task này sang task khác thông qua Output Parameter Ở bài sáu thì chúng truyền parameter bằng nội dung trong file. Bạn để ý task3 nó sẽ thực hiện các action được define in template:…

Read More “[Argo-Workflows] Lesson6: Output Parameter File” »

ArgoWorkflows

[DataDog] How does DataDog collect metrics from the Prometheus exporter endpoint

Posted on By nim No Comments on [DataDog] How does DataDog collect metrics from the Prometheus exporter endpoint
[DataDog] How does DataDog collect metrics from the Prometheus exporter endpoint

1) Introduce the design architecture Mô hình bạn có thể thấy như hình bên dưới. Thì mình dùng datadog agent trên k8s để collect metrics của cloudwatchNhưng mà datadog agent không để đọc trực tiếp vào Cloudwatch nên mình sẽ sử dụng 1 opensource của prometheus là prometheus cloudwatch exporter để get metrics từ cloud…

Read More “[DataDog] How does DataDog collect metrics from the Prometheus exporter endpoint” »

DataDog, Prometheus

[Datadog/Tracing] Apply the tracing datadog for PHP or WordPress.

Posted on By nim No Comments on [Datadog/Tracing] Apply the tracing datadog for PHP or WordPress.
[Datadog/Tracing] Apply the tracing datadog for PHP or WordPress.

1) Configure the datadog agent to receive the APM data. Như mình đã nói thì datadog cho chúng ta free.Và hiện tại mình đang dụng datadog để monitor cluster k8s(home lab), log (stdout). tiếp mình sẽ setup tracing trên datadog. Enable datadog via helm chart. thì chúng ta sẽ cần phải config helm như sau…

Read More “[Datadog/Tracing] Apply the tracing datadog for PHP or WordPress.” »

DataDog

[Promethues] Install Prometheus Operator on Kubernetes.

Posted on By nim No Comments on [Promethues] Install Prometheus Operator on Kubernetes.
[Promethues] Install Prometheus Operator on Kubernetes.

1) the preparing before install Prometheus Operator Đầu tiên thì bạn cần cài đặt trước các CustomResourceDefinition REPO URL: https://prometheus-community.github.io/helm-chartsCHART: prometheus-operator-crds:2.0.0 We need to press the sync button I am sure that you also have a problem with prometheuses.monitoring.coreos.comThis message: “CustomResourceDefinition.apiextensions.k8s.io “prometheuses.monitoring.coreos.com” is invalid: metadata.annotations: Too long: must have at most 262144 bytes“ Don’t be…

Read More “[Promethues] Install Prometheus Operator on Kubernetes.” »

Prometheus

[windows-exporter] Monitor the windows Kubernetes.

Posted on By nim No Comments on [windows-exporter] Monitor the windows Kubernetes.
[windows-exporter] Monitor the windows Kubernetes.

Hiện tại thì mỉnh sử dụng windows-exporter để monitor windows k8s.https://github.com/prometheus-community/windows_exporter Để integrate với windows_exporter thì bạn nên cài trước prometheus_operator. Mình đang test trên eks 1.24 1) Install the component of windows exporter. https://github.com/prometheus-community/windows_exporter/blob/master/kubernetes/windows-exporter-daemonset.yaml Lúc này thì trên prometheus-operator đang nhận được metrics của windows-exporter. https://grafana.com/grafana/dashboards/17264-windows-exporter-for-prometheus-dashboard-en-v20221026/ Hiện tại mình chỉ thấy là nó monitor…

Read More “[windows-exporter] Monitor the windows Kubernetes.” »

Prometheus

Experiences for IP Addresses Shortage on EKS Clusters

Posted on By nim No Comments on Experiences for IP Addresses Shortage on EKS Clusters
Experiences for IP Addresses Shortage on EKS Clusters

Link: https://medium.com/compass-true-north/experiences-for-ip-addresses-shortage-on-eks-clusters-a740f56ac2f5 In a fast-growing company, like Compass, things may become challenging for Cloud infrastructure teams. As serving more and more customers, many backend services scaled up in our Kubernetes clusters. Meanwhile, a variety of new backend services went online to satisfy new requirements. Recently, a big challenge for our Cloud Engineering team in Compass…

Read More “Experiences for IP Addresses Shortage on EKS Clusters” »

AWS - Amazon Web Service

[Talisman/DevSecOps] Discover the sensitive information in your code.

Posted on By nim No Comments on [Talisman/DevSecOps] Discover the sensitive information in your code.
[Talisman/DevSecOps] Discover the sensitive information in your code.

Chúng ta có 1 bài post.https://www.theregister.com/2015/01/06/dev_blunder_shows_github_crawling_with_keyslurping_bots/ Git Hooks and Talisman Introduction HandsOn – Talisman thực hiện kiểm tra hook trong code dưới laptop. Thực hiện cài đặt Talisman trên máy của bạn Bạn thấy pre-push sẽ được thê trong hook của git Bạn có thể setup scankhi commit. Khi bạn thúc hiện commit thì talisman…

Read More “[Talisman/DevSecOps] Discover the sensitive information in your code.” »

DevSecOps