Skip to content

NimTechnology

Trình bày các công nghệ CLOUD một cách dễ hiểu.

  • Kubernetes & Container
    • Docker
    • Kubernetes
      • Ingress
    • Helm Chart
    • Isito-EnvoyFilter
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Vault
    • Longhorn – Storage
    • VictoriaMetrics
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Spinnaker
    • Jenkins
    • Harbor
    • TeamCity
    • Git
      • Bitbucket
  • Coding
    • Terraform
      • GCP – Google Cloud
      • AWS – Amazon Web Service
    • Golang
    • Laravel
    • Python
    • Jquery & JavaScript
    • Selenium
  • Log & Monitor
    • Prometheus
    • Grafana
    • ELK
      • Kibana
      • Logstash
  • BareMetal
  • Toggle search form

[csi-driver-smb] Providing PVC with type ReadWriteMany base on SMB

Posted on December 15, 2022December 21, 2022 By nim No Comments on [csi-driver-smb] Providing PVC with type ReadWriteMany base on SMB

Mình đã viết nhiều bài về việc tạo PVC trên eks hay k8s

[AWS] Creat Persistent Volume on EKS via EBS.
Nhưng ebs lại không hỗ trợ readwriteone
[AWS/EKS] EFS CSI Driver – Create Persistent Volume Clain with ReadWriteMany type on EKS
efs thì đã hỗ trợ readwritemany nhưng không hỗ trợ windows node Group

Contents

  • 1) Install CSI Driver SMB
  • 2) Install SMB Server.
  • 3) Demo
  • Pay attention to

1) Install CSI Driver SMB

Vậy nếu trong cluster k8s của bạn có cả linux và windows node thì phải làm sao?
vâng chúng đó có 1 giải pháp đó là CSI Driver SMB

https://github.com/kubernetes-csi/csi-driver-smb
https://www.phillipsj.net/posts/how-to-use-the-windows-csi-proxy-and-csi-smb-driver-for-kubernetes/

Đầu tiên bạn cần cài đặt CSI Driver SMB thông qua helm chart.
Mình cung cấp 1 file application của argocd

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: csi-driver-smb-nimtechnology-staging
  namespace: argocd
spec:
  destination:
    namespace: kube-system
    name: 'arn:aws:eks:us-west-2:04370111XXXXX:cluster/dev-mdcl-nimtechnology-engines'
  project: meta-structure
  source:
    repoURL: https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
    targetRevision: "v1.9.0"
    chart: csi-driver-smb
    helm:
      values: |-
        windows:
          dsName: csi-smb-node-win
          enabled: true
Giờ bạn đã cài được csi-driver-smb

Check the windows node on the os for the existence of the following directory once the deployment has finished:

C:\var\lib\kubelet\plugins\smb.csi.k8s.io

2) Install SMB Server.

Tiếp theo bạn sẽ cần cài đặt SMB Server.

Trong bài test này chúng ta sẽ cài smb server trong cluster k8s
https://github.com/kubernetes-csi/csi-driver-smb/tree/master/deploy/example/smb-provisioner

The first step is to add the secret:

kubectl create secret generic smbcreds --from-literal username=windows --from-literal password="IsAwesome"

Tiếp đến là bạn cài đặt smb-server

Cách 1: dụng local disk of node
modify /smbshare-volume in deployment to specify different path for smb share data store

kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server.yaml
---
kind: Service
apiVersion: v1
metadata:
  name: smb-server
  labels:
    app: smb-server
spec:
  type: ClusterIP  # use "LoadBalancer" to get a public ip
  selector:
    app: smb-server
  ports:
    - port: 445
      name: smb-server
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: smb-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: smb-server
  template:
    metadata:
      name: smb-server
      labels:
        app: smb-server
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: smb-server
          image: andyzhangx/samba:win-fix
          env:
            - name: PERMISSIONS
              value: "0777"
            - name: USERNAME
              valueFrom:
                secretKeyRef:
                  name: smbcreds
                  key: username
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: smbcreds
                  key: password
          args: ["-u", "$(USERNAME);$(PASSWORD)", "-s", "share;/smbshare/;yes;no;no;all;none", "-p"]
          volumeMounts:
            - mountPath: /smbshare
              name: data-volume
          ports:
            - containerPort: 445
      volumes:
        - name: data-volume
          hostPath:
            path: /home/kubernetes/smbshare-volume  # modify this to specify another path to store smb share data
            type: DirectoryOrCreate

Cách 2: Create a Samba Server deployment on the network disk

kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server-networkdisk.yaml
---
kind: Service
apiVersion: v1
metadata:
  name: smb-server
  labels:
    app: smb-server
spec:
  type: ClusterIP  # use "LoadBalancer" to get a public ip
  selector:
    app: smb-server
  ports:
    - port: 445
      name: smb-server
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-networkdisk-smbshare
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
  #storageClassName: default  # storage provider is Azure disk in Azure and persistent disk in GCE.
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: smb-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: smb-server
  template:
    metadata:
      name: smb-server
      labels:
        app: smb-server
    spec:
      nodeSelector:
        "kubernetes.io/os": linux
      containers:
        - name: smb-server
          image: dperson/samba
          env:
            - name: PERMISSIONS
              value: "0777"
            - name: USERNAME
              valueFrom:
                secretKeyRef:
                  name: smbcreds
                  key: username
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: smbcreds
                  key: password
          args: ["-u", "$(USERNAME);$(PASSWORD)", "-s", "share;/smbshare/;yes;no;no;all;none", "-p"]
          volumeMounts:
            - mountPath: /smbshare
              name: data-volume
          ports:
            - containerPort: 445
      volumes:
        - name: data-volume
          persistentVolumeClaim:
            claimName: pvc-networkdisk-smbshare

Ok giờ chúng ta test pvc với smb.

Đầu tiêu bạn cần tạo 1 storageclass và có cách thông tin connect smb server.

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: smb
provisioner: smb.csi.k8s.io
parameters:
  # On Windows, "*.default.svc.cluster.local" could not be recognized by csi-proxy
  source: "//smb-server.default.svc.cluster.local/share"
  # if csi.storage.k8s.io/provisioner-secret is provided, will create a sub directory
  # with PV name under source
  csi.storage.k8s.io/provisioner-secret-name: "smbcreds"
  csi.storage.k8s.io/provisioner-secret-namespace: "<namespace>" # you need to change it
  csi.storage.k8s.io/node-stage-secret-name: "smbcreds"
  csi.storage.k8s.io/node-stage-secret-namespace: "<namespace>" # you need to change it
volumeBindingMode: Immediate
mountOptions:
  - dir_mode=0777
  - file_mode=0777
  - uid=1001
  - gid=1001

3) Demo

Giờ bạn tạo 1 pvc và deployment

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-smb
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: smb
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: pwsh-smb
  labels:
    app: pwsh
spec:
  replicas: 1
  template:
    metadata:
      name: pwsh
      labels:
        app: pwsh
    spec:
      nodeSelector:
        "kubernetes.io/os": windows
      containers:
        - name: pwsh
          image: mcr.microsoft.com/powershell:lts-nanoserver-1809
          command:
            - "pwsh.exe"
            - "-Command"
            - "while (1) { Add-Content -Encoding Ascii C:\\mnt\\smb\\data.txt $(Get-Date -Format u); sleep 1 }"
          volumeMounts:
            - name: smb
              mountPath: "/mnt/smb"
              subPath: subPath
      volumes:
        - name: smb
          persistentVolumeClaim:
            claimName: pvc-smb
  selector:
    matchLabels:
      app: pwsh

Then you can test by verifing that data.txt exists in the SMB share.

$ kubectl exec -it pwsh-smb-0  -- pwsh
C:/ $ ls mnt/smb

    Directory: C:\mnt\smb

Mode                 LastWriteTime         Length Name
----                 -------------         ------ ----
-a---           1/11/2022  8:27 PM           3476 data.txt

Pay attention to

Bạn sẽ thấy là khi bạn tạo storageclass mà sử dụng
source: "//smb-server.default.svc.cluster.local/share"

thì bạn sẽ thấy CSI Driver SMB sẽ không resolve được smb-server.default.svc.cluster.local nên bạn chỉ có thể điền thằng IP hoặc domain public vào storageclass

https://github.com/kubernetes-csi/csi-driver-smb/issues/294

AWS - Amazon Web Service

Post navigation

Previous Post: [eks] Kubernetes Cluster-AutoScaler error: Failed to watch *v1.CSIDriver
Next Post: [EKS] Checking your EKS cluster that is working efficiently.

More Related Articles

[EFS/EKS] Warning: config file does not have retry_nfs_mount_command item in section mount… AWS - Amazon Web Service
[AWS] Login Argocd via Cognito in AWS ArgoCD
[EKS/Terraform] Installing or Provisioning an EKS cluster through Terraform Module. AWS - Amazon Web Service
Manage Kubernetes Secrets With External Secrets Operator AWS - Amazon Web Service
[AWS] Provision AWS IAM Admin User as EKS Admin AWS - Amazon Web Service
[Terraform] – Terraform Beginner – Lesson 7: Terraform Modules AWS - Amazon Web Service

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Tham Gia Group DevOps nhé!
Để Nim có nhiều động lực ra nhiều bài viết.
Để nhận được những thông báo mới nhất.

Recent Posts

  • [Smartctl] Instruction check the health disk of Raspberry. January 16, 2023
  • [kubectl/Argocd] How to create a kubectl config file for serviceaccount or from the cluster secret of Argocd January 12, 2023
  • [Helm/Github] Create a public Helm chart repository with GitHub Pages January 8, 2023
  • [AWS] How to increase the disk size of a Windows EC2 machine? January 4, 2023
  • [Redis] ElastiCache-Redis Cross-Region Replication|Global DataStore January 3, 2023

Archives

  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021

Categories

  • BareMetal
  • CI/CD
    • ArgoCD
    • ArgoWorkflows
    • Git
      • Bitbucket
    • Harbor
    • Jenkins
    • Spinnaker
    • TeamCity
  • Coding
    • Golang
    • Jquery & JavaScript
    • Laravel
    • Python
    • Selenium
    • Terraform
      • AWS – Amazon Web Service
      • GCP – Google Cloud
  • Kubernetes & Container
    • Apache Kafka
      • Kafka
      • Kafka Connect
      • Lenses
    • Docker
    • Helm Chart
    • Isito-EnvoyFilter
    • Kubernetes
      • Ingress
    • Longhorn – Storage
    • Vault
    • VictoriaMetrics
  • Log & Monitor
    • ELK
      • Kibana
      • Logstash
    • Grafana
    • Prometheus
  • Uncategorized
  • Admin

Copyright © 2023 NimTechnology.