Mình đã viết nhiều bài về việc tạo PVC trên eks hay k8s
1) Install CSI Driver SMB
Vậy nếu trong cluster k8s của bạn có cả linux và windows node thì phải làm sao?
vâng chúng đó có 1 giải pháp đó là CSI Driver SMB
https://github.com/kubernetes-csi/csi-driver-smb
https://www.phillipsj.net/posts/how-to-use-the-windows-csi-proxy-and-csi-smb-driver-for-kubernetes/
Đầu tiên bạn cần cài đặt CSI Driver SMB thông qua helm chart.
Mình cung cấp 1 file application của argocd
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: csi-driver-smb-nimtechnology-staging
namespace: argocd
spec:
destination:
namespace: kube-system
name: 'arn:aws:eks:us-west-2:04370111XXXXX:cluster/dev-mdcl-nimtechnology-engines'
project: meta-structure
source:
repoURL: https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
targetRevision: "v1.9.0"
chart: csi-driver-smb
helm:
values: |-
windows:
dsName: csi-smb-node-win
enabled: true
Check the windows node on the os for the existence of the following directory once the deployment has finished:
C:\var\lib\kubelet\plugins\smb.csi.k8s.io
2) Install SMB Server.
Tiếp theo bạn sẽ cần cài đặt SMB Server.
Trong bài test này chúng ta sẽ cài smb server trong cluster k8s
https://github.com/kubernetes-csi/csi-driver-smb/tree/master/deploy/example/smb-provisioner
The first step is to add the secret:
kubectl create secret generic smbcreds --from-literal username=windows --from-literal password="IsAwesome"Tiếp đến là bạn cài đặt smb-server
Cách 1: dụng local disk of node
modify /smbshare-volume in deployment to specify different path for smb share data store
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server.yaml---
kind: Service
apiVersion: v1
metadata:
name: smb-server
labels:
app: smb-server
spec:
type: ClusterIP # use "LoadBalancer" to get a public ip
selector:
app: smb-server
ports:
- port: 445
name: smb-server
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: smb-server
spec:
replicas: 1
selector:
matchLabels:
app: smb-server
template:
metadata:
name: smb-server
labels:
app: smb-server
spec:
nodeSelector:
"kubernetes.io/os": linux
containers:
- name: smb-server
image: andyzhangx/samba:win-fix
env:
- name: PERMISSIONS
value: "0777"
- name: USERNAME
valueFrom:
secretKeyRef:
name: smbcreds
key: username
- name: PASSWORD
valueFrom:
secretKeyRef:
name: smbcreds
key: password
args: ["-u", "$(USERNAME);$(PASSWORD)", "-s", "share;/smbshare/;yes;no;no;all;none", "-p"]
volumeMounts:
- mountPath: /smbshare
name: data-volume
ports:
- containerPort: 445
volumes:
- name: data-volume
hostPath:
path: /home/kubernetes/smbshare-volume # modify this to specify another path to store smb share data
type: DirectoryOrCreate
Cách 2: Create a Samba Server deployment on the network disk
kubectl create -f https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/deploy/example/smb-provisioner/smb-server-networkdisk.yaml---
kind: Service
apiVersion: v1
metadata:
name: smb-server
labels:
app: smb-server
spec:
type: ClusterIP # use "LoadBalancer" to get a public ip
selector:
app: smb-server
ports:
- port: 445
name: smb-server
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-networkdisk-smbshare
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 100Gi
#storageClassName: default # storage provider is Azure disk in Azure and persistent disk in GCE.
---
kind: Deployment
apiVersion: apps/v1
metadata:
name: smb-server
spec:
replicas: 1
selector:
matchLabels:
app: smb-server
template:
metadata:
name: smb-server
labels:
app: smb-server
spec:
nodeSelector:
"kubernetes.io/os": linux
containers:
- name: smb-server
image: dperson/samba
env:
- name: PERMISSIONS
value: "0777"
- name: USERNAME
valueFrom:
secretKeyRef:
name: smbcreds
key: username
- name: PASSWORD
valueFrom:
secretKeyRef:
name: smbcreds
key: password
args: ["-u", "$(USERNAME);$(PASSWORD)", "-s", "share;/smbshare/;yes;no;no;all;none", "-p"]
volumeMounts:
- mountPath: /smbshare
name: data-volume
ports:
- containerPort: 445
volumes:
- name: data-volume
persistentVolumeClaim:
claimName: pvc-networkdisk-smbshare
Ok giờ chúng ta test pvc với smb.
Đầu tiêu bạn cần tạo 1 storageclass và có cách thông tin connect smb server.
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: smb provisioner: smb.csi.k8s.io parameters: # On Windows, "*.default.svc.cluster.local" could not be recognized by csi-proxy source: "//smb-server.default.svc.cluster.local/share" # if csi.storage.k8s.io/provisioner-secret is provided, will create a sub directory # with PV name under source csi.storage.k8s.io/provisioner-secret-name: "smbcreds" csi.storage.k8s.io/provisioner-secret-namespace: "<namespace>" # you need to change it csi.storage.k8s.io/node-stage-secret-name: "smbcreds" csi.storage.k8s.io/node-stage-secret-namespace: "<namespace>" # you need to change it volumeBindingMode: Immediate mountOptions: - dir_mode=0777 - file_mode=0777 - uid=1001 - gid=1001
3) Demo
Giờ bạn tạo 1 pvc và deployment
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-smb
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: smb
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: pwsh-smb
labels:
app: pwsh
spec:
replicas: 1
template:
metadata:
name: pwsh
labels:
app: pwsh
spec:
nodeSelector:
"kubernetes.io/os": windows
containers:
- name: pwsh
image: mcr.microsoft.com/powershell:lts-nanoserver-1809
command:
- "pwsh.exe"
- "-Command"
- "while (1) { Add-Content -Encoding Ascii C:\\mnt\\smb\\data.txt $(Get-Date -Format u); sleep 1 }"
volumeMounts:
- name: smb
mountPath: "/mnt/smb"
subPath: subPath
volumes:
- name: smb
persistentVolumeClaim:
claimName: pvc-smb
selector:
matchLabels:
app: pwsh
Then you can test by verifing that data.txt exists in the SMB share.
$ kubectl exec -it pwsh-smb-0 -- pwsh
C:/ $ ls mnt/smb
Directory: C:\mnt\smb
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 1/11/2022 8:27 PM 3476 data.txt
Pay attention to
Bạn sẽ thấy là khi bạn tạo storageclass mà sử dụng source: "//smb-server.default.svc.cluster.local/share"
thì bạn sẽ thấy CSI Driver SMB sẽ không resolve được smb-server.default.svc.cluster.local nên bạn chỉ có thể điền thằng IP hoặc domain public vào storageclass
