使用nfs在k8s集群中实现持久化存储

使用nfs在k8s集群中实现持久化存储

准备NFS服务192.168.1.244$ yum -y install nfs-utils rpcbind$ systemctl start nfs-server rpcbind$ systemctl enable nfs-server rpcbind$ mkdir -p /data/k8s$ cd /data/k8s$ echo 11111111 > index.html$ vim /etc/exports/data/k8s *(rw,async,no_root_squash)$ systemctl restart nfs-server$ exportfs -arv客户端测试，所有k8s节点都要安装nfs客户端$ yum -y install nfs-utils rpcbind$ systemctl start nfs rpcbind$ systemctl enable nfs rpcbind$ showmount -e 192.168.1.244

创建pod直接挂载nfs服务器

apiVersion: v1 kind: Pod metadata: labels: run: nginx name: podxx spec: volumes: - name: nfs nfs: server: 192.168.1.244 path: /data/k8s containers: - image: nginx name: nginx volumeMounts: - mountPath: /usr/share/nginx/html name: nfs

$ kubectl exec podxx -it bash

手动管理pv和pvc创建顺序：后端存储---pv---pvc---pod删除顺序：pod---pvc---pv1、创建pv

apiVersion: v1 kind: PersistentVolume metadata: name: pv2 labels: app: nfs spec: capacity: storage: 1Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Recycle nfs: path: /data/k8s server: 192.168.1.244

2、创建pvc

kind: PersistentVolumeClaim apiVersion: v1 metadata: name: pvc2-nfs spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi selector: matchLabels: app: nfs

上述pvc会自动和具有访问模式是ReadWriteOnce、storage大于等于1Gi、标签是app: nfs的pv进行绑定3创建pod使用pvc

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nfs-pvc spec: replicas: 3 template: metadata: labels: app: nfs-pvc spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent ports: - containerPort: 80 name: web volumeMounts: - name: www subPath: nginx1 #需要手动指定nfs服务器中的子目录，该目录会自动创建 mountPath: /usr/share/nginx/html volumes: - name: www persistentVolumeClaim: claimName: pvc2-nfs #第2步中创建的pvc

结果是：共享存储中的/data/k8s/nginx1会被挂载到上述pod中的/usr/share/nginx/html目录

使用StorageClass管理pv和pvc通常情况下，只有pv是动态生成的（使用StatefulSet的pvc模板除外），其他的还是需要手动创建动态生成pv需要StorageClass和nfs-client-provisioner的共同作用1、创建nfs-client-provisionerprovisione直接使用nfs服务器$ docker pull quay.io/external_storage/nfs-client-provisioner:latest

kind: Deployment apiVersion: extensions/v1beta1 metadata: name: nfs-client-provisioner spec: replicas: 1 strategy: type: Recreate template: metadata: labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: quay.io/external_storage/nfs-client-provisioner:latest volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 192.168.1.244 - name: NFS_PATH value: /data/k8s volumes: - name: nfs-client-root nfs: server: 192.168.1.244 path: /data/k8s --- apiVersion: v1 kind: ServiceAccount metadata: name: nfs-client-provisioner --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: nfs-client-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["list", "watch", "create", "update", "patch"] - apiGroups: [""] resources: ["endpoints"] verbs: ["create", "delete", "get", "list", "watch", "patch", "update"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner namespace: default roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io

2、创建StorageClass

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: course-nfs-storage provisioner: fuseim.pri/ifs

3、创建pvc并动态生成pv

apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-pvc annotations: volume.beta.kubernetes.io/storage-class: "course-nfs-storage" spec: accessModes: - ReadWriteMany resources: requests: storage: 500Mi

$ kubectl get pv #pv会自动生成，并和pvc绑定

4、使用上述pvc创建pod和手动管理pv和pvc不同的是，使用StorageClass管理pv和pvc在创建pod时不用手动指定子目录，会在存储服务器根目录/data/k8s内自动生成一个随机子目录

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nfs-pvc spec: replicas: 3 template: metadata: labels: app: nfs-pvc spec: containers: - name: nginx image: nginx imagePullPolicy: IfNotPresent ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumes: - name: www persistentVolumeClaim: claimName: test-pvc

经过测试，不管是手动创建一个pod，还是用deployment创建多个pod，都只能生成一个随机目录，也就是一对pvc和pv对应一个持久存储目录

5、创建StatefulSet时，直接在其pvc模板中使用StorageClas自动生成pv和pvc$ kubectl explain StatefulSet.specvolumeClaimTemplates <[]Object>

apiVersion: apps/v1beta1 kind: StatefulSet metadata: name: nfs-web spec: serviceName: "nginx" replicas: 2 template: metadata: labels: app: nfs-web spec: terminationGracePeriodSeconds: 10 containers: - name: nginx image: nginx ports: - containerPort: 80 name: web volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www annotations: volume.beta.kubernetes.io/storage-class: course-nfs-storage spec: accessModes: [ "ReadWriteOnce" ] resources: requests: storage: 1Gi

上述脚本会生成两个pvc，两个pv，两个pod，在共享存储根目录/data/k8s内会自动生成两个随机子目录，因为副本数是2和deployment不同，有状态的应用必须每个pod有一个持久存储文件夹，不能多个pod共享一个

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。