k8s部署mysql集群master--slave

k8s部署mysql集群master--slave

一、环境信息

CentOS Linux release 7.8.2003 (Core)

4.4.240-1.el7.elrepo.x86_64

k8s 集群版本：1.15

docker 版本：Docker version 17.03.3-ce

集群信息：

NAME STATUS ROLES AGE VERSION km-1.11 Ready master 7d7h v1.15.1 kn-1.11 Ready 7d7h v1.15.1 kn1-1.11 Ready 31h v1.15.1

二、MySQL的高可用方案

MySQL高可用方案采用主从复制+读写分离，即由单一的master和多个slave所构成。其中，客户端通过master对数据库进行写操作，通过slave端进行读操作。master出现问题后，可以将应用切换到slave端。 此方案是MySQL官方提供的一种高可用解决方案，节点间的数据同步采用MySQL Replication技术。MySQL Replication从一个MySQL数据库服务器（master）的数据复制到一个或多个MySQL数据库服务器（slave）。在默认情况下，复制是异步的；slave不需要一直接收来自主机的更新。根据配置，可以复制数据库中的所有数据库、选定的数据库，或者特定的表。

三、架构说明

通过mysql+xtrabackup的模式来组成数据库master+slave的模式，具体的架构图如下所示：

四、创建StorageClass

因为需要把数据库数据持久化存储，因此需要为pod自动创建pv，pvc

参考：storageclass]# cat nfs-client.yaml kind: Deployment apiVersion: apps/v1 metadata: name: nfs-client-provisioner namespace: blog spec: replicas: 1 strategy: type: Recreate selector: matchLabels: app: nfs-client-provisioner template: metadata: namespace: blog labels: app: nfs-client-provisioner spec: serviceAccountName: nfs-client-provisioner containers: - name: nfs-client-provisioner image: quay.io/external_storage/nfs-client-provisioner:latest volumeMounts: - name: nfs-client-root mountPath: /persistentvolumes env: - name: PROVISIONER_NAME value: fuseim.pri/ifs - name: NFS_SERVER value: 192.168.140.50 - name: NFS_PATH value: /data/k8s volumes: - name: nfs-client-root nfs: server: 192.168.140.50 path: /data/k8s

[root@km-1 storageclass]# cat nfs-client-sa.yaml apiVersion: v1 kind: ServiceAccount metadata: namespace: blog name: nfs-client-provisioner --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: blog name: nfs-client-provisioner-runner rules: - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "create", "delete"] - apiGroups: [""] resources: ["persistentvolumeclaims"] verbs: ["get", "list", "watch", "update"] - apiGroups: ["storage.k8s.io"] resources: ["storageclasses"] verbs: ["get", "list", "watch"] - apiGroups: [""] resources: ["events"] verbs: ["list", "watch", "create", "update", "patch"] - apiGroups: [""] resources: ["endpoints"] verbs: ["create", "delete", "get", "list", "watch", "patch", "update"] --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: blog name: run-nfs-client-provisioner subjects: - kind: ServiceAccount name: nfs-client-provisioner namespace: blog roleRef: kind: ClusterRole name: nfs-client-provisioner-runner apiGroup: rbac.authorization.k8s.io

[root@km-1 storageclass]# cat nfs-client-class.yaml apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: namespace: blog name: course-nfs-storage provisioner: fuseim.pri/ifs

五、staefulset搭建Mysql集群

1、创建yaml

Yaml配置如下：

[root@km-1 mysql]# cat configmap.yaml apiVersion: v1 kind: ConfigMap metadata: namespace: blog name: mysql labels: app: mysql data: master.cnf: | # Apply this config only on the master. [mysqld] log-bin slave.cnf: | # Apply this config only on slaves. [mysqld] super-read-only

部署headless服务，有状态服务都需要，让服务旗下的Pod彼此发现

[root@km-1 mysql]# cat service.yaml # Headless service for stable DNS entries of StatefulSet members. apiVersion: v1 kind: Service metadata: namespace: blog name: mysql labels: app: mysql spec: ports: - name: mysql port: 3306 clusterIP: None selector: app: mysql --- # Client service for connecting to any MySQL instance for reads. # For writes, you must instead connect to the master: mysql-0.mysql. #apiVersion: v1 #kind: Service #metadata: # namespace: blog # name: mysql-read # labels: # app: mysql #spec: # ports: #- name: mysql # port: 3306 #selector: # app: mysql

[root@km-1 mysql]# cat statefulset.yaml apiVersion: apps/v1 kind: StatefulSet metadata: namespace: blog name: mysql spec: selector: matchLabels: app: mysql serviceName: mysql replicas: 3 template: metadata: namespace: blog labels: app: mysql spec: initContainers: - name: init-mysql image: mysql:5.7 command: - bash - "-c" - | set -ex # Generate mysql server-id from pod ordinal index. [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} echo [mysqld] > /mnt/conf.d/server-id.cnf # Add an offset to avoid reserved server-id=0 value. echo server-id=$((100 + $ordinal)) >> /mnt/conf.d/server-id.cnf # Copy appropriate conf.d files from config-map to emptyDir. if [[ $ordinal -eq 0 ]]; then cp /mnt/config-map/master.cnf /mnt/conf.d/ else cp /mnt/config-map/slave.cnf /mnt/conf.d/ fi volumeMounts: - name: conf mountPath: /mnt/conf.d - name: config-map mountPath: /mnt/config-map - name: clone-mysql image: hub.c.163.com/combk8s/xtrabackup:1.0 command: - bash - "-c" - | set -ex # Skip the clone if data already exists. [[ -d /var/lib/mysql/mysql ]] && exit 0 # Skip the clone on master (ordinal index 0). [[ `hostname` =~ -([0-9]+)$ ]] || exit 1 ordinal=${BASH_REMATCH[1]} [[ $ordinal -eq 0 ]] && exit 0 # Clone data from previous peer. ncat --recv-only mysql-$(($ordinal-1)).mysql 3307 | xbstream -x -C /var/lib/mysql # Prepare the backup. xtrabackup --prepare --target-dir=/var/lib/mysql volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d containers: - name: mysql image: mysql:5.7 env: - name: MYSQL_ALLOW_EMPTY_PASSWORD value: "1" ports: - name: mysql containerPort: 3306 volumeMounts: - name: data mountPath: /var/lib/mysql subPath: mysql - name: conf mountPath: /etc/mysql/conf.d resources: requests: # cpu: 200m memory: 1Gi livenessProbe: exec: command: ["mysqladmin", "ping"] initialDelaySeconds: 30 periodSeconds: 10 timeoutSeconds: 5 readinessProbe: exec: # Check we can execute queries over TCP (skip-networking is off). command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"] initialDelaySeconds: 5 periodSeconds: 2 timeoutSeconds: 1 - name: xtrabackup image: hub.c.163.com/combk8s/xtrabackup:1.0 ports: - name: xtrabackup containerPort: 3307 command: - bash - "-c" - | set -ex cd /var/lib/mysql # Determine binlog position of cloned data, if any. if [[ -f xtrabackup_slave_info ]]; then # XtraBackup already generated a partial "CHANGE MASTER TO" query # because we're cloning from an existing slave. mv xtrabackup_slave_info change_master_to.sql.in # Ignore xtrabackup_binlog_info in this case (it's useless). rm -f xtrabackup_binlog_info elif [[ -f xtrabackup_binlog_info ]]; then # We're cloning directly from master. Parse binlog position. [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1 rm xtrabackup_binlog_info echo "CHANGE MASTER TO MASTER_LOG_FILE='${BASH_REMATCH[1]}',\ MASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in fi # Check if we need to complete a clone by starting replication. if [[ -f change_master_to.sql.in ]]; then echo "Waiting for mysqld to be ready (accepting connections)" until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done echo "Initializing replication from clone position" # In case of container restart, attempt this at-most-once. mv change_master_to.sql.in change_master_to.sql.orig mysql -h 127.0.0.1 <

详细解释看：mysql]# kubectl get statefulset/mysql -n blog -o wide NAME READY AGE CONTAINERS IMAGES mysql 2/2 29h mysql,xtrabackup mysql:5.7,hub.c.163.com/combk8s/xtrabackup:1.0

[root@km-1 mysql]# kubectl get pods -lapp=mysql --watch --namespace=blog NAME READY STATUS RESTARTS AGE mysql-0 2/2 Running 9 29h mysql-1 2/2 Running 7 29h

如果遇到失败，查看日志执行：

kubectl logs -f pod/mysql-0 --all-containers=true -n blog

2、MySQL部署环境验证

通过运行一个临时的容器(使用mysql:5.7镜像)，使用MySQL 客户端发送测试请求给MySQL master节点（主机名为mysql-0.mysql；跨命名空间的话，主机名请使用mysql-0.mysql.blog）

kubectl run mysql-client --image=mysql:5.7 -it --rm --restart=Never -- mysql -h mysql-0.mysql.blog

CREATE DATABASE demo; CREATE TABLE demo.messages (message VARCHAR(250)); INSERT INTO demo.messages VALUES ('hello');

使用主机名为mysql-read来发送测试请求给服务器：

kubectl run mysql-client --image=mysql:5.7 -i -t --rm --restart=Never -- mysql -h mysql-read.blog

写入数据

insert into message values("hello world");

此时会报错 ERROR 1290 (HY000): The MySQL server is running with the --super-read-only option so it cannot execute this statement这是因为mysql-1是一个只读备库，无法进行写操作。

3、扩缩容

//扩容至5副本 kubectl scale statefulset mysql --replicas=5 //缩容只2副本 kubectl scale statefulset mysql --replicas=2

4、 部署对外读服务

apiVersion: v1 kind: Service metadata: namespace: blog name: mysql-read labels: app: mysql spec: ports: - name: mysql port: 3306 targetPort: 3306 nodePort: 30036 type: NodePort selector: app: mysql

5、部署对外读写服务

[root@k8s-master ~]# kubectl get pods --show-labels -n blog | grep mysql-0 | awk '{print $6}' | awk -F, '{print $3}' statefulset.kubernetes.io/pod-name=mysql-0

得到该标签后写入如下selector字段

apiVersion: v1 kind: Service metadata: namespace: blog name: mysql-writeandread labels: app: mysql spec: ports: - name: mysql port: 3306 targetPort: 3306 nodePort: 30006 selector: statefulset.kubernetes.io/pod-name: mysql-0 type: NodePort

使用集群任意节点IP:nodePort进行连接

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。