【K8S运维知识汇总】第2天11：安装部署运算节点服务——kube-proxy

【K8S运维知识汇总】第2天11：安装部署运算节点服务——kube-proxy

部署kube-proxy

集群规划

主机名 角色 ipHDSS7-21.host.com kube-proxy 192.168.153.21 HDSS7-22.host.com kube-proxy 192.168.153.22注意：这里部署文档以HDSS7-21.host.com主机为例，另外一台运算节点安装部署方法类似

签发kube-proxy证书

运维主机HDSS7-200.host.com上：

# 签发生成证书签名请求（CSR）的JSON配置文件[root@localhost harbor]# cd /opt/certs/[root@hdss7-200 certs]# vi /opt/certs/kube-proxy-csr.json{ "CN": "system:kube-proxy", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "ST": "shengzheng", "L": "shengzheng", "O": "od", "OU": "ops" } ]}

生成证书

[root@hdss7-200 certs]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client kube-proxy-csr.json |cfssl-json -bare kube-proxy-client[root@hdss7-200 certs]# ll-rw-r--r-- 1 root root 1005 12月 12 10:23 kube-proxy-client.csr-rw------- 1 root root 1679 12月 12 10:23 kube-proxy-client-key.pem-rw-r--r-- 1 root root 1375 12月 12 10:23 kube-proxy-client.pem-rw-r--r-- 1 root root 267 12月 12 10:22 kube-proxy-csr.json

分发证书，将证书拷贝到node节点，注意私钥文件属性600

[root@hdss7-21 ~]# cd /opt/kubernetes/server/bin/cert/[root@hdss7-21 cert]# scp 10.4.7.200:/opt/certs/kube-proxy-client-key.pem .[root@hdss7-21 cert]# scp 10.4.7.200:/opt/certs/kube-proxy-client.pem .

在conf文件夹下创建配置

(只做一次，然后将kube-proxy.kubeconfig拷贝至各个node节点)

[root@hdss7-21 cert]# cd /opt/kubernetes/server/bin/conf# --server=此IP地址是keeplive的VIP地址，注意修改[root@hdss7-21 conf]# kubectl config set-cluster myk8s \ --certificate-authority=/opt/kubernetes/server/bin/cert/ca.pem \ --embed-certs=true \ --server=\ --kubeconfig=kube-proxy.kubeconfig[root@hdss7-21 conf]# lsaudit.yaml k8s-node.yaml kubelet.kubeconfig kube-proxy.kubeconfig[root@hdss7-21 conf]# kubectl config set-credentials kube-proxy \ --client-certificate=/opt/kubernetes/server/bin/cert/kube-proxy-client.pem \ --client-key=/opt/kubernetes/server/bin/cert/kube-proxy-client-key.pem \ --embed-certs=true \ --kubeconfig=kube-proxy.kubeconfig[root@hdss7-21 conf]# kubectl config set-context myk8s-context \ --cluster=myk8s \ --user=kube-proxy \ --kubeconfig=kube-proxy.kubeconfig[root@hdss7-21 conf]# kubectl config use-context myk8s-context --kubeconfig=kube-proxy.kubeconfig

第一台node节点部署完成后，将生成的配置文件拷贝至各个Node节点

[root@hdss7-22 cert]# cd /opt/kubernetes/server/bin/conf[root@hdss7-22 conf]# scp 10.4.7.21:/opt/kubernetes/server/bin/conf/kube-proxy.kubeconfig .

分别在2台主机执行

加载linux内核中的ipvs模块

– 脚本需要设置成开启自动运行

[root@hdss7-21 conf]# vi /root/ipvs.sh#!/bin/bashipvs_mods_dir="/usr/lib/modules/$(uname -r)/kernel/net/netfilter/ipvs"for i in $(ls $ipvs_mods_dir|grep -o "^[^.]*")do /sbin/modinfo -F filename $i &>/dev/null if [ $? -eq 0 ];then /sbin/modprobe $i fidone[root@hdss7-21 conf]# chmod +x /root/ipvs.sh

执行脚本

[root@hdss7-21 conf]# /root/ipvs.sh

查看内核是否加载ipvs模块

[root@hdss7-21 conf]# lsmod | grep ip_vs ip_vs_wrr 12697 0 ip_vs_wlc 12519 0 ip_vs_sh 12688 0 ip_vs_sed 12519 0 ip_vs_rr 12600 0 ip_vs_pe_sip 12740 0 nf_conntrack_sip 33860 1 ip_vs_pe_sipip_vs_nq 12516 0 ip_vs_lc 12516 0 ip_vs_lblcr 12922 0 ip_vs_lblc 12819 0 ip_vs_ftp 13079 0 ip_vs_dh 12688 0 ip_vs 145497 24 ip_vs_dh,ip_vs_lc,ip_vs_nq,ip_vs_rr,ip_vs_sh,ip_vs_ftp,ip_vs_sed,ip_vs_wlc,ip_vs_wrr,ip_vs_pe_sip,ip_vs_lblcr,ip_vs_lblcnf_nat 26787 3 ip_vs_ftp,nf_nat_ipv4,nf_nat_masquerade_ipv4nf_conntrack 133095 8 ip_vs,nf_nat,nf_nat_ipv4,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_sip,nf_conntrack_ipv4libcrc32c 12644 4 xfs,ip_vs,nf_nat,nf_conntrack

设置开机自动启动

[root@hdss7-21 ~]# vi /etc/rc.d/rc.local/root/ipvs.sh

开启开机自启动脚本功能

[root@hdss7-21 ~]# chmod +x /etc/rc.d/rc.local[root@hdss7-21 ~]# mkdir -p /usr/lib/system/system/[root@hdss7-21 ~]# vi /usr/lib/system/system/rc-local.service[Install]WantedBy=multi-user.target[root@hdss7-21 ~]# ln -s '/lib/systemd/system/rc-local.service' '/etc/systemd/system/multi-user.target.wants/rc-local.service'

开启 rc-local.service 服务：

[root@hdss7-21 ~]# systemctl start rc-local.service[root@hdss7-21 ~]# systemctl enable rc-local.service[root@localhost conf]# systemctl status rc-local.service● rc-local.service - /etc/rc.d/rc.local Compatibility Loaded: loaded (/usr/lib/systemd/system/rc-local.service; enabled; vendor preset: disabled) Active: active (exited) since 日 2020-06-28 21:42:04 CST; 10s ago6月 28 21:42:04 hdss7-22.com systemd[1]: Starting /etc/rc.d/rc.local Compatibility...6月 28 21:42:04 hdss7-22.com systemd[1]: Started /etc/rc.d/rc.local Compatibility.

创建kube-proxy启动脚本

HDSS-7-21.host.com：[root@hdss7-22 ~]# vi /opt/kubernetes/server/bin/kube-proxy.sh#!/bin/sh./kube-proxy \ --cluster-cidr 172.7.0.0/16 \ --hostname-override hdss7-21.host.com \ --proxy-mode=ipvs \ --ipvs-scheduler=nq \ --kubeconfig ./conf/kube-proxy.kubeconfig # 备注： --hostname-override hdss7-21.host.com 根据实际情况修改主机名[root@hdss7-22 ~]# chmod +x /opt/kubernetes/server/bin/kube-proxy.sh[root@hdss7-22 ~]# mkdir -p /data/logs/kubernetes/kube-proxy

创建supervisord启动脚本

# [program:kube-proxy-7-21] 根据情况修改主机名[root@hdss7-22 ~]# vi /etc/supervisord.d/kube-proxy.ini[program:kube-proxy-7-21]command=/opt/kubernetes/server/bin/kube-proxy.sh ; the program (relative uses PATH, can take args)numprocs=1 ; number of processes copies to start (def 1)directory=/opt/kubernetes/server/bin ; directory to cwd to before exec (def no cwd)autostart=true ; start at supervisord start (default: true)autorestart=true ; retstart at unexpected quit (default: true)startsecs=30 ; number of secs prog must stay running (def. 1)startretries=3 ; max # of serial start failures (default 3)exitcodes=0,2 ; 'expected' exit codes for process (default 0,2)stopsignal=QUIT ; signal used to kill process (default TERM)stopwaitsecs=10 ; max num secs to wait b4 SIGKILL (default 10)user=root ; setuid to this UNIX account to run the programredirect_stderr=true ; redirect proc stderr to stdout (default false)stdout_logfile=/data/logs/kubernetes/kube-proxy/proxy.stdout.log ; stderr log path, NONE for none; default AUTOstdout_logfile_maxbytes=64MB ; max # logfile bytes b4 rotation (default 50MB)stdout_logfile_backups=4 ; # of stdout logfile backups (default 10)stdout_capture_maxbytes=1MB ; number of bytes in 'capturemode' (default 0)stdout_events_enabled=false ; emit events on stdout writes (default false)[root@hdss7-22 ~]# supervisorctl update[root@hdss7-22 ~]# supervisorctl statuskube-proxy-7-22 RUNNING pid 6873, uptime 0:28:15[root@hdss7-22 ~]# netstat -luntp |grep kube-proxytcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN 7310/./kube-proxy tcp6 0 0 :::10256 :::* LISTEN 7310/./kube-proxy

查看ipvs是否生效

[root@hdss7-21 ~]# yum install -y ipvsadm # 只安装，不启动[root@hdss7-21 ~]# ipvsadm -LnIP Virtual Server version 1.2.1 (size=4096)Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConnTCP 192.168.0.1:443 nq -> 10.4.7.21:6443 Masq 1 0 0 -> 10.4.7.22:6443 Masq 1 0 0 # 注意：kube-proxy集群各主机启动脚本略有不同，部署其他节点注意修改[root@hdss7-21 ~]# cat /data/logs/kubernetes/kube-proxy/proxy.stdout.log

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。