云计算-使用Kubeadm在阿里云搭建单Master多Node的K8S

云计算-使用Kubeadm在阿里云搭建单Master多Node的K8S

云计算-使用Kubeadm在阿里云搭建单Master多Node的K8S

​阿里云的机器搭建 ​

进入阿里云选择云服务器 ECS

单击管理控制台

创建我的ECS

开始创建，然后选择​按量付费​，其他可以自行选择

​机器环境配置 ​

​本次搭建三台机器，一台Master,两台Node,默认情况下是在三台机器上都操作，说明的是在一台master机器上操作,在阿里云上搭建的时候，ip使用阿里的私有ip,只有浏览器访问的时候使用IP​

安装个可能用到的基础包

yum install -y openssh-clients openssh-clients ntpdate yum-utils device-mapper-persistent-data lvm2 wget net-tools nfs-utils lrzsz gcc gcc-c++ make cmake libxml2-devel openssl-devel curl curl-devel unzip sudo ntp libaio-devel wget vim ncurses-devel autoconf automake zlib-devel python-devel epel-release openssh-server socat ipvsadm conntrack ntpdate telnet ipvsadm

修改主机hosts文件

修改每台机器的/etc/hosts文件，主要添加三行

172.17.75.162 kubeadm001 kubeadm001172.17.75.160 kubeadm002 kubeadm002172.17.75.161 kubeadm003 kubeadm003

主机间免密登录

​主要是kubeadm001和其他机器之间的免密登录​

输入ssh-keygen,任何一路回车，不输入任何密码

ssk-keygen

把本地生成的密钥文件拷贝到远程主机kubeadm002和kubeadm003上面

ssh-copy-id kubeadm002

ssh-copy-id kubeadm003

关闭交换区，提升性能

swapoff -a

为什么关闭交换区？

Swap是交换区，在机器内存不够的情况下，就会适应Swap，但是Swap的性能不是很好，K8S设计的时候为了提升性能，默认不允许使用Swap分区。Kubeadm初始化的时候会检测Swap是否关闭，如果没有关闭，初始化会失败。​如果不想关闭交换分区，安装k8s的时候可以指定-- ignore -preflight-errors=Swap​解决。

修改机器内核参数

modprobe br_netfilter

echo "modprobe br_netfilter" >> /etc/profile

cat > /etc/sysctl.d/k8s.conf <

sysctl -p /etc/sysctl.d/k8s.conf

关闭防火墙

systemctl stop firewalld && systemctl disable firewalld && systemctl status firewalld

如果你不习惯使用firewalld,可以安装​​iptables​​，在三台机器上做如下操作

安装​​iptables​​基础包

yum install iptables-services -y

关闭​​iptables​​

service iptables stop && systemctl disable iptabl

清空防火墙规则

iptables -F

关闭selinux

使用​​getenforce ​​查看selinux是否关闭

[root@kubeadm003 ~]# getenforce

​如果结果不是​Disabled,使用以下命令关闭selinux

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

​注​：修改selinux需要重启机器。

时间同步

把三台机器上的时间同网络时间进行同步

将机器时间同网络时间进行同步

ntpdate cn.pool.ntp.org

做一个定时任务将机器时间每小时和网络时间同步一次

crontab -e

添加定时任务

* */1 * * * /usr/sbin/ntpdate cn.pool.ntp.org

重启定时任务

systemctl start crond

开启ipvs

在​​/etc/sysconfig/modules​​目录下添加ipvs.modules文件内容如下

vi /etc/sysconfig/modules/ipvs.modules

#!/bin/bashipvs_modules="ip_vs ip_vs_lc ip_vs_wlc ip_vs_rr ip_vs_wrr ip_vs_lblc ip_vs_lblcr ip_vs_dh ip_vs_sh ip_vs_nq ip_vs_sed ip_vs_ftp nf_conntrack"for kernel_module in ${ipvs_modules}; do /sbin/modinfo -F filename ${kernel_module} > /dev/null 2>&1 if [ 0 -eq 0 ]; then /sbin/modprobe ${kernel_module} fidone

将​​ipvs.modules​​添加为可执行文件，并执行文件和查看执行情况

chmod +x /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep ip_vs

什么是ipvs

ipvs(IP Virtual Server)实现的是传输层负载均衡，是Linux内核的一部分。ipvs运行在宿主机上，充当负载均衡器。

配置K8S组件需要的repo源

使用vi 添加/etc/yum.repos.d/kubernetes.repo

vi /etc/yum.repos.d/kubernetes.repo

文件内容如下

[kubernetes] name=Kubernetes baseurl=enabled=1 gpgcheck=0

安装 docker 服务

旧版本的卸载

在安装新的docker的时候，机器可能在以前安装过docker旧版本，我们需要将其进行卸载：

yum remove docker \ docker-client \ docker-client-latest \ docker-common \ docker-latest \ docker-latest-logrotate \ docker-logrotate \ docker-engine

镜像仓的设置

一般我们在国内的话使用国内的镜像仓，我一般使用阿里的镜像仓

使用阿里国内源安装docker yum-config-manager \ --add-repo \ sudo sed -i 's+download.docker.com+mirrors.aliyun.com/docker-ce+' /etc/yum.repos.d/docker-ce.repo

更新并安装Docker-ce

docker-ce 社区版 , docker-ee企业版， 一般安装社区版，企业版需要授权

#更新并安装Docker-CE sudo yum makecache fastyum install -y docker-ce docker-ce-cli containerd.io

启动安装并查看Docker状态

systemctl start docker && systemctl enable docker && systemctl status docker

配置 docker 镜像加速器和驱动

vi /etc/docker/daemon.json

本文添加了多个镜像加速器，包含阿里镜像加速器、163镜像加速器等加速器

{ "registry-mirrors":[""exec-opts": ["native.cgroupdriver=systemd"]}

阿里云加速器每一个人的都不一样,添加如下

添加以后重启​​docker​​​和​​daemon-reload​​

systemctl daemon-reload && systemctl restart docker && systemctl status docker

​k8s安装 ​

安装初始化 k8s 需要的软件包

安装​​kubelet​​​、​​kubeadm​​​、​​kubectl​​

yum install -y kubelet-1.20.6 kubeadm-1.20.6 kubectl-1.20.6

将​​kubelet​​​设置为开启自启动以及查看​​kubelet​​状态

systemctl enable kubelet && systemctl status kubelet

现在看起来​​kubelet​​​还不是​​running​​状态，现在这个状态是正常的，k8s正常启动启动以后，状态就正常了

​注​：

​​kubeadm​​​:​​kubeadm​​是一个工具,用于初始化k8s集群​​kubelet​​​：​​kubelet​​​需要安装在集群的所有结点，作用是启动Pod​​​kubectl​​​:​​kubectl​​可以部署和管理应用的工具

使用kubeadm初始化k8s集群

​这个操作只需要在master节点上操作就可以了​，本次搭建k8s的master内网节点是172.17.75.165

--kubernetes-version=1.20.6   表示版本--apiserver-advertise-address=172.17.75.165  表示master节点在阿里云的内网IP--image-repository registry.aliyuncs.com/google_containers   指定镜像仓的地址

kubeadm init --kubernetes-version=1.20.6 --apiserver-advertise-address=172.17.75.165 --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --ignore-preflight-errors=SystemVerification

kubeadm默认拉取镜像的地址是k8s.grc.io，但是在国内访问效果差，所以设置为国内的镜像仓效果会更好。

如下说明在master上安装成功

配置kubectl的配置文件config就是上图标出的文件

mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/config

配置好以后查看nodes

kubectl get nodes

使用​​kubeadm token create​​命令生成添加节点的指令

kubeadm token create --print-join-command

​生成的指令如下，每次生成可能都会不同​

kubeadm join 172.17.75.165:6443 --token vsnwiz.9tdg2hklcmx26gbb --discovery-token-ca-cert-hash sha256:03e79bdddf8a15be78cfefb717275fda128de570bec60a1a332719c94bedc5c6

​我们需要将上面的​​kubeadm join​​​指令在kubeadm002和kubeadm003两台机器上运行​

再次查看nodes节点

kubectl get nodes

我们发现新添加的node节点的​ROLES​是空的，也表示它是工作节点，但是我们可以将变成work

kubectl label node kubeadm002 node-role.kubernetes.io/worker=workerkubectl label node kubeadm003 node-role.kubernetes.io/worker=worker

然后再次查看结果变为

[root@kubeadm001 ~]# kubectl get nodesNAME STATUS ROLES AGE VERSIONkubeadm001 NotReady control-plane,master 14m v1.20.6kubeadm002 NotReady worker 5m48s v1.20.6kubeadm003 NotReady worker 5m44s v1.20.6

​上面的STSTUS中的NotReady表示没有安装网卡插件​

安装k8s网络组件-Calico

​链接：提取码：1234 编写一个​​calico.yaml​​​，我将它上传到百度云盘，可以自行下载，也可以到apply -f calico.yamlkubectl apply -f get pod -n kube-system

再次查看k8s的集群状态

kubectl get nodes

我们发现现在集群的STATUS的状态变成了​Ready​,说明集群已经正常运行

测试k8s创建pod是否可以正常访问网络

创建一个busybox容器,并进入到容器内部

kubectl run busybox --image busybox:1.28 --restart=Never --rm -it busybox -- sh

然后在里面输入​​ping baidu.com​​

通过ping我们发现网络可以访问，说明Calico安装正常

测试coredns是否正常

还是上面的busybos容器,输入以下命令

nslookup kubernetes.default.svc.cluster.local

结果为以下，10.96.0.10就是我们coreDNS的clusterIP，说明coredns正常

Server: 10.96.0.10Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.localName: kubernetes.default.svc.cluster.localAddress 1: 10.96.0.1 kubernetes.default.svc.cluster.local

k8s集群中安装tomcat服务

创建一个​​tomcat.yaml​​

vi tomcat.yaml

apiVersion: v1 #pod属于k8s核心组v1kind: Pod #创建的是一个Pod资源metadata: #元数据 name: tomcat-pod #pod名字 namespace: default #pod所属的名称空间 labels: app: myapp #pod具有的标签 env: dev #pod具有的标签spec: containers: #定义一个容器，容器是对象列表，下面可以有多个name - name: tomcat-pod-test #容器的名字 ports: - containerPort: 8080 image: tomcat:8.5-jre8-alpine #容器使用的镜像 imagePullPolicy: IfNotPresent

使用kubect部署tomcat

kubectl apply -f tomcat.yaml

查看pods

[root@kubeadm001 ~]# kubectl get podsNAME READY STATUS RESTARTS AGEtomcat-pod 1/1 Running 0 58s

创建tomcat的Service，​​tomcat-service.yaml​​

vi tomcat-service.yaml

内容如下

apiVersion: v1kind: Servicemetadata: name: tomcatspec: type: NodePort ports: - port: 8080 nodePort: 30080 selector: app: myapp env: dev

运行tomcat的​​service​​

kubectl apply -f tomcat-service.yaml

查看service的状态

kubectl get svc

在浏览器中（8.134.58.124表示master节点）

​安装k8s可视化UI界面dashboard ​

创建kubernetes-dashboard.yaml文件

# Copyright 2017 The Kubernetes Authors.## Licensed under the Apache License, Version 2.0 (the "License");# you may not use this file except in compliance with the License.# You may obtain a copy of the License at## Unless required by applicable law or agreed to in writing, software# distributed under the License is distributed on an "AS IS" BASIS,# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.# See the License for the specific language governing permissions and# limitations under the License.apiVersion: v1kind: Namespacemetadata: name: kubernetes-dashboard---apiVersion: v1kind: ServiceAccountmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboard---kind: ServiceapiVersion: v1metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboardspec: ports: - port: 443 targetPort: 8443 selector: k8s-app: kubernetes-dashboard---apiVersion: v1kind: Secretmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-certs namespace: kubernetes-dashboardtype: Opaque---apiVersion: v1kind: Secretmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-csrf namespace: kubernetes-dashboardtype: Opaquedata: csrf: ""---apiVersion: v1kind: Secretmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-key-holder namespace: kubernetes-dashboardtype: Opaque---kind: ConfigMapapiVersion: v1metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard-settings namespace: kubernetes-dashboard---kind: RoleapiVersion: rbac.authorization.k8s.io/v1metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboardrules: # Allow Dashboard to get, update and delete Dashboard exclusive secrets. - apiGroups: [""] resources: ["secrets"] resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"] verbs: ["get", "update", "delete"] # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map. - apiGroups: [""] resources: ["configmaps"] resourceNames: ["kubernetes-dashboard-settings"] verbs: ["get", "update"] # Allow Dashboard to get metrics. - apiGroups: [""] resources: ["services"] resourceNames: ["heapster", "dashboard-metrics-scraper"] verbs: ["proxy"] - apiGroups: [""] resources: ["services/proxy"] resourceNames: ["heapster", """dashboard-metrics-scraper", " verbs: ["get"]---kind: ClusterRoleapiVersion: rbac.authorization.k8s.io/v1metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboardrules: # Allow Metrics Scraper to get metrics from the Metrics server - apiGroups: ["metrics.k8s.io"] resources: ["pods", "nodes"] verbs: ["get", "list", "watch"]---apiVersion: rbac.authorization.k8s.io/v1kind: RoleBindingmetadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboardroleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: kubernetes-dashboardsubjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard---apiVersion: rbac.authorization.k8s.io/v1kind: ClusterRoleBindingmetadata: name: kubernetes-dashboardroleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: kubernetes-dashboardsubjects: - kind: ServiceAccount name: kubernetes-dashboard namespace: kubernetes-dashboard---kind: DeploymentapiVersion: apps/v1metadata: labels: k8s-app: kubernetes-dashboard name: kubernetes-dashboard namespace: kubernetes-dashboardspec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: kubernetes-dashboard template: metadata: labels: k8s-app: kubernetes-dashboard spec: containers: - name: kubernetes-dashboard image: kubernetesui/dashboard:v2.0.0-beta8 imagePullPolicy: IfNotPresent ports: - containerPort: 8443 protocol: TCP args: - --auto-generate-certificates - --namespace=kubernetes-dashboard # Uncomment the following line to manually specify Kubernetes API server Host # If not specified, Dashboard will attempt to auto discover the API server and connect # to it. Uncomment only if the default does not work. # - --apiserver-host= volumeMounts: - name: kubernetes-dashboard-certs mountPath: /certs # Create on-disk volume to store exec logs - mountPath: /tmp name: tmp-volume livenessProbe: scheme: HTTPS path: / port: 8443 initialDelaySeconds: 30 timeoutSeconds: 30 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 volumes: - name: kubernetes-dashboard-certs secret: secretName: kubernetes-dashboard-certs - name: tmp-volume emptyDir: {} serviceAccountName: kubernetes-dashboard nodeSelector: "beta.kubernetes.io/os": linux # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule---kind: ServiceapiVersion: v1metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboardspec: ports: - port: 8000 targetPort: 8000 selector: k8s-app: dashboard-metrics-scraper---kind: DeploymentapiVersion: apps/v1metadata: labels: k8s-app: dashboard-metrics-scraper name: dashboard-metrics-scraper namespace: kubernetes-dashboardspec: replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: k8s-app: dashboard-metrics-scraper template: metadata: labels: k8s-app: dashboard-metrics-scraper annotations: seccomp.security.alpha.kubernetes.io/pod: 'runtime/default' spec: containers: - name: dashboard-metrics-scraper image: kubernetesui/metrics-scraper:v1.0.1 imagePullPolicy: IfNotPresent ports: - containerPort: 8000 protocol: TCP livenessProbe: scheme: HTTP path: / port: 8000 initialDelaySeconds: 30 timeoutSeconds: 30 volumeMounts: - mountPath: /tmp name: tmp-volume securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsUser: 1001 runAsGroup: 2001 serviceAccountName: kubernetes-dashboard nodeSelector: "beta.kubernetes.io/os": linux # Comment the following tolerations if Dashboard must not be deployed on master tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule volumes: - name: tmp-volume emptyDir: {}

安装 kubernetes-dashboard

kubectl apply -f kubernetes-dashboard.yaml

查看dashboard的pod状态(需要等待一点时间)

kubectl get pods -n kubernetes-dashboard

查看结果为如下，说明安装成功

NAME READY STATUS RESTARTS AGEdashboard-metrics-scraper-778b77d469-r6cwm 1/1 Running 0 2mkubernetes-dashboard-648b6964bf-gnbw4 1/1 Running 0 2m

查看dashboard的前端service

kubectl get svc -n kubernetes-dashboard

结果为

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEdashboard-metrics-scraper ClusterIP 10.97.73.249 8000/TCP 2m57skubernetes-dashboard ClusterIP 10.106.69.73 443/TCP 2m57s

我们发现kubernetes-dashboard的TYPE状态ClusterIP,我们需要将其修改为​NodePort​状态,

kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard

再次查看 kubernetes-dashboard的svc

[root@kubeadm001 ~]# kubectl get svc -n kubernetes-dashboardNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEdashboard-metrics-scraper ClusterIP 10.97.73.249 8000/TCP 7m31skubernetes-dashboard NodePort 10.106.69.73 443:31284/TCP 7m31s

查看到端口是31284,然后使用任何一个node:31284进行查看

可看到出现了dashboard界面

通过token令牌访问dashboard

创建管理员token，具有查看任何空间的权限，可以管理所有资源对象

kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole=cluster-admin --serviceaccount=kubernetes-dashboard:kubernetes-dashboard

查看kubernetes-dashboard名称空间下的secret

kubectl get secret -n kubernetes-dashboard

找到对应的带有oken的kubernetes-dashboard-token-66r22 ，​每次kubernetes-dashboard-token的name不同​

kubectl describe secret kubernetes-dashboard-token-9mwrn -n kubernetes-dashboard

通过 kubernetes-dashboard 创建容器

打开 kubernetes 的 dashboard 界面点开右上角红色箭头标注的 “+”，如下图所示：

选择Create from form

​在dashboard的左侧选择Services​

​上图可看到刚才创建的nginx的service在宿主机映射的端口是32545，在浏览器访问：http://8.134.74.86:32545/​

版权声明：本文内容由网络用户投稿，版权归原作者所有，本站不拥有其著作权，亦不承担相应法律责任。如果您发现本站中有涉嫌抄袭或描述失实的内容，请联系我们jiasou666@gmail.com 处理，核实后本网站将在24小时内删除侵权内容。