资料库 – 中文知识库

Kubernetes二进制部署：详解K8s集群从零搭建全过程

—

by

admin
in

引言
Kubernetes（简称K8s）作为现代容器编排的基石，以其强大的自动化部署、扩缩和管理能力，赢得了无数开发者和运维工程师的青睐。本文将详细介绍如何通过二进制方式部署一个高可用的Kubernetes集群，涵盖从环境初始化到集群测试的全过程。无论你是初学者还是有一定经验的运维人员，这篇文章都将为你提供宝贵的参考。
一、Kubernetes项目架构概述
在开始部署之前，了解Kubernetes的基本架构是至关重要的。Kubernetes采用Master-Slave主从架构，其中Master节点负责集群管理和资源调度，Node节点负责运行Pod，Pod内可以包含多个容器。
1.1 集群IP地址规划
Master节点：192.168.10.10, 192.168.10.11, 192.168.10.12
Node节点：192.168.10.20, 192.168.10.21
etcd节点：192.168.10.30, 192.168.10.31, 192.168.10.32
VIP（虚拟IP）：192.168.10.100
1.2 安装软件列表
操作系统：CentOS 7.6
Docker CE
etcd
Kubernetes v1.20
Nginx
Keepalived
二、环境初始化
在开始部署之前，需要对所有节点进行初始化配置。
2.1 设置静态IP和主机名
# 设置静态IP
nmcli con mod eth0 ipv4.addresses 192.168.10.10/24
nmcli con mod eth0 ipv4.gateway 192.168.10.1
nmcli con mod eth0 ipv4.dns 8.8.8.8
nmcli con up eth0

# 设置主机名
hostnamectl set-hostname master01
2.2 编辑hosts文件
cat << EOF >> /etc/hosts
192.168.10.10 master01
192.168.10.11 master02
192.168.10.12 master03
192.168.10.20 node01
192.168.10.21 node02
192.168.10.30 etcd01
192.168.10.31 etcd02
192.168.10.32 etcd03
EOF
2.3 实现主机间无密码登录
ssh-keygen -t rsa -b 2048 -N “” -f ~/.ssh/id_rsa
ssh-copy-id root@master01
ssh-copy-id root@master02
ssh-copy-id root@master03
ssh-copy-id root@node01
ssh-copy-id root@node02
ssh-copy-id root@etcd01
ssh-copy-id root@etcd02
ssh-copy-id root@etcd03
2.4 关闭firewalld和selinux
systemctl stop firewalld
systemctl disable firewalld
setenforce 0
sed -i ‘s/^SELINUX=enforcing$/SELINUX=disabled/’ /etc/selinux/config
2.5 关闭swap分区
swapoff -a
sed -i ‘/swap/d’ /etc/fstab
2.6 调整内核参数
cat << EOF >> /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF
sysctl -p
2.7 设置阿里云repo源和配置时间同步
yum install -y ntp
ntpdate ntp1.aliyun.com
crontab -e
*/5 * * * * ntpdate ntp1.aliyun.com
三、安装Docker引擎
3.1 安装iptables和开启ipvs
yum install -y iptables ipvsadm
modprobe ip_vs
3.2 安装Docker CE
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager –add-repo https://download.docker.com/linux/centos/docker-ce.repo
yum install -y docker-ce
systemctl start docker
systemctl enable docker
3.3 配置docker镜像加速器
mkdir -p /etc/docker
cat << EOF > /etc/docker/daemon.json
{
“registry-mirrors”: [“https://mirror.ccs.tencentyun.com”]
}
EOF
systemctl restart docker
四、搭建etcd集群
4.1 配置工作目录
mkdir -p /opt/etcd/{bin,cfg,ssl}
4.2 安装证书签发工具cfssl
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
chmod +x cfssl_linux-amd64
mv cfssl_linux-amd64 /usr/local/bin/cfssl

wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
chmod +x cfssljson_linux-amd64
mv cfssljson_linux-amd64 /usr/local/bin/cfssljson

wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
chmod +x cfssl-certinfo_linux-amd64
mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo
4.3 配置ca证书
cat << EOF | cfssl gencert -initca - | cfssljson -bare /opt/etcd/ssl/ca { "CN": "etcd CA", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF 4.4 生成etcd证书 cat << EOF | cfssl gencert \ -ca=/opt/etcd/ssl/ca.pem \ -ca-key=/opt/etcd/ssl/ca-key.pem \ -config=/opt/etcd/ssl/ca-config.json \ -hostname="etcd01,etcd02,etcd03,192.168.10.30,192.168.10.31,192.168.10.32" \ - | cfssljson -bare /opt/etcd/ssl/etcd { "CN": "etcd", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "CN", "L": "Beijing", "ST": "Beijing" } ] } EOF 4.5 部署etcd集群 cat << EOF > /opt/etcd/cfg/etcd.conf
#[Member]
ETCD_NAME=”etcd01″
ETCD_DATA_DIR=”/var/lib/etcd/default.etcd”
ETCD_LISTEN_PEER_URLS=”https://192.168.10.30:2380″
ETCD_LISTEN_CLIENT_URLS=”https://192.168.10.30:2379″

#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS=”https://192.168.10.30:2380″
ETCD_ADVERTISE_CLIENT_URLS=”https://192.168.10.30:2379″
ETCD_INITIAL_CLUSTER=”etcd01=https://192.168.10.30:2380,etcd02=https://192.168.10.31:2380,etcd03=https://192.168.10.32:2380″
ETCD_INITIAL_CLUSTER_STATE=”new”
ETCD_INITIAL_CLUSTER_TOKEN=”etcd-cluster”
EOF

cat << EOF > /lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd.conf
ExecStart=/opt/etcd/bin/etcd \
–name=\$ETCD_NAME \
–data-dir=\$ETCD_DATA_DIR \
–listen-peer-urls=\$ETCD_LISTEN_PEER_URLS \
–listen-client-urls=\$ETCD_LISTEN_CLIENT_URLS \
–advertise-client-urls=\$ETCD_ADVERTISE_CLIENT_URLS \
–initial-advertise-peer-urls=\$ETCD_INITIAL_ADVERTISE_PEER_URLS \
–initial-cluster=\$ETCD_INITIAL_CLUSTER \
–initial-cluster-state=\$ETCD_INITIAL_CLUSTER_STATE \
–initial-cluster-token=\$ETCD_INITIAL_CLUSTER_TOKEN
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start etcd
systemctl enable etcd
五、安装Kubernetes组件
5.1 下载二进制包
wget https://dl.k8s.io/v1.20.0/kubernetes-server-linux-amd64.tar.gz
tar -zxvf kubernetes-server-linux-amd64.tar.gz
cd kubernetes/server/bin
cp kube-apiserver kube-controller-manager kube-scheduler kubectl /usr/local/bin/
5.2 部署apiserver
cat << EOF > /opt/kubernetes/cfg/kube-apiserver.conf
KUBE_APISERVER_OPTS=”–logtostderr=false \\
–v=2 \\
–log-dir=/opt/kubernetes/logs \\
–etcd-servers=https://192.168.10.30:2379,https://192.168.10.31:2379,https://192.168.10.32:2379 \\
–bind-address=192.168.10.10 \\
–secure-port=6443 \\
–advertise-address=192.168.10.10 \\
–allow-privileged=true \\
–service-cluster-ip-range=10.0.0.0/24 \\
–enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction \\
–authorization-mode=RBAC,Node \\
–enable-bootstrap-token-auth=true \\
–token-auth-file=/opt/kubernetes/cfg/token.csv \\
–service-node-port-range=30000-32767 \\
–kubelet-client-certificate=/opt/kubernetes/ssl/server.pem \\
–kubelet-client-key=/opt/kubernetes/ssl/server-key.pem \\
–tls-cert-file=/opt/kubernetes/ssl/server.pem \\
–tls-private-key-file=/opt/kubernetes/ssl/server-key.pem \\
–client-ca-file=/opt/kubernetes/ssl/ca.pem \\
–service-account-key-file=/opt/kubernetes/ssl/ca-key.pem \\
–etcd-cafile=/opt/etcd/ssl/ca.pem \\
–etcd-certfile=/opt/etcd/ssl/etcd.pem \\
–etcd-keyfile=/opt/etcd/ssl/etcd-key.pem”
EOF

cat << EOF > /lib/systemd/system/kube-apiserver.service
[Unit]
Description=Kubernetes API Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-apiserver.conf
ExecStart=/usr/local/bin/kube-apiserver \$KUBE_APISERVER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start kube-apiserver
systemctl enable kube-apiserver
5.3 部署kubectl
cat << EOF > /opt/kubernetes/cfg/kube-controller-manager.conf
KUBE_CONTROLLER_MANAGER_OPTS=”–logtostderr=false \\
–v=2 \\
–log-dir=/opt/kubernetes/logs \\
–master=127.0.0.1:8080 \\
–bind-address=127.0.0.1 \\
–allocate-node-cidrs=true \\
–cluster-cidr=10.244.0.0/16 \\
–service-cluster-ip-range=10.0.0.0/24 \\
–cluster-signing-cert-file=/opt/kubernetes/ssl/ca.pem \\
–cluster-signing-key-file=/opt/kubernetes/ssl/ca-key.pem \\
–service-account-private-key-file=/opt/kubernetes/ssl/ca-key.pem \\
–root-ca-file=/opt/kubernetes/ssl/ca.pem \\
–leader-elect=true”
EOF

cat << EOF > /lib/systemd/system/kube-controller-manager.service
[Unit]
Description=Kubernetes Controller Manager
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-controller-manager.conf
ExecStart=/usr/local/bin/kube-controller-manager \$KUBE_CONTROLLER_MANAGER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start kube-controller-manager
systemctl enable kube-controller-manager
5.4 部署kube-scheduler
cat << EOF > /opt/kubernetes/cfg/kube-scheduler.conf
KUBE_SCHEDULER_OPTS=”–logtostderr=false \\
–v=2 \\
–log-dir=/opt/kubernetes/logs \\
–master=127.0.0.1:8080 \\
–leader-elect=true”
EOF

cat << EOF > /lib/systemd/system/kube-scheduler.service
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-scheduler.conf
ExecStart=/usr/local/bin/kube-scheduler \$KUBE_SCHEDULER_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start kube-scheduler
systemctl enable kube-scheduler
六、部署Kubernetes工作节点
6.1 安装kubelet
cat << EOF > /opt/kubernetes/cfg/kubelet.conf
KUBELET_OPTS=”–logtostderr=false \\
–v=2 \\
–log-dir=/opt/kubernetes/logs \\
–hostname-override=node01 \\
–kubeconfig=/opt/kubernetes/cfg/kubelet.kubeconfig \\
–bootstrap-kubeconfig=/opt/kubernetes/cfg/bootstrap.kubeconfig \\
–config=/opt/kubernetes/cfg/kubelet-config.yml \\
–cert-dir=/opt/kubernetes/ssl \\
–pod-infra-container-image=lxyang/k8s-pause:3.1″
EOF

cat << EOF > /lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
EnvironmentFile=/opt/kubernetes/cfg/kubelet.conf
ExecStart=/usr/local/bin/kubelet \$KUBELET_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start kubelet
systemctl enable kubelet
6.2 安装kube-proxy
cat << EOF > /opt/kubernetes/cfg/kube-proxy.conf
KUBE_PROXY_OPTS=”–logtostderr=false \\
–v=2 \\
–log-dir=/opt/kubernetes/logs \\
–config=/opt/kubernetes/cfg/kube-proxy-config.yml”
EOF

cat << EOF > /lib/systemd/system/kube-proxy.service
[Unit]
Description=Kubernetes Kube-proxy
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
EnvironmentFile=/opt/kubernetes/cfg/kube-proxy.conf
ExecStart=/usr/local/bin/kube-proxy \$KUBE_PROXY_OPTS
Restart=on-failure

[Install]
WantedBy=multi-user.target
EOF

systemctl daemon-reload
systemctl start kube-proxy
systemctl enable kube-proxy
七、集群测试
7.1 检查集群状态
kubectl get cs
kubectl get nodes
7.2 部署一个测试应用
kubectl create deployment nginx –image=nginx
kubectl expose deployment nginx –port=80 –type=NodePort
7.3 验证应用运行状态
kubectl get pods
kubectl get svc
curl :
八、高可用配置
8.1 安装Keepalived
yum install -y keepalived
cat << EOF > /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
router_id LVS_DEVEL
}

vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.10.100
}
}
EOF

systemctl start keepalived
systemctl enable keepalived
8.2 安装Nginx
yum install -y nginx
cat << EOF > /etc/nginx/nginx.conf
worker_processes 1;

events {
worker_connections 1024;
}

http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;

upstream k8s-apiserver {
server 192.168.10.10:6443;
server 192.168.10.11:6443;
server 192.168.10.12:6443;
}

server {
listen 80;
server_name localhost;

location / {
proxy_pass http://k8s-apiserver;
proxy_set_header Host \$host;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto \$scheme;
}
}
}
EOF

systemctl start nginx
systemctl enable nginx
九、总结
通过本文的详细步骤，我们已经成功搭建了一个高可用的Kubernetes集群。从环境初始化到各个组件的安装配置，每一步都经过精心设计和详细说明。希望这篇文章能为你在实际操作中提供有力的参考，让你在Kubernetes的世界里游刃有余。
十、参考资料
Kubernetes官方文档：
https://kubernetes.io/docs/
Docker官方文档：
https://docs.docker.com/
etcd官方文档：
https://etcd.io/docs/
希望你在Kubernetes的部署和管理之路上越走越远，探索更多的可能性！