ubuntu 24.04.4,使用kubeadm在线安装k8s-v1.34.10

系统:ubuntu 24.04.4

集群规划(3 节点)

master节点,主机名:k8s-master-01,ip:172.29.128.1

工作节点1:主机名:k8s-worker-01,ip:172.29.132.68 

工作节点2:主机名:k8s-worker-02,ip:172.29.132.69 

 

1, 前置统一操作(所有 3 台机器全部执行)

1. 配置主机名与 hosts 解析

1.1 设置对应主机名

# master节点执行
hostnamectl set-hostname k8s-master-01
# worker01执行
hostnamectl set-hostname k8s-worker-01
# worker02执行
hostnamectl set-hostname k8s-worker-02

1.2 统一写入 /etc/hosts(三台都执行)

cat >> /etc/hosts << EOF
172.29.128.1    k8s-master-01
172.29.132.68   k8s-worker-01
172.29.132.69   k8s-worker-02
EOF

 

2. 关闭交换分区(必须永久关闭)

# 临时关闭
swapoff -a
# 永久注释fstab交换分区
sed -i '/swap/s/^/#/' /etc/fstab
# 验证无swap输出
cat /etc/fstab | grep swap

 

3,关闭防火墙

systemctl stop ufw
systemctl disable ufw

 

如果不关闭防火墙,要放行端口

1) Master 节点放行端口:

ufw allow 6443/tcp
ufw allow 2379:2380/tcp
ufw allow 10250/tcp
ufw allow 10259/tcp
ufw allow 10257/tcp
ufw allow 30000:32767/tcp
ufw reload

2) Worker 节点放行端口:

ufw allow 10250/tcp
ufw allow 30000:32767/tcp
ufw reload

 

4. 内核网络参数转发(容器网络必备)

# 加载内核模块modprobe overlay
modprobe ip_tables
modprobe br_netfilter
modprobe ip_vs
modprobe ip_vs_rr
modprobe ip_vs_wrr
modprobe ip_vs_sh# 持久化sysctl配置
cat > /etc/sysctl.d/k8s.conf << EOF
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOFsysctl --system
# 校验输出全部为1
sysctl net.bridge.bridge-nf-call-iptables net.ipv4.ip_forward

 

4,安装依赖

apt update && apt install -y ca-certificates curl gnupg lsb-release

 

二. 安装 containerd

1,配置 containerd 源
# 阿里云镜像站docker公钥
curl -fsSL https://mirrors.aliyun.com/docker-ce/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/trusted.gpg.d/docker.gpg
# 添加阿里云docker-ce源,可以安装containerd
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/trusted.gpg.d/docker.gpg] https://mirrors.aliyun.com/docker-ce/linux/ubuntu noble stable" | tee /etc/apt/sources.list.d/docker.listapt update

 

查看 containerd.io 可用版本

apt-cache madison containerd.io

 

2,安装containerd

apt install -y containerd.io

 

3,初始化 containerd 配置 + 开启 systemd cgroup(K8s 强制要求)

# 生成默认配置文件
containerd config default > /etc/containerd/config.toml# 修改配置:systemd_cgroup=true、pause镜像适配1.34版本
sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml
sed -i 's#sandbox_image = "registry.k8s.io/pause:3.8"#sandbox_image = "registry.k8s.io/pause:3.10"#g' /etc/containerd/config.toml# 重启并设置开机自启
systemctl restart containerd
systemctl enable containerd
systemctl status containerd

 

三,安装 Kubernetes v1.34.10 工具链(kubeadm/kubelet/kubectl)

1,添加 K8s 官方 APT 源
# 导入k8s密钥
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.34/deb/Release.key | gpg --dearmor -o /usr/share/keyrings/kubernetes-apt-keyring.gpg

# 写入apt源
echo 'deb [signed-by=/usr/share/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.34/deb/ /' | tee /etc/apt/sources.list.d/kubernetes.listapt update

 

重要避坑:
K8s 源是分支锁定
1) /v1.34/deb/ = 只能装 1.34.x;想要 1.35 必须改源地址为 /v1.35/deb/,不能一个源同时看到多个大版本。
2) containerd.io 版本 和 K8s 版本没有强制绑定,但有兼容矩阵,kubeadm 1.34 推荐 containerd ≥1.7;生产建议 2.0+,推荐 2.3.x。

2,查看源里有哪些包版本

# 查询kubeadm所有可用版本
apt-cache madison kubeadm
# kubectl
apt-cache madison kubectl
# kubelet
apt-cache madison kubelet

 

输出版本:

root@k8s-master-01:/data/k8s-app-deploy# apt-cache madison kubeadmkubeadm | 1.34.10-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.9-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.8-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.7-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.6-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.5-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.4-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.3-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.2-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.1-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packageskubeadm | 1.34.0-1.1 | https://pkgs.k8s.io/core:/stable:/v1.34/deb  Packages

 

3,锁定版本安装 v1.34.10

K8S_VERSION="1.34.10-1.1"
apt install -y kubelet=${K8S_VERSION} kubeadm=${K8S_VERSION} kubectl=${K8S_VERSION}# 锁定版本防止自动升级
apt-mark hold kubelet kubeadm kubectl
# 校验版本
kubeadm version
kubectl version --client

 

四,master节点初始化集群

 1,kubeadm init 命令(适配 v1.34.10)

kubeadm init \
--apiserver-advertise-address=172.29.128.1 \
--kubernetes-version=v1.34.10 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=192.168.0.0/16 \
--cri-socket unix:///run/containerd/containerd.sock \
--image-repository registry.aliyuncs.com/google_containers  #关键,走国内镜像源

注意:k8s-v1.22.x及以前的版本,镜像地址用:registry.aliyuncs.com/k8sxio

参数说明

  1. --apiserver-advertise-address:master 本机内网 IP
  2. pod-network-cidr=192.168.0.0/16:适配 Calico 标准网段
  3. cri-socket 指定 containerd 套接字,避免默认 docker 适配报错

如果初始化master节点失败,看kubelet日志

journalctl -u kubelet -f

 

2, 配置 kubectl 普通用户权限(master 执行)

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
# 验证集群状态(此时节点NotReady,未装CNI)
kubectl get nodes

 

3,如果初化集群失改,先导出镜像,再初始化

1)执行kubeadm init之前,可以先把kubeadm镜像拉下来:

kubeadm config images pull \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.34.10

或者配置 containerd 全局镜像加速(一劳永逸,无需每次加参数)

编辑 /etc/containerd/config.toml,添加 registry.k8s.io 镜像转发:

[plugins."io.containerd.grpc.v1.cri".registry.mirrors][plugins."io.containerd.grpc.v1.cri".registry.mirrors."registry.k8s.io"]endpoint = ["https://registry.aliyuncs.com/google_containers"]

配置全局镜像转发后,重启containerd

systemctl restart containerd

 

2)备选国内镜像

中科大:mirrors.ustc.edu.cn/google_containers

kubeadm config images pull --image-repository mirrors.ustc.edu.cn/google_containers --kubernetes-version v1.34.10

华为云:swr.cn-north-4.myhuaweicloud.com/ddn-k8s

ubeadm config images pull --image-repository swr.cn-north-4.myhuaweicloud.com/ddn-k8s --kubernetes-version v1.34.10

 

3)如果安装的时候镜像拉取失败,导出镜像,上传到所有节点并导入

#导出镜像
kubeadm config images pull \
--image-repository registry.aliyuncs.com/google_containers \
--kubernetes-version v1.34.10#镜像改名
nerdctl tag registry.aliyuncs.com/google_containers/etcd:3.6.5-0   registry.k8s.io/etcd:3.6.5-0
nerdctl tag registry.aliyuncs.com/google_containers/pause:3.10.1  registry.k8s.io/pause:3.10.1
nerdctl tag registry.aliyuncs.com/google_containers/coredns:v1.12.1  registry.k8s.io/coredns/coredns:v1.12.1
nerdctl tag registry.aliyuncs.com/google_containers/kube-proxy:v1.34.10  registry.k8s.io/kube-proxy:v1.34.10
nerdctl tag registry.aliyuncs.com/google_containers/kube-scheduler:v1.34.10  registry.k8s.io/kube-scheduler:v1.34.10
nerdctl tag registry.aliyuncs.com/google_containers/kube-controller-manager:v1.34.10  registry.k8s.io/kube-controller-manager:v1.34.10
nerdctl tag registry.aliyuncs.com/google_containers/kube-apiserver:v1.34.10  registry.k8s.io/kube-apiserver:v1.34.10#导出镜像
nerdctl save -o k8s-1.34.10-images.tar \
registry.k8s.io/kube-apiserver:v1.34.10 \
registry.k8s.io/kube-controller-manager:v1.34.10 \
registry.k8s.io/kube-scheduler:v1.34.10 \
registry.k8s.io/kube-proxy:v1.34.10 \
registry.k8s.io/etcd:3.6.5-0 \
registry.k8s.io/coredns/coredns:v1.12.1 \
registry.k8s.io/pause:3.10.1 \#上传到所有节点,并导入
nerdctl -n k8s.io load -i k8s-1.34.10-images.tar

再执行kubeadm init,去掉 --image-repository 参数

kubeadm init \
--apiserver-advertise-address=172.29.128.1 \
--kubernetes-version=v1.34.10 \
--service-cidr=10.96.0.0/12 \
--pod-network-cidr=192.168.0.0/16 \
--cri-socket unix:///run/containerd/containerd.sock

 

5,如果安装失败,先清除安装的残留缓存,再重新初始化

kubeadm reset -f
rm -rf /etc/kubernetes /var/lib/kubelet /var/lib/etcd
rm -rf /var/run/kubernetes
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
ipvsadm -C

 

6,安装成功输出:

Your Kubernetes control-plane has initialized successfully!To start using your cluster, you need to run the following as a regular user:mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:export KUBECONFIG=/etc/kubernetes/admin.confYou should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:kubeadm join 172.29.128.1:6443 --token oy1nul.n1xz32q53qds022t \--discovery-token-ca-cert-hash sha256:23f8cc787eb0dd830c05003974eaba3460ac05fe72b000fe5fbec755d572b48e

 

五, worker 节点加入集群

1,保存上面输出的 join 命令

kubeadm join 172.29.128.1:6443 --token xxx --discovery-token-ca-cert-hash sha256:xxx

 

2,如果忘记了join命令,重新生成token(在master 执行)

# 重新生成join命令
kubeadm token create --print-join-command

 输出:

kubeadm join 172.29.128.1:6443 --token xxxxx --discovery-token-ca-cert-hash sha256:xxxx

 

3, 工作节点加入集群

所有工作节点执行上面的 kubeadm join命令加入集群

 

4,Master 节点验证集群完整状态

# 查看所有节点
kubectl get nodes
# 查看集群组件健康状态
kubectl get cs
# 查看所有系统pod
kubectl get pods -n kube-system

 

六,安装calico网络 

1 部署 Calico CNI 网络插件(master 执行)

Calico v3.32.1 完美兼容 K8s v1.34

curl -O calico-v3.32.1.yaml https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/calico.yaml

2,替换为国内镜像

原镜像:quay.io/calico/xxx
替换为:m.daocloud.io/quay.io/calico/xxx

sed -i 's|quay.io/calico|m.daocloud.io/quay.io/calico|g' calico-v3.32.1.yaml

查看镜像配置

grep "image:" calico-v3.32.1.yaml
#只显示镜像名
awk '/image:\s*/{print $2}' calico-v3.32.1.yaml | sort -u

输出:

m.daocloud.io/quay.io/calico/cni:v3.32.1
m.daocloud.io/quay.io/calico/kube-controllers:v3.32.1
m.daocloud.io/quay.io/calico/node:v3.32.1

3,部署calico

kubectl apply -f calico-v3.32.1.yaml
# 等待calico全部pod就绪
kubectl get pods -n kube-system -w
# 全部ready后节点状态变为Ready
kubectl get nodes

其他配套清单(按需下载)

# 带Typha(50节点以上集群)
curl -O calico-typha-v3.32.1.yaml https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/calico-typha.yaml
# canal(calico网络策略+flannel底层)
curl -O canal-v3.32.1.yaml https://raw.githubusercontent.com/projectcalico/calico/v3.32.1/manifests/canal.yaml

 

七,运维命令

1. Master 节点允许调度 Pod(单 master 无多余机器时)

kubectl taint nodes k8s-master-01 node-role.kubernetes.io/control-plane:NoSchedule-

 

2. 配置 kubectl 命令自动补全

apt install bash-completion -y
echo 'source <(kubectl completion bash)' >> ~/.bashrc
echo 'alias k=kubectl' >> ~/.bashrc
echo 'complete -F __start_kubectl k' >> ~/.bashrc
source ~/.bashrc

 

3. 常见故障排查命令

# kubelet日志
journalctl -u kubelet -f
# containerd日志
journalctl -u containerd
# 查看集群事件
kubectl get events -w
# 重置集群(重装场景)
kubeadm reset -f
rm -rf /var/lib/kubelet /etc/kubernetes