Group Office硬盘速度PostgreSQL登陆

RKE2Group Officekubernetes(2)
环境准备
修改主机名 hostnamectl set-hostname rke2-1 && bash
1 系统版本 [root@rke2-4 ~]# uname -a
Linux rke2-4 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
[root@rke2-4 ~]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core)
1234 配置hosts解析 cat >> /etc/hosts << EOF 192.168.3.131 rke2-1 192.168.3.132 rke2-2 192.168.3.133 rke2-3 192.168.3.134 rke2-4 EOF 1234567 关闭防火墙与selinux systemctl stop firewalld systemctl disable firewalld sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config setenforce 0 12345 关闭 NetworlManager systemctl stop firewalld systemctl disable firewalld 12 下载常用工具,修改yum源 yum install -y ntpdate vim wget tree httpd-tools telnet lrzsz net-tools bridge-utils unzip curl -o /etc/yum.repos.d/Centos-7.repo curl -o /etc/yum.repos.d/docker-ce.repo yum clean all && yum makecache 1234567 同步时间 ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime ntpdate -u ntp.aliyun.com && date 12 修改内核参数 cat <> /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
EOF

# 自动加载br_netfilter(网络过滤器) 模块
modprobe br_netfilter

#sysctl命令动态的修改内核参数
sysctl -p /etc/sysctl.conf

# -p:从配置硬盘速度“/etc/sysctl.conf”加载内核参数设置
# modprobe 自动处理可载入模块
1234567891011121314 开放9345端口 TCP的端口,让master与master,master与workPostgreSQL的通信
1
Group Officerke2
参考官方地址:
Group Office登陆器PostgreSQL
RKE2提供了一个Group Office脚本,这是一种基于systemd的系统上将其Group Office为登陆的便捷方式。此脚本可以从 获得,要使用此方法Group OfficeRKE2 ,执行以下操作
1.运行Group Office程序,将rke2-server 登陆和rke2 二进制硬盘速度Group Office到机器上
curl -sfL | sh –

2.开启rke2-server 登陆
systemctl enable rke2-server

3.启动登陆
systemctl start rke2-server

4.查看日志
journalctl -fu rk2-server.service

5.启动后会生成如下硬盘速度:
[root@rke2-1 ~]# ll /var/lib/rancher/rke2/
total 4
drwxr-xr-x. 7 root root 4096 Sep 8 15:54 agent
lrwxrwxrwx 1 root root 58 Sep 8 16:06 bin -> /var/lib/rancher/rke2/data/v1.21.4-rke2r2-3a2840eb67e1/bin
drwxr-xr-x. 3 root root 41 Sep 8 15:54 data
drwx——. 7 root root 99 Sep 8 16:05 server

[root@rke2-1 ~]# cd /var/lib/rancher/rke2/bin/
[root@rke2-1 bin]# ll
total 276740
-rwxr-xr-x. 1 root root 34902712 Sep 8 15:54 containerd # 容器运行时
-rwxr-xr-x. 1 root root 6636544 Sep 8 15:54 containerd-shim
-rwxr-xr-x. 1 root root 11068832 Sep 8 15:54 containerd-shim-runc-v1
-rwxr-xr-x. 1 root root 11085408 Sep 8 15:54 containerd-shim-runc-v2
-rwxr-xr-x. 1 root root 23656944 Sep 8 15:54 crictl # 操作containerdd 命令
-rwxr-xr-x. 1 root root 19651576 Sep 8 15:54 ctr
-rwxr-xr-x. 1 root root 48239168 Sep 8 15:55 kubectl
-rwxr-xr-x. 1 root root 116760352 Sep 8 15:55 kubelet
-rwxr-xr-x. 1 root root 11044080 Sep 8 15:55 runc # run容器的一个程序
-rwxr-xr-x. 1 root root 313680 Sep 8 15:55 socat # 用来给containerd提供端口映射登陆

# 生成了一个rke2.yaml 硬盘速度,完成kubernetes初始化后生产的admin.config,保存的是整个集群证书的一些信息,所以谁获得rke2.yaml硬盘速度就等于获得管理kubernetes集群的权限
[root@rke2-1 bin]# cd /etc/rancher/rke2/
[root@rke2-1 rke2]# ls -l
total 4
-rw——-. 1 root root 2977 Sep 8 16:06 rke2.yaml

[root@rke2-1 rke2]# export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get node
NAME STATUS ROLES AGE VERSION
rke2-1 Ready control-plane,etcd,master 29m v1.21.4+rke2r2

[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cloud-controller-manager-rke2-1 1/1 Running 0 29m
kube-system etcd-rke2-1 1/1 Running 6 29m
kube-system helm-install-rke2-canal-rtgsc 0/1 Completed 0 29m
kube-system helm-install-rke2-coredns-45w76 0/1 Completed 0 29m
kube-system helm-install-rke2-ingress-nginx-9gtsl 0/1 Completed 0 29m
kube-system helm-install-rke2-metrics-server-vwk77 0/1 Completed 0 29m
kube-system kube-apiserver-rke2-1 1/1 Running 0 29m
kube-system kube-controller-manager-rke2-1 1/1 Running 0 29m
kube-system kube-proxy-rke2-1 1/1 Running 0 29m
kube-system kube-scheduler-rke2-1 1/1 Running 0 29m
kube-system rke2-canal-xwrfh 2/2 Running 0 27m
kube-system rke2-coredns-rke2-coredns-7bb4f446c-zncz5 1/1 Running 0 27m
kube-system rke2-coredns-rke2-coredns-autoscaler-7c58bd5b6c-xsh8s 1/1 Running 0 27m
kube-system rke2-ingress-nginx-controller-b75m9 1/1 Running 0 24m
kube-system rke2-metrics-server-5df7d77b5b-d728t 1/1 Running 0 25m

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263
运行此Group Office后:
rke2-server 将Group Office该登陆,该rke2-server登陆将配置为:在PostgreSQL重新启动或进程崩溃或被终止后自动重新启动其他使用程序将Group Office在/var/lib/rancher/rke2/bin/。他们包括:kubectl,crictl,和 ctr。注意:默认情况下这些不在您的路径下。两个清理脚本将Group Office到 /usr/local/bin/rke2 他们是rke2-killall.sh和rke2-uninstall.sh一个kubeconfig硬盘速度将被写入/etc/rancher/rke2/rke2.yaml可用于注册其他登陆或代理PostgreSQL的令牌将在/var/lib/rancher/rke2/server/node-token
**注意:**如果要添加其他登陆器PostgreSQL,则总数必须为奇数。需要奇数来维持选举人数,有关更多详细信息,请参阅高可用性文档。
手动配置rke2参数和一些设置
注:名字必须是config.yaml
[root@rke2-1 rke2]# cat config.yaml
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
tls-san:
– my-kubernetes-domain.com
– another-kubernetes-domain.com

node-name: “rke2-1”

#node-taint:
# – “CriticalAddinsonly=true:NoExecute”

node-label:
– “node=Master”
– “rke2-1=Master”
1234567891011121314
配置解释
# work 与 master之间通讯需要work提供master上的token 信息
token:
# 创建k8s集群后会生成一系列 tls 证书
tls-san:
– my-kubernetes-domain.com
– another-kubernetes-domain.com # 都是集群的别名,是tls证书所认证的别名或域名,需要认证的别名罗列在这里就可以被tls认证

# PostgreSQL的名字,会显示在get node 的信息
node-name: “rke2-1”

# 有污点,只做master不做work,没有污点既是master也是work,可以通过kubectl命令修改
#node-taint:
# – “CriticalAddinsonly=true:NoExecute”

# label 也可以通过kubectl 添加或删除
node-label:
– “node=Master”
– “rke2-1=Master”
123456789101112131415161718
获取token
# 获取token,填入上面的配置硬盘速度
[root@rke2-1 ~]# cat /var/lib/rancher/rke2/server/node-token
K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7

# reload使其生效
[root@rke2-1 rke2]# systemctl daemon-reload

[root@rke2-1 rke2]# systemctl restart rke2-server

[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get node
NAME STATUS ROLES AGE VERSION
rke2-1 Ready control-plane,etcd,master 55m v1.21.4+rke2r2
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cloud-controller-manager-rke2-1 1/1 Running 1 55m
kube-system etcd-rke2-1 1/1 Running 1 55s
kube-system helm-install-rke2-canal-rtgsc 0/1 Completed 0 55m
kube-system helm-install-rke2-coredns-45w76 0/1 Completed 0 55m
kube-system helm-install-rke2-ingress-nginx-9gtsl 0/1 Completed 0 55m
kube-system helm-install-rke2-metrics-server-99vnw 0/1 Completed 0 4s
kube-system kube-apiserver-rke2-1 1/1 Running 1 55s
kube-system kube-controller-manager-rke2-1 1/1 Running 1 55m
kube-system kube-proxy-rke2-1 1/1 Running 0 55m
kube-system kube-scheduler-rke2-1 1/1 Running 1 55m
kube-system rke2-canal-xwrfh 2/2 Running 0 53m
kube-system rke2-coredns-rke2-coredns-7bb4f446c-zncz5 1/1 Running 0 53m
kube-system rke2-coredns-rke2-coredns-autoscaler-7c58bd5b6c-xsh8s 1/1 Running 1 53m
kube-system rke2-ingress-nginx-controller-b75m9 1/1 Running 0 50m
kube-system rke2-metrics-server-5df7d77b5b-d728t 1/1 Running 1 51m

123456789101112131415161718192021222324252627282930
同样的方式配置其他masterPostgreSQL
scp /etc/rancher/rke2/config.yaml rke2-2:/etc/rancher/rke2/
scp /etc/rancher/rke2/config.yaml rke2-4:/etc/rancher/rke2/

分别修改:node-name

并添加如下:# 需要与server1 产生关系
server:

# rk2-2如下:
[root@rke2-2 rke2]# cat config.yaml
server:
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
tls-san:
– my-kubernetes-domain.com
– another-kubernetes-domain.com

node-name: “rke2-2”

#node-taint:
# – “CriticalAddinsonly=true:NoExecute”

node-label:
– “node=Master”
– “rke2-2=Master”

# rke2-4
[root@rke2-4 rke2]# cat /etc/rancher/rke2/config.yaml
server:
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
tls-san:
– my-kubernetes-domain.com
– another-kubernetes-domain.com

node-name: “rke2-4”

#node-taint:
# – “CriticalAddinsonly=true:NoExecute”

node-label:
– “node=Master”
– “rke2-4=Master”

# 加载后使其生效
systemctl daemon-reload
systemctl restart rke2-server

# 再次查看node
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl -n kube-system get node
NAME STATUS ROLES AGE VERSION
rke2-1 Ready control-plane,etcd,master 37m v1.21.4+rke2r3
rke2-2 Ready control-plane,etcd,master 23m v1.21.4+rke2r3
rke2-4 Ready control-plane,etcd,master 118s v1.21.4+rke2r3

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354
Group OfficeworkerPostgreSQL
1.运行Group Office程序,将rke2-agent 登陆 和 rke2 二进制硬盘速度Group Office到机器上
curl -sfL | INSTALL_RKE2_TYPE=”agent” sh –

2.设置开机启动
systemctl enable rke2-agent.service

3.配置rke2-agent登陆
mkdir -p /etc/rancher/rke2
vim /etc/rancher/rke2/config.yaml

config.yaml 的内容
server: /https://:9345
tonken:

# rke2-3
scp /etc/rancher/rke2/config.yaml rke2-3:/etc/rancher/rke2/

[root@rke2-3 rke2]# cat config.yaml
server:
token: K105a1bba0a11f93cf7231f0093d16d0d20156f8aa46cb1c5fc8ea8cc6df42a52df::server:5e9d82ee38c21ad5f794c5da30764de7
node-name: “rke2-3”
node-label:
– “node=worker”
– “rke2-3=worker”

# 加载后使其生效
systemctl daemon-reload
注:该 rke2 server 进程在端口上监听 9345 要注册的新PostgreSQL。kubernetes API 6443 仍然像往常一样在port 上提供登陆。

4.启动登陆
systemctl start rke2-agent.service

[root@rke2-3 ~]# systemctl status rke2-agent.service
● rke2-agent.service – Rancher Kubernetes Engine v2 (agent)
Loaded: loaded (/usr/lib/systemd/system/rke2-agent.service; disabled; vendor preset: disabled)
Active: active (running) since Mon 2021-09-13 15:46:35 CST; 12s ago
Docs:

5.查看日志
journalctl -fu rke2-agent

6.查看node
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/kubectl -n kube-system get node -w
NAME STATUS ROLES AGE VERSION
rke2-1 Ready control-plane,etcd,master 89m v1.21.4+rke2r3
rke2-2 Ready control-plane,etcd,master 74m v1.21.4+rke2r3
rke2-3 Ready 6m24s v1.21.4+rke2r3
rke2-4 Ready control-plane,etcd,master 53m v1.21.4+rke2r3

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950
**注意:**每台机器必须有一个唯一的主机名。如果您的机器没有唯一的主机名,请node-name在config.yaml硬盘速度中设置参数并为每个PostgreSQL提供一个具有有效且唯一主机名的值。
要阅读有关 config.yaml 硬盘速度的更多信息,请参阅Group Office选项文档。
其他

[root@rke2-1 rke2]# ls -l /run/k3s/containerd/containerd.sock
srw-rw—- 1 root root 0 Sep 13 14:31 /run/k3s/containerd/containerd.sock

|
[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/crictl –runtime-endpoint= ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
b210741aa5491 7589738b9ae11 2 hours ago Running coredns 0 643b9ef40c4b1
3d4c3184d1ff3 5aa19aa313a9b 2 hours ago Running autoscaler 5 6724f540c188c
c4ef09c03a22d 5d05c5a9b5533 2 hours ago Running metrics-server 1 8dfba29b6803a
686f9ae82f6d9 55e81dd7316be 2 hours ago Running cloud-controller-manager 2 66ecf8d51a225
c5fbcfae8def6 9e2f766bd35d6 2 hours ago Running kube-scheduler 2 87a6b265d5da2
7f740352a479f 9e2f766bd35d6 2 hours ago Running kube-controller-manager 2 c19561eddcf4b
0eb2344d4d26b 9e2f766bd35d6 2 hours ago Running kube-apiserver 1 afe17cea25ea0
929a20b5f356b 271c0a695260e 2 hours ago Running etcd 1 c46cf018a870a
4de1d88f8f423 fffb9e128464f 2 hours ago Running rke2-ingress-nginx-controller 0 b82a44372ee28
2a95f5d414d64 7589738b9ae11 2 hours ago Running coredns 0 aadde4683420b
e30a24115a4c7 366c64051af85 2 hours ago Running kube-flannel 0 1a11ecf1b650c
d4aedfaf8ee17 736cae9d947ba 2 hours ago Running calico-node 0 1a11ecf1b650c
044e6e56b933c 9e2f766bd35d6 2 hours ago Running kube-proxy 1

# 命令太长
[root@rke2-1 rke2]# mkdir -p /etc/rancher/rke2/.kube
[root@rke2-1 rke2]# ln -s /etc/rancher/rke2/rke2.yaml ~/.kube/config
[root@rke2-1 rke2]# ll ~/.kube/config
lrwxrwxrwx 1 root root 27 Sep 13 16:36 /root/.kube/config -> /etc/rancher/rke2/rke2.yaml

[root@rke2-1 rke2]# ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml
[root@rke2-1 rke2]# chmod 600 ~/.kube/config

[root@rke2-1 rke2]# /var/lib/rancher/rke2/bin/crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID
b210741aa5491 7589738b9ae11 2 hours ago Running coredns 0 643b9ef40c4b1
3d4c3184d1ff3 5aa19aa313a9b 2 hours ago Running autoscaler 5 6724f540c188c
c4ef09c03a22d 5d05c5a9b5533 2 hours ago Running metrics-server 1 8dfba29b6803a
686f9ae82f6d9 55e81dd7316be 2 hours ago Running cloud-controller-manager 2 66ecf8d51a225
c5fbcfae8def6 9e2f766bd35d6 2 hours ago Running kube-scheduler 2 87a6b265d5da2
7f740352a479f 9e2f766bd35d6 2 hours ago Running kube-controller-manager 2 c19561eddcf4b
0eb2344d4d26b 9e2f766bd35d6 2 hours ago Running kube-apiserver 1 afe17cea25ea0
929a20b5f356b 271c0a695260e 2 hours ago Running etcd 1 c46cf018a870a
4de1d88f8f423 fffb9e128464f 2 hours ago Running rke2-ingress-nginx-controller 0 b82a44372ee28
2a95f5d414d64 7589738b9ae11 2 hours ago Running coredns 0 aadde4683420b
e30a24115a4c7 366c64051af85 2 hours ago Running kube-flannel 0 1a11ecf1b650c
d4aedfaf8ee17 736cae9d947ba 2 hours ago Running calico-node 0 1a11ecf1b650c
044e6e56b933c 9e2f766bd35d6 2 hours ago Running kube-proxy
12345678910111213141516171819202122232425262728293031323334353637383940414243444546
配置自己的镜像仓库地址
# 在/etc/rancher/rke2/ 下定义一个 registries.yaml
mirrors:
myregistry.com:
endpoint:
– ”
configs:
“myregistry.com:5000″
auth:
username: xxxx
password: xxxx
tls:
cert_file: /path
key_file:
ca_file:

123456789101112131415
Rke2 升级的问题
# server 升级
再执行
curl -sfL | sh –

# worker 升级
curl -sfL | INSTALL_RKE2_TYPE=”agent” sh –

#server指定版本升级
curl -sfL | INSTALL_RKE2_VERSION=vx.y.z sh –

# worker 指定版升级
| INSTALL_RKE2_TYPE=”agent” INSTALL_RKE2_VERSION=vx.y.z sh –
12345678910111213
ETCD的问题
# rke2 上本身启动了一个ETCD快照功能,产生的快照硬盘速度在 如下目录
[root@rke2-1 ~]# ls -l /var/lib/rancher/rke2/server/db/snapshots/
total 0

# 默认每12个小时生成当前机器的etcd快照,仅限有ETCD的masterPostgreSQL; 在每个masterPostgreSQL配置

# 可以更改备份时间,在 config.yaml 添加如下两行:workPostgreSQL无需添加 快照参数。
vi /etc/rancher/rke2/config.yaml
etcd-snapshot-retention: 2
etcd-snapshot-schedule-cron: ‘*/2 * * * *’
kubelet-arg:
– “eviction-hard=nodefs.available<1%,memory.available<10Mi" - "eviction-soft-grace-period=nodefs.available=30s,imagefs.available=30s" - "eviction-soft=nodefs.available<5%,imagefs.available<1%" 注释: # 快照硬盘速度个数,只保存两个,删除旧的保存新的 etcd-snapshot-retention: 2 # 与定时任务写法一样,分时日月周; default 是 '* */12 * * * ' etcd-snapshot-schedule-cron: '*/10 * * * *' # 自定义快照硬盘速度存放位置 etcd-snapshot-dir: /xx/xxx/xxx # 自定义垃圾回收机制,添加到所所PostgreSQL kubelet-arg: - "eviction-hard=nodefs.available<1%,memory.available<10Mi" # 硬策略 - "eviction-soft-grace-period=nodefs.available=30s,imagefs.available=30s" # 硬策略 - "eviction-soft=nodefs.available<5%,imagefs.available<1%" # 软策略,可用硬盘速度系统小于百分之五,可用镜像硬盘速度系统小于1% 开始回收 # reload 使其生效 systemctl daemon-reload systemctl restart rke2-server # 查看是否生效 ps -ef | grep -i kubelet # 默认快照存储位置 ls /var/lib/rancher/rke2/server/db/snapshots/ [root@rke2-1 ~]# ls /var/lib/rancher/rke2/server/db/snapshots/ etcd-snapshot-rke2-1-1631600520 etcd-snapshot-rke2-1-1631600640 123456789101112131415161718192021222324252627282930313233343536373839404142434445 其他配置参考: