1 kubeadm创建集群#

1.1 安装kubeadm#

一台兼容的 Linux 主机。Kubernetes 项目为基于 Debian 和 Red Hat 的 Linux 发行版以及一些不提供包管理器的发行版提供通用的指令
每台机器 2 GB 或更多的 RAM （如果少于这个数字将会影响你应用的运行内存)
2 CPU 核或更多
集群中的所有机器的网络彼此均能相互连接(公网和内网都可以)
- 设置防火墙放行规则
节点之中不可以有重复的主机名、MAC 地址或 product_uuid。请参见这里了解更多详细信息。
- 设置不同hostname
开启机器上的某些端口。请参见这里了解更多详细信息。
- 内网互信
禁用交换分区。为了保证 kubelet 正常工作，你必须禁用交换分区。
- 永久关闭

1.2 为所有机器执行下面操作#

1
#各个机器设置自己的域名
2
hostnamectl set-hostname xxxx
3

4

5
# 将 SELinux 设置为 permissive 模式（相当于将其禁用）
6
sudo setenforce 0
7
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
8

9
#关闭swap
10
swapoff -a
11
sed -ri 's/.*swap.*/#&/' /etc/fstab
12

13
#允许 iptables 检查桥接流量
14
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
15
br_netfilter
16
EOF
17

18
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
19
net.bridge.bridge-nf-call-ip6tables = 1
20
net.bridge.bridge-nf-call-iptables = 1
21
EOF
22
sudo sysctl --system

1.3 安装kubelet、kubeadm、kubectl#

1
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
2
[kubernetes]
3
name=Kubernetes
4
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
5
enabled=1
6
gpgcheck=0
7
repo_gpgcheck=0
8
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
9
   http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
10
exclude=kubelet kubeadm kubectl
11
EOF
12

13

14
sudo yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9 --disableexcludes=kubernetes
15

16
sudo systemctl enable --now kubelet

1.4 下载各个机器需要的镜像#

1
sudo tee ./images.sh <<-'EOF'
2
#!/bin/bash
3
images=(
4
kube-apiserver:v1.20.9
5
kube-proxy:v1.20.9
6
kube-controller-manager:v1.20.9
7
kube-scheduler:v1.20.9
8
coredns:1.7.0
9
etcd:3.4.13-0
10
pause:3.2
11
)
12
for imageName in ${images[@]} ; do
13
docker pull registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/$imageName
14
done
15
EOF
16

17
chmod +x ./images.sh && ./images.sh

1.5 初始化主节点#

1
#所有机器添加master域名映射，以下需要修改为自己的
2
echo "192.168.59.168  cluster-endpoint" >> /etc/hosts
3

4

5

6
#主节点初始化
7
kubeadm init \
8
--apiserver-advertise-address=192.168.59.168 \
9
--control-plane-endpoint=cluster-endpoint \
10
--image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images \
11
--kubernetes-version v1.20.9 \
12
--service-cidr=10.96.0.0/16 \
13
--pod-network-cidr=172.168.0.0/16
14

15
#所有网络范围不重叠

安装成功会显示

1
Your Kubernetes control-plane has initialized successfully!
2

3
To start using your cluster, you need to run the following as a regular user:
4

5
  mkdir -p $HOME/.kube
6
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
7
  sudo chown $(id -u):$(id -g) $HOME/.kube/config
8

9
Alternatively, if you are the root user, you can run:
10

11
  export KUBECONFIG=/etc/kubernetes/admin.conf
12

13
You should now deploy a pod network to the cluster.
14
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
15
  https://kubernetes.io/docs/concepts/cluster-administration/addons/
16

17
You can now join any number of control-plane nodes by copying certificate authorities
18
and service account keys on each node and then running the following as root:
19

20
  kubeadm join cluster-endpoint:6443 --token qyv320.7cl4f3rqyitveqin \
21
    --discovery-token-ca-cert-hash sha256:2c9d7390c7b7716d948ddf6715baa47e44f02cd7c2cd7f0da0ee333ccc19b3ed \
22
    --control-plane
23

24
Then you can join any number of worker nodes by running the following on each as root:
25

26
kubeadm join cluster-endpoint:6443 --token qyv320.7cl4f3rqyitveqin \
27
    --discovery-token-ca-cert-hash sha256:2c9d7390c7b7716d948ddf6715baa47e44f02cd7c2cd7f0da0ee333ccc19b3ed

1.6 主节点执行上面展示的命令#

1
  mkdir -p $HOME/.kube
2
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
3
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

1.7 部署网络插件#

1
curl https://docs.projectcalico.org/manifests/calico.yaml -O
2
kubectl apply -f calico.yaml

1.8 修改配置#

如果前面修改了地址，那么这个地址也需要进行修改

把原来#192.168.0.0/16修改成下面这个

1.9 查看部署k8s所有集群#

1
kubectl get pods -A

1.10 worker节点加入集群#

执行这个命令，不过这个令牌只能24小时之内有效

如果超过24小时，需要重置令牌

1
kubeadm token create --print-join-command

执行命令，这个即为新令牌

1.11 查看启动情况#

成功启动

1.12 安装可视化界面#

1.12.1 在~目录下面新建dashboard.yaml文件#

1
# Copyright 2017 The Kubernetes Authors.
2
#
3
# Licensed under the Apache License, Version 2.0 (the "License");
4
# you may not use this file except in compliance with the License.
5
# You may obtain a copy of the License at
6
#
7
#     http://www.apache.org/licenses/LICENSE-2.0
8
#
9
# Unless required by applicable law or agreed to in writing, software
10
# distributed under the License is distributed on an "AS IS" BASIS,
11
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12
# See the License for the specific language governing permissions and
13
# limitations under the License.
14

15
apiVersion: v1
16
kind: Namespace
17
metadata:
18
  name: kubernetes-dashboard
19

20
---
21

22
apiVersion: v1
23
kind: ServiceAccount
24
metadata:
25
  labels:
26
    k8s-app: kubernetes-dashboard
27
  name: kubernetes-dashboard
28
  namespace: kubernetes-dashboard
29

30
---
31

32
kind: Service
33
apiVersion: v1
34
metadata:
35
  labels:
36
    k8s-app: kubernetes-dashboard
37
  name: kubernetes-dashboard
38
  namespace: kubernetes-dashboard
39
spec:
40
  ports:
41
    - port: 443
42
      targetPort: 8443
43
  selector:
44
    k8s-app: kubernetes-dashboard
45

46
---
47

48
apiVersion: v1
49
kind: Secret
50
metadata:
51
  labels:
52
    k8s-app: kubernetes-dashboard
53
  name: kubernetes-dashboard-certs
54
  namespace: kubernetes-dashboard
55
type: Opaque
56

57
---
58

59
apiVersion: v1
60
kind: Secret
61
metadata:
62
  labels:
63
    k8s-app: kubernetes-dashboard
64
  name: kubernetes-dashboard-csrf
65
  namespace: kubernetes-dashboard
66
type: Opaque
67
data:
68
  csrf: ""
69

70
---
71

72
apiVersion: v1
73
kind: Secret
74
metadata:
75
  labels:
76
    k8s-app: kubernetes-dashboard
77
  name: kubernetes-dashboard-key-holder
78
  namespace: kubernetes-dashboard
79
type: Opaque
80

81
---
82

83
kind: ConfigMap
84
apiVersion: v1
85
metadata:
86
  labels:
87
    k8s-app: kubernetes-dashboard
88
  name: kubernetes-dashboard-settings
89
  namespace: kubernetes-dashboard
90

91
---
92

93
kind: Role
94
apiVersion: rbac.authorization.k8s.io/v1
95
metadata:
96
  labels:
97
    k8s-app: kubernetes-dashboard
98
  name: kubernetes-dashboard
99
  namespace: kubernetes-dashboard
100
rules:
101
  # Allow Dashboard to get, update and delete Dashboard exclusive secrets.
102
  - apiGroups: [""]
103
    resources: ["secrets"]
104
    resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs", "kubernetes-dashboard-csrf"]
105
    verbs: ["get", "update", "delete"]
106
    # Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
107
  - apiGroups: [""]
108
    resources: ["configmaps"]
109
    resourceNames: ["kubernetes-dashboard-settings"]
110
    verbs: ["get", "update"]
111
    # Allow Dashboard to get metrics.
112
  - apiGroups: [""]
113
    resources: ["services"]
114
    resourceNames: ["heapster", "dashboard-metrics-scraper"]
115
    verbs: ["proxy"]
116
  - apiGroups: [""]
117
    resources: ["services/proxy"]
118
    resourceNames: ["heapster", "http:heapster:", "https:heapster:", "dashboard-metrics-scraper", "http:dashboard-metrics-scraper"]
119
    verbs: ["get"]
120

121
---
122

123
kind: ClusterRole
124
apiVersion: rbac.authorization.k8s.io/v1
125
metadata:
126
  labels:
127
    k8s-app: kubernetes-dashboard
128
  name: kubernetes-dashboard
129
rules:
130
  # Allow Metrics Scraper to get metrics from the Metrics server
131
  - apiGroups: ["metrics.k8s.io"]
132
    resources: ["pods", "nodes"]
133
    verbs: ["get", "list", "watch"]
134

135
---
136

137
apiVersion: rbac.authorization.k8s.io/v1
138
kind: RoleBinding
139
metadata:
140
  labels:
141
    k8s-app: kubernetes-dashboard
142
  name: kubernetes-dashboard
143
  namespace: kubernetes-dashboard
144
roleRef:
145
  apiGroup: rbac.authorization.k8s.io
146
  kind: Role
147
  name: kubernetes-dashboard
148
subjects:
149
  - kind: ServiceAccount
150
    name: kubernetes-dashboard
151
    namespace: kubernetes-dashboard
152

153
---
154

155
apiVersion: rbac.authorization.k8s.io/v1
156
kind: ClusterRoleBinding
157
metadata:
158
  name: kubernetes-dashboard
159
roleRef:
160
  apiGroup: rbac.authorization.k8s.io
161
  kind: ClusterRole
162
  name: kubernetes-dashboard
163
subjects:
164
  - kind: ServiceAccount
165
    name: kubernetes-dashboard
166
    namespace: kubernetes-dashboard
167

168
---
169

170
kind: Deployment
171
apiVersion: apps/v1
172
metadata:
173
  labels:
174
    k8s-app: kubernetes-dashboard
175
  name: kubernetes-dashboard
176
  namespace: kubernetes-dashboard
177
spec:
178
  replicas: 1
179
  revisionHistoryLimit: 10
180
  selector:
181
    matchLabels:
182
      k8s-app: kubernetes-dashboard
183
  template:
184
    metadata:
185
      labels:
186
        k8s-app: kubernetes-dashboard
187
    spec:
188
      containers:
189
        - name: kubernetes-dashboard
190
          image: kubernetesui/dashboard:v2.3.1
191
          imagePullPolicy: Always
192
          ports:
193
            - containerPort: 8443
194
              protocol: TCP
195
          args:
196
            - --auto-generate-certificates
197
            - --namespace=kubernetes-dashboard
198
            # Uncomment the following line to manually specify Kubernetes API server Host
199
            # If not specified, Dashboard will attempt to auto discover the API server and connect
200
            # to it. Uncomment only if the default does not work.
201
            # - --apiserver-host=http://my-address:port
202
          volumeMounts:
203
            - name: kubernetes-dashboard-certs
204
              mountPath: /certs
205
              # Create on-disk volume to store exec logs
206
            - mountPath: /tmp
207
              name: tmp-volume
208
          livenessProbe:
209
            httpGet:
210
              scheme: HTTPS
211
              path: /
212
              port: 8443
213
            initialDelaySeconds: 30
214
            timeoutSeconds: 30
215
          securityContext:
216
            allowPrivilegeEscalation: false
217
            readOnlyRootFilesystem: true
218
            runAsUser: 1001
219
            runAsGroup: 2001
220
      volumes:
221
        - name: kubernetes-dashboard-certs
222
          secret:
223
            secretName: kubernetes-dashboard-certs
224
        - name: tmp-volume
225
          emptyDir: {}
226
      serviceAccountName: kubernetes-dashboard
227
      nodeSelector:
228
        "kubernetes.io/os": linux
229
      # Comment the following tolerations if Dashboard must not be deployed on master
230
      tolerations:
231
        - key: node-role.kubernetes.io/master
232
          effect: NoSchedule
233

234
---
235

236
kind: Service
237
apiVersion: v1
238
metadata:
239
  labels:
240
    k8s-app: dashboard-metrics-scraper
241
  name: dashboard-metrics-scraper
242
  namespace: kubernetes-dashboard
243
spec:
244
  ports:
245
    - port: 8000
246
      targetPort: 8000
247
  selector:
248
    k8s-app: dashboard-metrics-scraper
249

250
---
251

252
kind: Deployment
253
apiVersion: apps/v1
254
metadata:
255
  labels:
256
    k8s-app: dashboard-metrics-scraper
257
  name: dashboard-metrics-scraper
258
  namespace: kubernetes-dashboard
259
spec:
260
  replicas: 1
261
  revisionHistoryLimit: 10
262
  selector:
263
    matchLabels:
264
      k8s-app: dashboard-metrics-scraper
265
  template:
266
    metadata:
267
      labels:
268
        k8s-app: dashboard-metrics-scraper
269
      annotations:
270
        seccomp.security.alpha.kubernetes.io/pod: 'runtime/default'
271
    spec:
272
      containers:
273
        - name: dashboard-metrics-scraper
274
          image: kubernetesui/metrics-scraper:v1.0.6
275
          ports:
276
            - containerPort: 8000
277
              protocol: TCP
278
          livenessProbe:
279
            httpGet:
280
              scheme: HTTP
281
              path: /
282
              port: 8000
283
            initialDelaySeconds: 30
284
            timeoutSeconds: 30
285
          volumeMounts:
286
          - mountPath: /tmp
287
            name: tmp-volume
288
          securityContext:
289
            allowPrivilegeEscalation: false
290
            readOnlyRootFilesystem: true
291
            runAsUser: 1001
292
            runAsGroup: 2001
293
      serviceAccountName: kubernetes-dashboard
294
      nodeSelector:
295
        "kubernetes.io/os": linux
296
      # Comment the following tolerations if Dashboard must not be deployed on master
297
      tolerations:
298
        - key: node-role.kubernetes.io/master
299
          effect: NoSchedule
300
      volumes:
301
        - name: tmp-volume
302
          emptyDir: {}

1.12.2 应用yaml#

kubectl apply -f dashboard.yaml

1.12.3 设置访问端口#

kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard

找到type，修改成NodePort

1.12.4 找到端口，安全组放行#

1
kubectl get svc -A |grep kubernetes-dashboard
2
## 找到端口，在安全组放行

访问： https://集群任意IP:端口 https://139.198.165.238:32759

1.12.5 创建访问账号#

1
#创建访问账号，准备一个yaml文件； vi dash.yaml
2
apiVersion: v1
3
kind: ServiceAccount
4
metadata:
5
  name: admin-user
6
  namespace: kubernetes-dashboard
7
---
8
apiVersion: rbac.authorization.k8s.io/v1
9
kind: ClusterRoleBinding
10
metadata:
11
  name: admin-user
12
roleRef:
13
  apiGroup: rbac.authorization.k8s.io
14
  kind: ClusterRole
15
  name: cluster-admin
16
subjects:
17
- kind: ServiceAccount
18
  name: admin-user
19
  namespace: kubernetes-dashboard

1
kubectl apply -f dash.yaml

1
#获取访问令牌
2
kubectl -n kubernetes-dashboard get secret $(kubectl -n kubernetes-dashboard get sa/admin-user -o jsonpath="{.secrets[0].name}") -o go-template="{{.data.token | base64decode}}"

eyJhbGciOiJSUzI1NiIsImtpZCI6InVzQTI2MDFlNTFHM2NYUGM0T05rTlhYV2tPc0NtMUU5VkNKZEVYOGcxczQifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLW02bGRrIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhZjMzMWU0YS03ZTY4LTRmMzktOGQwZi0xZDgyMTFkZGMyYjgiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZXJuZXRlcy1kYXNoYm9hcmQ6YWRtaW4tdXNlciJ9.yrv-EZBJOcy4EyBpxJNP-FXxfDFgipKMusT3h6jIbdLeb9WPwQR44Qbuf8MIeCF9jYckoOegLjBn1dH3raIHLAb-dIlbl4hOVGQ9HyWDiPnkU5Lg0QySKQ617Eg1EBu3-3uXcDKEyB37Qx43LupEPnB4fHt7wBHg_8d8GPsE7gDpUE6FHhpbMq4u-gdodLRZW_aOdw-An801qQ7gGnLOs8vUL-_RyEoUsFNl8472yNc7qzg6M6DnQqfTrsFcqY8VN00G7BqIChjVPj7O2nw8ZwuElEudE-S20ZjY-Pn4qscygvNcHUbvFspXOht_FJlnLvtqyqXMw6DJtZtkSyeg9A

2 控制Pod#

2.0 名称空间#

1
kubectl create ns hello
2
kubectl delete ns hello

通过配置文件创建名称空间

1.创建yaml

2.编辑yaml

1
apiVersion: v1
2
kind: Namespace
3
metadata:
4
  name: hello

3.kubectl apply -f hello.yaml

2.1 创建pod#

1
kubectl run mynginx --image=nginx
2
# 查看default名称空间的Pod
3
kubectl get pod
4
# 描述
5
kubectl describe pod 你自己的Pod名字
6
# 删除
7
kubectl delete pod Pod名字
8
# 查看Pod的运行日志
9
kubectl logs Pod名字
10
#通过配置文件删除
11
kubectl delete -f xxx.yaml
12
# 每个Pod - k8s都会分配一个ip
13
kubectl get pod -owide
14
# 使用Pod的ip+pod里面运行容器的端口
15
curl 192.168.169.136
16
#使用部署创建容器，这时候一个挂了可以在别的机器创建
17
kubectl create deployment mytomcat --image=tomcat:8.5.68
18
#如果删除就需要使用kubectl delete deploy mytomcat来进行删除

2.2 多副本#

kubectl create deployment my-dep —image=nginx —replicas=3

1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  labels:
5
    app: my-dep
6
  name: my-dep
7
spec:
8
  replicas: 3
9
  selector:
10
    matchLabels:
11
      app: my-dep
12
  template:
13
    metadata:
14
      labels:
15
        app: my-dep
16
    spec:
17
      containers:
18
      - image: nginx
19
        name: nginx

2.3 扩缩容#

1
kubectl scale --replicas=5 deployment/my-dep

1
kubectl edit deployment my-dep
2

3
#修改 replicas

2.4 滚动更新#

1
kubectl set image deployment/my-dep nginx=nginx:1.16.1 --record
2
kubectl rollout status deployment/my-dep

2.5 版本回退#

1
#历史记录
2
kubectl rollout history deployment/my-dep
3

4

5
#查看某个历史详情
6
kubectl rollout history deployment/my-dep --revision=2
7

8
#回滚(回到上次)
9
kubectl rollout undo deployment/my-dep
10

11
#回滚(回到指定版本)
12
kubectl rollout undo deployment/my-dep --to-revision=2

2.6 service#

1
#暴露Deploy
2
kubectl expose deployment my-dep --port=8000 --target-port=80
3

4
#使用标签检索Pod
5
kubectl get pod -l app=my-dep
6
kubectl get service

2.7 ClusterIP#

1
# 等同于没有--type的
2
kubectl expose deployment my-dep --port=8000 --target-port=80 --type=ClusterIP

1
apiVersion: v1
2
kind: Service
3
metadata:
4
  labels:
5
    app: my-dep
6
  name: my-dep
7
spec:
8
  ports:
9
  - port: 8000
10
    protocol: TCP
11
    targetPort: 80
12
  selector:
13
    app: my-dep
14
  type: ClusterIP

2.8 NodePort#

nodePort就可以通过外网进行访问

1
kubectl expose deployment my-dep --port=8000 --target-port=80 --type=NodePort

通过各个虚拟机ip的30388端口即可访问

1
apiVersion: v1
2
kind: Service
3
metadata:
4
  labels:
5
    app: my-dep
6
  name: my-dep
7
spec:
8
  ports:
9
  - port: 8000
10
    protocol: TCP
11
    targetPort: 80
12
  selector:
13
    app: my-dep
14
  type: NodePort

2.9 ingress#

service统一网关入口

配置ingress.yaml

1
apiVersion: v1
2
kind: Namespace
3
metadata:
4
  name: ingress-nginx
5
  labels:
6
    app.kubernetes.io/name: ingress-nginx
7
    app.kubernetes.io/instance: ingress-nginx
8

9
---
10
# Source: ingress-nginx/templates/controller-serviceaccount.yaml
11
apiVersion: v1
12
kind: ServiceAccount
13
metadata:
14
  labels:
15
    helm.sh/chart: ingress-nginx-3.33.0
16
    app.kubernetes.io/name: ingress-nginx
17
    app.kubernetes.io/instance: ingress-nginx
18
    app.kubernetes.io/version: 0.47.0
19
    app.kubernetes.io/managed-by: Helm
20
    app.kubernetes.io/component: controller
21
  name: ingress-nginx
22
  namespace: ingress-nginx
23
automountServiceAccountToken: true
24
---
25
# Source: ingress-nginx/templates/controller-configmap.yaml
26
apiVersion: v1
27
kind: ConfigMap
28
metadata:
29
  labels:
30
    helm.sh/chart: ingress-nginx-3.33.0
31
    app.kubernetes.io/name: ingress-nginx
32
    app.kubernetes.io/instance: ingress-nginx
33
    app.kubernetes.io/version: 0.47.0
34
    app.kubernetes.io/managed-by: Helm
35
    app.kubernetes.io/component: controller
36
  name: ingress-nginx-controller
37
  namespace: ingress-nginx
38
data:
39
---
40
# Source: ingress-nginx/templates/clusterrole.yaml
41
apiVersion: rbac.authorization.k8s.io/v1
42
kind: ClusterRole
43
metadata:
44
  labels:
45
    helm.sh/chart: ingress-nginx-3.33.0
46
    app.kubernetes.io/name: ingress-nginx
47
    app.kubernetes.io/instance: ingress-nginx
48
    app.kubernetes.io/version: 0.47.0
49
    app.kubernetes.io/managed-by: Helm
50
  name: ingress-nginx
51
rules:
52
  - apiGroups:
53
      - ''
54
    resources:
55
      - configmaps
56
      - endpoints
57
      - nodes
58
      - pods
59
      - secrets
60
    verbs:
61
      - list
62
      - watch
63
  - apiGroups:
64
      - ''
65
    resources:
66
      - nodes
67
    verbs:
68
      - get
69
  - apiGroups:
70
      - ''
71
    resources:
72
      - services
73
    verbs:
74
      - get
75
      - list
76
      - watch
77
  - apiGroups:
78
      - extensions
79
      - networking.k8s.io   # k8s 1.14+
80
    resources:
81
      - ingresses
82
    verbs:
83
      - get
84
      - list
85
      - watch
86
  - apiGroups:
87
      - ''
88
    resources:
89
      - events
90
    verbs:
91
      - create
92
      - patch
93
  - apiGroups:
94
      - extensions
95
      - networking.k8s.io   # k8s 1.14+
96
    resources:
97
      - ingresses/status
98
    verbs:
99
      - update
100
  - apiGroups:
101
      - networking.k8s.io   # k8s 1.14+
102
    resources:
103
      - ingressclasses
104
    verbs:
105
      - get
106
      - list
107
      - watch
108
---
109
# Source: ingress-nginx/templates/clusterrolebinding.yaml
110
apiVersion: rbac.authorization.k8s.io/v1
111
kind: ClusterRoleBinding
112
metadata:
113
  labels:
114
    helm.sh/chart: ingress-nginx-3.33.0
115
    app.kubernetes.io/name: ingress-nginx
116
    app.kubernetes.io/instance: ingress-nginx
117
    app.kubernetes.io/version: 0.47.0
118
    app.kubernetes.io/managed-by: Helm
119
  name: ingress-nginx
120
roleRef:
121
  apiGroup: rbac.authorization.k8s.io
122
  kind: ClusterRole
123
  name: ingress-nginx
124
subjects:
125
  - kind: ServiceAccount
126
    name: ingress-nginx
127
    namespace: ingress-nginx
128
---
129
# Source: ingress-nginx/templates/controller-role.yaml
130
apiVersion: rbac.authorization.k8s.io/v1
131
kind: Role
132
metadata:
133
  labels:
134
    helm.sh/chart: ingress-nginx-3.33.0
135
    app.kubernetes.io/name: ingress-nginx
136
    app.kubernetes.io/instance: ingress-nginx
137
    app.kubernetes.io/version: 0.47.0
138
    app.kubernetes.io/managed-by: Helm
139
    app.kubernetes.io/component: controller
140
  name: ingress-nginx
141
  namespace: ingress-nginx
142
rules:
143
  - apiGroups:
144
      - ''
145
    resources:
146
      - namespaces
147
    verbs:
148
      - get
149
  - apiGroups:
150
      - ''
151
    resources:
152
      - configmaps
153
      - pods
154
      - secrets
155
      - endpoints
156
    verbs:
157
      - get
158
      - list
159
      - watch
160
  - apiGroups:
161
      - ''
162
    resources:
163
      - services
164
    verbs:
165
      - get
166
      - list
167
      - watch
168
  - apiGroups:
169
      - extensions
170
      - networking.k8s.io   # k8s 1.14+
171
    resources:
172
      - ingresses
173
    verbs:
174
      - get
175
      - list
176
      - watch
177
  - apiGroups:
178
      - extensions
179
      - networking.k8s.io   # k8s 1.14+
180
    resources:
181
      - ingresses/status
182
    verbs:
183
      - update
184
  - apiGroups:
185
      - networking.k8s.io   # k8s 1.14+
186
    resources:
187
      - ingressclasses
188
    verbs:
189
      - get
190
      - list
191
      - watch
192
  - apiGroups:
193
      - ''
194
    resources:
195
      - configmaps
196
    resourceNames:
197
      - ingress-controller-leader-nginx
198
    verbs:
199
      - get
200
      - update
201
  - apiGroups:
202
      - ''
203
    resources:
204
      - configmaps
205
    verbs:
206
      - create
207
  - apiGroups:
208
      - ''
209
    resources:
210
      - events
211
    verbs:
212
      - create
213
      - patch
214
---
215
# Source: ingress-nginx/templates/controller-rolebinding.yaml
216
apiVersion: rbac.authorization.k8s.io/v1
217
kind: RoleBinding
218
metadata:
219
  labels:
220
    helm.sh/chart: ingress-nginx-3.33.0
221
    app.kubernetes.io/name: ingress-nginx
222
    app.kubernetes.io/instance: ingress-nginx
223
    app.kubernetes.io/version: 0.47.0
224
    app.kubernetes.io/managed-by: Helm
225
    app.kubernetes.io/component: controller
226
  name: ingress-nginx
227
  namespace: ingress-nginx
228
roleRef:
229
  apiGroup: rbac.authorization.k8s.io
230
  kind: Role
231
  name: ingress-nginx
232
subjects:
233
  - kind: ServiceAccount
234
    name: ingress-nginx
235
    namespace: ingress-nginx
236
---
237
# Source: ingress-nginx/templates/controller-service-webhook.yaml
238
apiVersion: v1
239
kind: Service
240
metadata:
241
  labels:
242
    helm.sh/chart: ingress-nginx-3.33.0
243
    app.kubernetes.io/name: ingress-nginx
244
    app.kubernetes.io/instance: ingress-nginx
245
    app.kubernetes.io/version: 0.47.0
246
    app.kubernetes.io/managed-by: Helm
247
    app.kubernetes.io/component: controller
248
  name: ingress-nginx-controller-admission
249
  namespace: ingress-nginx
250
spec:
251
  type: ClusterIP
252
  ports:
253
    - name: https-webhook
254
      port: 443
255
      targetPort: webhook
256
  selector:
257
    app.kubernetes.io/name: ingress-nginx
258
    app.kubernetes.io/instance: ingress-nginx
259
    app.kubernetes.io/component: controller
260
---
261
# Source: ingress-nginx/templates/controller-service.yaml
262
apiVersion: v1
263
kind: Service
264
metadata:
265
  annotations:
266
  labels:
267
    helm.sh/chart: ingress-nginx-3.33.0
268
    app.kubernetes.io/name: ingress-nginx
269
    app.kubernetes.io/instance: ingress-nginx
270
    app.kubernetes.io/version: 0.47.0
271
    app.kubernetes.io/managed-by: Helm
272
    app.kubernetes.io/component: controller
273
  name: ingress-nginx-controller
274
  namespace: ingress-nginx
275
spec:
276
  type: NodePort
277
  ports:
278
    - name: http
279
      port: 80
280
      protocol: TCP
281
      targetPort: http
282
    - name: https
283
      port: 443
284
      protocol: TCP
285
      targetPort: https
286
  selector:
287
    app.kubernetes.io/name: ingress-nginx
288
    app.kubernetes.io/instance: ingress-nginx
289
    app.kubernetes.io/component: controller
290
---
291
# Source: ingress-nginx/templates/controller-deployment.yaml
292
apiVersion: apps/v1
293
kind: Deployment
294
metadata:
295
  labels:
296
    helm.sh/chart: ingress-nginx-3.33.0
297
    app.kubernetes.io/name: ingress-nginx
298
    app.kubernetes.io/instance: ingress-nginx
299
    app.kubernetes.io/version: 0.47.0
300
    app.kubernetes.io/managed-by: Helm
301
    app.kubernetes.io/component: controller
302
  name: ingress-nginx-controller
303
  namespace: ingress-nginx
304
spec:
305
  selector:
306
    matchLabels:
307
      app.kubernetes.io/name: ingress-nginx
308
      app.kubernetes.io/instance: ingress-nginx
309
      app.kubernetes.io/component: controller
310
  revisionHistoryLimit: 10
311
  minReadySeconds: 0
312
  template:
313
    metadata:
314
      labels:
315
        app.kubernetes.io/name: ingress-nginx
316
        app.kubernetes.io/instance: ingress-nginx
317
        app.kubernetes.io/component: controller
318
    spec:
319
      dnsPolicy: ClusterFirst
320
      containers:
321
        - name: controller
322
          image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/ingress-nginx-controller:v0.46.0
323
          imagePullPolicy: IfNotPresent
324
          lifecycle:
325
            preStop:
326
              exec:
327
                command:
328
                  - /wait-shutdown
329
          args:
330
            - /nginx-ingress-controller
331
            - --election-id=ingress-controller-leader
332
            - --ingress-class=nginx
333
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
334
            - --validating-webhook=:8443
335
            - --validating-webhook-certificate=/usr/local/certificates/cert
336
            - --validating-webhook-key=/usr/local/certificates/key
337
          securityContext:
338
            capabilities:
339
              drop:
340
                - ALL
341
              add:
342
                - NET_BIND_SERVICE
343
            runAsUser: 101
344
            allowPrivilegeEscalation: true
345
          env:
346
            - name: POD_NAME
347
              valueFrom:
348
                fieldRef:
349
                  fieldPath: metadata.name
350
            - name: POD_NAMESPACE
351
              valueFrom:
352
                fieldRef:
353
                  fieldPath: metadata.namespace
354
            - name: LD_PRELOAD
355
              value: /usr/local/lib/libmimalloc.so
356
          livenessProbe:
357
            failureThreshold: 5
358
            httpGet:
359
              path: /healthz
360
              port: 10254
361
              scheme: HTTP
362
            initialDelaySeconds: 10
363
            periodSeconds: 10
364
            successThreshold: 1
365
            timeoutSeconds: 1
366
          readinessProbe:
367
            failureThreshold: 3
368
            httpGet:
369
              path: /healthz
370
              port: 10254
371
              scheme: HTTP
372
            initialDelaySeconds: 10
373
            periodSeconds: 10
374
            successThreshold: 1
375
            timeoutSeconds: 1
376
          ports:
377
            - name: http
378
              containerPort: 80
379
              protocol: TCP
380
            - name: https
381
              containerPort: 443
382
              protocol: TCP
383
            - name: webhook
384
              containerPort: 8443
385
              protocol: TCP
386
          volumeMounts:
387
            - name: webhook-cert
388
              mountPath: /usr/local/certificates/
389
              readOnly: true
390
          resources:
391
            requests:
392
              cpu: 100m
393
              memory: 90Mi
394
      nodeSelector:
395
        kubernetes.io/os: linux
396
      serviceAccountName: ingress-nginx
397
      terminationGracePeriodSeconds: 300
398
      volumes:
399
        - name: webhook-cert
400
          secret:
401
            secretName: ingress-nginx-admission
402
---
403
# Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
404
# before changing this value, check the required kubernetes version
405
# https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
406
apiVersion: admissionregistration.k8s.io/v1
407
kind: ValidatingWebhookConfiguration
408
metadata:
409
  labels:
410
    helm.sh/chart: ingress-nginx-3.33.0
411
    app.kubernetes.io/name: ingress-nginx
412
    app.kubernetes.io/instance: ingress-nginx
413
    app.kubernetes.io/version: 0.47.0
414
    app.kubernetes.io/managed-by: Helm
415
    app.kubernetes.io/component: admission-webhook
416
  name: ingress-nginx-admission
417
webhooks:
418
  - name: validate.nginx.ingress.kubernetes.io
419
    matchPolicy: Equivalent
420
    rules:
421
      - apiGroups:
422
          - networking.k8s.io
423
        apiVersions:
424
          - v1beta1
425
        operations:
426
          - CREATE
427
          - UPDATE
428
        resources:
429
          - ingresses
430
    failurePolicy: Fail
431
    sideEffects: None
432
    admissionReviewVersions:
433
      - v1
434
      - v1beta1
435
    clientConfig:
436
      service:
437
        namespace: ingress-nginx
438
        name: ingress-nginx-controller-admission
439
        path: /networking/v1beta1/ingresses
440
---
441
# Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
442
apiVersion: v1
443
kind: ServiceAccount
444
metadata:
445
  name: ingress-nginx-admission
446
  annotations:
447
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
448
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
449
  labels:
450
    helm.sh/chart: ingress-nginx-3.33.0
451
    app.kubernetes.io/name: ingress-nginx
452
    app.kubernetes.io/instance: ingress-nginx
453
    app.kubernetes.io/version: 0.47.0
454
    app.kubernetes.io/managed-by: Helm
455
    app.kubernetes.io/component: admission-webhook
456
  namespace: ingress-nginx
457
---
458
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
459
apiVersion: rbac.authorization.k8s.io/v1
460
kind: ClusterRole
461
metadata:
462
  name: ingress-nginx-admission
463
  annotations:
464
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
465
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
466
  labels:
467
    helm.sh/chart: ingress-nginx-3.33.0
468
    app.kubernetes.io/name: ingress-nginx
469
    app.kubernetes.io/instance: ingress-nginx
470
    app.kubernetes.io/version: 0.47.0
471
    app.kubernetes.io/managed-by: Helm
472
    app.kubernetes.io/component: admission-webhook
473
rules:
474
  - apiGroups:
475
      - admissionregistration.k8s.io
476
    resources:
477
      - validatingwebhookconfigurations
478
    verbs:
479
      - get
480
      - update
481
---
482
# Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
483
apiVersion: rbac.authorization.k8s.io/v1
484
kind: ClusterRoleBinding
485
metadata:
486
  name: ingress-nginx-admission
487
  annotations:
488
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
489
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
490
  labels:
491
    helm.sh/chart: ingress-nginx-3.33.0
492
    app.kubernetes.io/name: ingress-nginx
493
    app.kubernetes.io/instance: ingress-nginx
494
    app.kubernetes.io/version: 0.47.0
495
    app.kubernetes.io/managed-by: Helm
496
    app.kubernetes.io/component: admission-webhook
497
roleRef:
498
  apiGroup: rbac.authorization.k8s.io
499
  kind: ClusterRole
500
  name: ingress-nginx-admission
501
subjects:
502
  - kind: ServiceAccount
503
    name: ingress-nginx-admission
504
    namespace: ingress-nginx
505
---
506
# Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
507
apiVersion: rbac.authorization.k8s.io/v1
508
kind: Role
509
metadata:
510
  name: ingress-nginx-admission
511
  annotations:
512
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
513
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
514
  labels:
515
    helm.sh/chart: ingress-nginx-3.33.0
516
    app.kubernetes.io/name: ingress-nginx
517
    app.kubernetes.io/instance: ingress-nginx
518
    app.kubernetes.io/version: 0.47.0
519
    app.kubernetes.io/managed-by: Helm
520
    app.kubernetes.io/component: admission-webhook
521
  namespace: ingress-nginx
522
rules:
523
  - apiGroups:
524
      - ''
525
    resources:
526
      - secrets
527
    verbs:
528
      - get
529
      - create
530
---
531
# Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
532
apiVersion: rbac.authorization.k8s.io/v1
533
kind: RoleBinding
534
metadata:
535
  name: ingress-nginx-admission
536
  annotations:
537
    helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
538
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
539
  labels:
540
    helm.sh/chart: ingress-nginx-3.33.0
541
    app.kubernetes.io/name: ingress-nginx
542
    app.kubernetes.io/instance: ingress-nginx
543
    app.kubernetes.io/version: 0.47.0
544
    app.kubernetes.io/managed-by: Helm
545
    app.kubernetes.io/component: admission-webhook
546
  namespace: ingress-nginx
547
roleRef:
548
  apiGroup: rbac.authorization.k8s.io
549
  kind: Role
550
  name: ingress-nginx-admission
551
subjects:
552
  - kind: ServiceAccount
553
    name: ingress-nginx-admission
554
    namespace: ingress-nginx
555
---
556
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
557
apiVersion: batch/v1
558
kind: Job
559
metadata:
560
  name: ingress-nginx-admission-create
561
  annotations:
562
    helm.sh/hook: pre-install,pre-upgrade
563
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
564
  labels:
565
    helm.sh/chart: ingress-nginx-3.33.0
566
    app.kubernetes.io/name: ingress-nginx
567
    app.kubernetes.io/instance: ingress-nginx
568
    app.kubernetes.io/version: 0.47.0
569
    app.kubernetes.io/managed-by: Helm
570
    app.kubernetes.io/component: admission-webhook
571
  namespace: ingress-nginx
572
spec:
573
  template:
574
    metadata:
575
      name: ingress-nginx-admission-create
576
      labels:
577
        helm.sh/chart: ingress-nginx-3.33.0
578
        app.kubernetes.io/name: ingress-nginx
579
        app.kubernetes.io/instance: ingress-nginx
580
        app.kubernetes.io/version: 0.47.0
581
        app.kubernetes.io/managed-by: Helm
582
        app.kubernetes.io/component: admission-webhook
583
    spec:
584
      containers:
585
        - name: create
586
          image: docker.io/jettech/kube-webhook-certgen:v1.5.1
587
          imagePullPolicy: IfNotPresent
588
          args:
589
            - create
590
            - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
591
            - --namespace=$(POD_NAMESPACE)
592
            - --secret-name=ingress-nginx-admission
593
          env:
594
            - name: POD_NAMESPACE
595
              valueFrom:
596
                fieldRef:
597
                  fieldPath: metadata.namespace
598
      restartPolicy: OnFailure
599
      serviceAccountName: ingress-nginx-admission
600
      securityContext:
601
        runAsNonRoot: true
602
        runAsUser: 2000
603
---
604
# Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
605
apiVersion: batch/v1
606
kind: Job
607
metadata:
608
  name: ingress-nginx-admission-patch
609
  annotations:
610
    helm.sh/hook: post-install,post-upgrade
611
    helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
612
  labels:
613
    helm.sh/chart: ingress-nginx-3.33.0
614
    app.kubernetes.io/name: ingress-nginx
615
    app.kubernetes.io/instance: ingress-nginx
616
    app.kubernetes.io/version: 0.47.0
617
    app.kubernetes.io/managed-by: Helm
618
    app.kubernetes.io/component: admission-webhook
619
  namespace: ingress-nginx
620
spec:
621
  template:
622
    metadata:
623
      name: ingress-nginx-admission-patch
624
      labels:
625
        helm.sh/chart: ingress-nginx-3.33.0
626
        app.kubernetes.io/name: ingress-nginx
627
        app.kubernetes.io/instance: ingress-nginx
628
        app.kubernetes.io/version: 0.47.0
629
        app.kubernetes.io/managed-by: Helm
630
        app.kubernetes.io/component: admission-webhook
631
    spec:
632
      containers:
633
        - name: patch
634
          image: docker.io/jettech/kube-webhook-certgen:v1.5.1
635
          imagePullPolicy: IfNotPresent
636
          args:
637
            - patch
638
            - --webhook-name=ingress-nginx-admission
639
            - --namespace=$(POD_NAMESPACE)
640
            - --patch-mutating=false
641
            - --secret-name=ingress-nginx-admission
642
            - --patch-failure-policy=Fail
643
          env:
644
            - name: POD_NAMESPACE
645
              valueFrom:
646
                fieldRef:
647
                  fieldPath: metadata.namespace
648
      restartPolicy: OnFailure
649
      serviceAccountName: ingress-nginx-admission
650
      securityContext:
651
        runAsNonRoot: true
652
        runAsUser: 2000

1
kubectl get pod,svc -n ingress-nginx
2
#查看

2.9.1 ingress域名访问#

1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  name: ingress-host-bar
5
spec:
6
  ingressClassName: nginx
7
  rules:
8
  - host: "hello.atguigu.com"
9
    http:
10
      paths:
11
      - pathType: Prefix
12
        path: "/"
13
        backend:
14
          service:
15
            name: hello-server
16
            port:
17
              number: 8000
18
  - host: "demo.atguigu.com"
19
    http:
20
      paths:
21
      - pathType: Prefix
22
        path: "/nginx"  # 把请求会转给下面的服务，下面的服务一定要能处理这个路径，不能处理就是404
23
        backend:
24
          service:
25
            name: nginx-demo  ## java，比如使用路径重写，去掉前缀nginx
26
            port:
27
              number: 8000

2.9.2 路径重写#

1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  annotations:
5
    nginx.ingress.kubernetes.io/rewrite-target: /$2
6
  name: ingress-host-bar
7
spec:
8
  ingressClassName: nginx
9
  rules:
10
  - host: "hello.atguigu.com"
11
    http:
12
      paths:
13
      - pathType: Prefix
14
        path: "/"
15
        backend:
16
          service:
17
            name: hello-server
18
            port:
19
              number: 8000
20
  - host: "demo.atguigu.com"
21
    http:
22
      paths:
23
      - pathType: Prefix
24
        path: "/nginx(/|$)(.*)"  # 把请求会转给下面的服务，下面的服务一定要能处理这个路径，不能处理就是404
25
        backend:
26
          service:
27
            name: nginx-demo  ## java，比如使用路径重写，去掉前缀nginx
28
            port:
29
              number: 8000

2.9.3 流量限制#

1
apiVersion: networking.k8s.io/v1
2
kind: Ingress
3
metadata:
4
  name: ingress-limit-rate
5
  annotations:
6
    nginx.ingress.kubernetes.io/limit-rps: "1"
7
spec:
8
  ingressClassName: nginx
9
  rules:
10
  - host: "haha.atguigu.com"
11
    http:
12
      paths:
13
      - pathType: Exact
14
        path: "/"
15
        backend:
16
          service:
17
            name: nginx-demo
18
            port:
19
              number: 8000

2.10 存储抽象#

2.10.1 环境准备#

所有节点：

1
#所有机器安装
2
yum install -y nfs-utils

主节点：

1
#nfs主节点
2
echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exports
3

4
mkdir -p /nfs/data
5
systemctl enable rpcbind --now
6
systemctl enable nfs-server --now
7
#配置生效
8
exportfs -r

从节点：

1
#更改为主节点的ip地址
2
showmount -e 192.168.59.168
3

4
#执行以下命令挂载 nfs 服务器上的共享目录到本机路径 /root/nfsmount
5

6

7
mkdir -p /nfs/data
8

9
mount -t nfs 192.168.59.168:/nfs/data /nfs/data
10
# 写入一个测试文件
11
echo "hello nfs server" > /nfs/data/test.txt

原生方式数据挂载

主节点创建目录/nfs/data/nginx-pv

1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  labels:
5
    app: nginx-pv-demo
6
  name: nginx-pv-demo
7
spec:
8
  replicas: 2
9
  selector:
10
    matchLabels:
11
      app: nginx-pv-demo
12
  template:
13
    metadata:
14
      labels:
15
        app: nginx-pv-demo
16
    spec:
17
      containers:
18
      - image: nginx
19
        name: nginx
20
        volumeMounts:
21
        - name: html
22
          mountPath: /usr/share/nginx/html
23
      volumes:
24
        - name: html
25
          nfs:
26
            server: 192.168.59.168
27
            path: /nfs/data/nginx-pv

2.10.2 pv-pvc#

1.创建pv池#

1
#nfs主节点
2
mkdir -p /nfs/data/01
3
mkdir -p /nfs/data/02
4
mkdir -p /nfs/data/03

1
apiVersion: v1
2
kind: PersistentVolume
3
metadata:
4
  name: pv01-10m
5
spec:
6
  capacity:
7
    storage: 10M
8
  accessModes:
9
    - ReadWriteMany
10
  storageClassName: nfs
11
  nfs:
12
    path: /nfs/data/01
13
    server: 192.168.59.168
14
---
15
apiVersion: v1
16
kind: PersistentVolume
17
metadata:
18
  name: pv02-1gi
19
spec:
20
  capacity:
21
    storage: 1Gi
22
  accessModes:
23
    - ReadWriteMany
24
  storageClassName: nfs
25
  nfs:
26
    path: /nfs/data/02
27
    server: 192.168.59.168
28
---
29
apiVersion: v1
30
kind: PersistentVolume
31
metadata:
32
  name: pv03-3gi
33
spec:
34
  capacity:
35
    storage: 3Gi
36
  accessModes:
37
    - ReadWriteMany
38
  storageClassName: nfs
39
  nfs:
40
    path: /nfs/data/03
41
    server: 192.168.59.168

2 PVC的创建和绑定#

创建pvc

1
kind: PersistentVolumeClaim
2
apiVersion: v1
3
metadata:
4
  name: nginx-pvc
5
spec:
6
  accessModes:
7
    - ReadWriteMany
8
  resources:
9
    requests:
10
      storage: 200Mi
11
  storageClassName: nfs

创建pod绑定pvc

1
apiVersion: apps/v1
2
kind: Deployment
3
metadata:
4
  labels:
5
    app: nginx-deploy-pvc
6
  name: nginx-deploy-pvc
7
spec:
8
  replicas: 2
9
  selector:
10
    matchLabels:
11
      app: nginx-deploy-pvc
12
  template:
13
    metadata:
14
      labels:
15
        app: nginx-deploy-pvc
16
    spec:
17
      containers:
18
      - image: nginx
19
        name: nginx
20
        volumeMounts:
21
        - name: html
22
          mountPath: /usr/share/nginx/html
23
      volumes:
24
        - name: html
25
          persistentVolumeClaim:
26
            claimName: nginx-pvc

2.10.3 ConfigMap#

1 把之前的配置文件创建为配置集

创建一个配置文件redis.conf

1
# 创建配置，redis保存到k8s的etcd；
2
kubectl create cm redis-conf --from-file=redis.conf

1
kubectl get cm
2
#查看当前configMap

创建yaml文件，并用kubectl apply导入

1
apiVersion: v1
2
data:    #data是所有真正的数据，key：默认是文件名   value：配置文件的内容
3
  redis.conf: |
4
    appendonly yes
5
kind: ConfigMap
6
metadata:
7
  name: redis-conf
8
  namespace: default

创建pod

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: redis
5
spec:
6
  containers:
7
  - name: redis
8
    image: redis
9
    command:
10
      - redis-server
11
      - "/redis-master/redis.conf"  #指的是redis容器内部的位置
12
    ports:
13
    - containerPort: 6379
14
    volumeMounts:
15
    - mountPath: /data
16
      name: data
17
    - mountPath: /redis-master
18
      name: config
19
  volumes:
20
    - name: data
21
      emptyDir: {}
22
    - name: config
23
      configMap:
24
        name: redis-conf
25
        items:
26
        - key: redis.conf
27
          path: redis.conf

检查默认配置

1
kubectl exec -it redis -- redis-cli
2

3
127.0.0.1:6379> CONFIG GET appendonly
4
127.0.0.1:6379> CONFIG GET requirepass

1
#修改redis.conf内容
2
kubectl edit cm redis-conf

然后查看，发现改变

2.11 secret#

Secret 对象类型用来保存敏感信息，例如密码、OAuth 令牌和 SSH 密钥。将这些信息放在 secret 中比放在 Pod 的定义或者容器镜像中来说更加安全和灵活。

1
kubectl create secret docker-registry leifengyang-docker \
2
--docker-username=leifengyang \
3
--docker-password=Lfy123456 \
4
--docker-email=534096094@qq.com
5

6
##命令格式
7
kubectl create secret docker-registry regcred \
8
  --docker-server=<你的镜像仓库服务器> \
9
  --docker-username=<你的用户名> \
10
  --docker-password=<你的密码> \
11
  --docker-email=<你的邮箱地址>

1
apiVersion: v1
2
kind: Pod
3
metadata:
4
  name: private-nginx
5
spec:
6
  containers:
7
  - name: private-nginx
8
    image: leifengyang/guignginx:v1.0
9
  imagePullSecrets:
10
  - name: leifengyang-docker

3 快速搭建kubernetes#

3.1 开通服务器#

4c8g；centos7.9；防火墙放行 30000~32767；指定hostname

1
hostnamectl set-hostname node1

3.2 快速安装#

官网教程#

在 Linux 上以 All-in-One 模式安装 KubeSphere

步骤 1：准备 Linux 机器#

若要以 All-in-One 模式进行安装，您仅需参考以下对机器硬件和操作系统的要求准备一台主机。

硬件推荐配置#

操作系统	最低配置
Ubuntu 16.04, 18.04, 20.04, 22.04	2 核 CPU，4 GB 内存，40 GB 磁盘空间
Debian Buster, Stretch	2 核 CPU，4 GB 内存，40 GB 磁盘空间
CentOS 7.x	2 核 CPU，4 GB 内存，40 GB 磁盘空间
Red Hat Enterprise Linux 7	2 核 CPU，4 GB 内存，40 GB 磁盘空间
SUSE Linux Enterprise Server 15/openSUSE Leap 15.2	2 核 CPU，4 GB 内存，40 GB 磁盘空间

依赖项要求#

KubeKey 可以将 Kubernetes 和 KubeSphere 一同安装。针对不同的 Kubernetes 版本，需要安装的依赖项可能有所不同。您可以参考以下列表，查看是否需要提前在节点上安装相关的依赖项。

依赖项	Kubernetes 版本 ≥ 1.18	Kubernetes 版本 < 1.18
socat	必须	可选但建议
conntrack	必须	可选但建议
ebtables	可选但建议	可选但建议
ipset	可选但建议	可选但建议

安装依赖组件#

也就是上面的Kubernetes依赖里的内容，全部安装
centos

1
yum install -y ebtables socat ipset conntrack

ubuntu

1
apt-get install -y ebtables socat ipset conntrack

基本配置要求#

安装 k8s 之前需要关闭 SELinux、关闭 Swap 分区、关闭防火墙

关闭SELinux#

centos#

1.查看 selinux 状态

1
sestatus

若系统返回的参数信息SELinux status显示为disabled，则表示SELinux已关闭。
2.临时关闭SeLinux

1
setenforce 0

3.永久关闭SeLinux ，运行运行以下命令，编辑SELinux的config配置文件。

1
vi /etc/selinux/config

找到SELINUX=enforcing或SELINUX=permissive字段，按i进入编辑模式，将参数修改为SELINUX=disabled。

ubuntu#

1
setenforce 0 && sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinuxconfig

关闭防火墙#

centos#

1
# 停止防火墙服务
2
systemctl stop firewalld
3
# 禁用防火墙服务
4
systemctl disable firewalld
5
# 查看防火墙状态
6
systemctl status firewalld

ubuntu#

1
# 停止防火墙服务
2
sudo systemctl stop ufw.service
3
# 禁用防火墙服务
4
sudo systemctl disable ufw.service
5
# 查看防火墙状态
6
sudo ufw status

如果输出：Status: inactive ，说明防火墙已经成功关闭！

关闭swap分区#

centos#

1
swapoff -a
2
echo "vm.swappiness=0" >> /etc/sysctl.conf
3
sysctl -p /etc/sysctl.conf

unbuntu#

1
swapoff -a && sed -ri 's/.*swap.*/#&/' /etc/fstab

步骤 2：下载 KubeKey#

从 GitHub Release Page 下载 KubeKey 或直接使用以下命令(ubuntu使用bash替换sh)。

1
curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

备注执行以上命令会下载最新版 KubeKey，您可以修改命令中的版本号下载指定版本。

为 kk 添加可执行权限：

1
chmod +x kk

步骤 3：开始安装#

只需执行一个命令即可进行安装，其模板如下所示：

1
./kk create cluster [--with-kubernetes version] [--with-kubesphere version]

同时安装 Kubernetes 和 KubeSphere

1
./kk create cluster --with-kubernetes v1.22.12 --with-kubesphere v3.4.1

安装 KubeSphere 3.4 的建议 Kubernetes 版本：v1.20.x、v1.21.x、v1.22.x、v1.23.x、* v1.24.x、* v1.25.x 和 * v1.26.x。带星号的版本可能出现边缘节点部分功能不可用的情况。因此，如需使用边缘节点，推荐安装 v1.23.x。如果不指定 Kubernetes 版本，KubeKey 将默认安装 Kubernetes v1.23.10。有关受支持的 Kubernetes 版本的更多信息，请参见支持矩阵。
一般来说，对于 All-in-One 安装，您无需更改任何配置。
如果您在这一步的命令中不添加标志 --with-kubesphere，则不会部署 KubeSphere，KubeKey 将只安装 Kubernetes。如果您添加标志 --with-kubesphere 时不指定 KubeSphere 版本，则会安装最新版本的 KubeSphere。
KubeKey 会默认安装 OpenEBS 为开发和测试环境提供 LocalPV 以方便新用户。对于其他存储类型，请参见持久化存储配置。

执行该命令后，KubeKey将检查您的安装环境，结果显示在一张表格中。有关详细信息，请参见节点要求和依赖项要求。输入yes 继续安装流程

步骤 4：验证安装结果#

输入以下命令以检查安装结果。

1
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

输出信息会显示 Web 控制台的 IP 地址和端口号，默认的 NodePort 是 30880。现在，您可以使用默认的帐户和密码 (admin/P@88w0rd) 通过 :30880 访问控制台。

1
#####################################################
2

3
###              Welcome to KubeSphere!           ###
4

5
#####################################################
6

7

8

9
Console: http://192.168.0.2:30880
10

11
Account: admin
12

13
Password: P@88w0rd
14

15

16

17
NOTES：
18

19
  1. After you log into the console, please check the
20

21
     monitoring status of service components in
22

23
     "Cluster Management". If any service is not
24

25
     ready, please wait patiently until all components
26

27
     are up and running.
28

29
  2. Please change the default password after login.
30

31

32

33
#####################################################
34

35
https://kubesphere.io             20xx-xx-xx xx:xx:xx
36

37
#####################################################

备注您可能需要配置端口转发规则并在安全组中开放端口，以便外部用户访问控制台。

登录至控制台后，您可以在系统组件中查看各个组件的状态。如果要使用相关服务，您可能需要等待部分组件启动并运行。您也可以使用 kubectl get pod \--all-namespaces 来检查 KubeSphere 相关组件的运行状况。

开启KubeSphere 应用商店#

KubeSphere 应用商店
使用admin登录Kubersphere平台，进入集群管理页面，然后点击【定制资源定义】，在右边搜索框搜索ClusterConfiguration，如下，然后点击搜索到的结果

然后点击右边的三个点，在弹出的菜单中点击【编辑YAML】

搜索 openpitrix，并将 enabled 的 false 改为 true，完成后保存

然后可以到kubernetes集群后台执行如下命令查看启用日志

1
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f

安装日志出现如下内容，表示安装成功

安装完成后，Kubersphere平台会显示应用商店，点击进去如下所示

也可以在不登录控制台的情况下直接访问 <节点 IP 地址>:30880/apps 进入应用商店。
如 http://192.168.3.120:30880/apps
且KubeSphere 3.2.x 中的应用商店启用后，OpenPitrix 页签不会显示在系统组件页面。

启用可插拔组件#

启用可插拔组件

概述#

从 2.1.0 版本开始，KubeSphere 解耦了一些核心功能组件。这些组件设计成了可插拔式，您可以在安装之前或之后启用它们。如果您不启用它们，KubeSphere 会默认以最小化进行安装部署。
不同的可插拔组件部署在不同的命名空间中。您可以根据需求启用任意组件。强烈建议您安装这些可插拔组件来深度体验 KubeSphere 提供的全栈特性和功能。

资源要求#

在您启用可插拔组件之前，请确保您的环境中有足够的资源，具体参见下表。否则，可能会因为缺乏资源导致组件崩溃。

⚠️ 注意 CPU 和内存的资源请求和限制均指单个副本的要求。

参考文献#

Ubuntu关闭防火墙、关闭selinux、关闭swap_ubuntu 关闭selinux-CSDN博客
 Llinux 开启或关闭SELinux - 飞行日志 - 博客园
 安装部署KubeSphere管理kubernetes_error: pipeline[createclusterpipeline] execute fai-CSDN博客
 使用kubekey部署k8s集群和kubesphere、在已有k8s集群上部署kubesphere-CSDN博客
 Kubersphere----Kubersphere启用应用商店_kubesphere开启应用商店-CSDN博客
 Kubesphere应用商店_kubesphere开启应用商店-CSDN博客
 kubernetes之5—kubesphere管理平台 - 原因与结果 - 博客园

3.3 通过minikube安装#

安装 Minikube#

Minikube 是一个单文件的二进制包，安装十分简单，在已经完成Docker 安装的前提下，使用以下命令可以下载并安装最新版的 Minikube。

1
 curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

安装 Kubectl 工具#

Minikube 中很多提供了许多子命令以代替 Kubectl 的功能，安装 Minikube 时并不会一并安装 Kubectl。但是 Kubectl 作为集群管理的命令行，要了解 Kubernetes 是无论如何绕不过去的，通过以下命令可以独立安装 Kubectl 工具。

1
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/am

启动 Kubernetes 集群#

有了 Minikube，通过 start 子命令就可以一键部署和启动 Kubernetes 集群了，具体命令如下：

1
minikube start --iso-url=https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/iso/minikube-v1.6.0.iso --registry-mirror=https://registry.docker-cn.com --image-mirror-country=cn --image-repository=registry.cn-hangzhou.aliyuncs.com/google_containers --vm-driver=none --memory=2200mb

然后看报错，需要什么安装什么

4 快速搭建kubernetes集群#

4.1 修改linux主机名#

hostnamectl set-hostname master

4.2 在master节点下载KubeKey#

export KKZONE=cn

curl -sfL https://get-kk.kubesphere.io | VERSION=v1.1.1 sh -

chmod +x kk

4.3 创建集群配置文件#

1
./kk create config --with-kubernetes v1.20.4 --with-kubesphere v3.1.1

修改配置

4.4 创建集群#

1
./kk create cluster -f config-sample.yaml

5 使用kubephere创建mysql#

5.1 设置配置#

5.2 创建mysql副本#

下一步

5.3 开放集群访问#

点击创建就创建成功

然后通过此端口就可外网访问

6 通过应用商店部署rabbitmq#

6.1 kebesphere开放应用商店#

已安装过KS通过管理界面进行设置若之前已安装过KS，则可通过KS管理界面左侧菜单CRD -> 搜索clusterconfiguration -> 然后编辑其下资源ks-installer，如下图：

同样设置openpitrix.store.enabled为true：

Kubesphere 应用商店详细开启说明可参见： https://kubesphere.io/zh/docs/pluggable-components/app-store/

发现新文章

1 kubeadm创建集群#

1.1 安装kubeadm#

1.2 为所有机器执行下面操作#

1.3 安装kubelet、kubeadm、kubectl#

1.4 下载各个机器需要的镜像#

1.5 初始化主节点#

1.6 主节点执行上面展示的命令#

1.7 部署网络插件#

1.8 修改配置#

1.9 查看部署k8s所有集群#

1.10 worker节点加入集群#

1.11 查看启动情况#

1.12 安装可视化界面#

1.12.1 在~目录下面新建dashboard.yaml文件#

1.12.2 应用yaml#

1.12.3 设置访问端口#

1.12.4 找到端口，安全组放行#

1.12.5 创建访问账号#

2 控制Pod#

2.0 名称空间#

2.1 创建pod#

2.2 多副本#

2.3 扩缩容#

2.4 滚动更新#

2.5 版本回退#

2.6 service#

2.7 ClusterIP#

2.8 NodePort#

2.9 ingress#

2.9.1 ingress域名访问#

2.9.2 路径重写#

2.9.3 流量限制#

2.10 存储抽象#

2.10.1 环境准备#

2.10.2 pv-pvc#

1.创建pv池#

2 PVC的创建和绑定#

2.10.3 ConfigMap#

2.11 secret#

3 快速搭建kubernetes#

3.1 开通服务器#

3.2 快速安装#

官网教程#

步骤 1：准备 Linux 机器#

硬件推荐配置#

依赖项要求#

安装依赖组件#

基本配置要求#

关闭SELinux#

centos#

ubuntu#

关闭防火墙#

centos#

ubuntu#

关闭swap分区#

centos#

unbuntu#

步骤 2：下载 KubeKey#

步骤 3：开始安装#

步骤 4：验证安装结果#

开启KubeSphere 应用商店#

启用可插拔组件#

概述#

资源要求#

参考文献#

相关视频#

3.3 通过minikube安装#

安装 Minikube#

安装 Kubectl 工具#

启动 Kubernetes 集群#

4 快速搭建kubernetes集群#

4.1 修改linux主机名#

4.2 在master节点下载KubeKey#

4.3 创建集群配置文件#

4.4 创建集群#

5 使用kubephere创建mysql#

5.1 设置配置#

5.2 创建mysql副本#

5.3 开放集群访问#