持续集成系列------容器编排平台k8s多主集群搭建

 

文章目录

持续集成系列------容器编排平台k8s多主集群搭建

k8s简介

环境准备

升级内核

docker安装

安装kubeadm,kubelet,kubectl

配置系统相关参数

集群高可用

配置kubelet

ssh免密配置

配置k8s1

配置k8s2

配置k8s3

配置kubectl

配置网络组件calico(任意一台master执行)

配置命令自动补全

配置前端负载均衡 traefik

traefik实现https代理（全网最简单方式）

dashboard部署

应用部署示例（jenkins）

加入新node节点

troubleshoot

 

## 目标 gitlab+jenkins+docker+harbor+k8s初步实现持续集成

k8s简介

Kubernetes（k8s）是自动化容器操作的开源平台，这些操作包括部署，调度和节点集群间扩展。

 

使用Kubernetes可以：

 

自动化容器的部署和复制

随时扩展或收缩容器规模

将容器组织成组，并且提供容器间的负载均衡

很容易地升级应用程序容器的新版本

提供容器弹性，如果容器失效就替换它，等等…

Kubernetes解决的问题：

 

调度 - 容器应该在哪个机器上运行

生命周期和健康状况 - 容器在无错的条件下运行

服务发现 - 容器在哪，怎样与它通信

监控 - 容器是否运行正常

认证 - 谁能访问容器

容器聚合 - 如何将多个容器合并成一个工程

Kubernetes组件组成：

 

kubectl

客户端命令行工具，将接受的命令格式化后发送给kube-apiserver，作为整个系统的操作入口。

kube-apiserver

作为整个系统的控制入口，以REST API服务提供接口。

kube-controller-manager

用来执行整个系统中的后台任务，包括节点状态状况、Pod个数、Pods和Service的关联等。

kube-scheduler

负责节点资源管理，接受来自kube-apiserver创建Pods任务，并分配到某个节点。

etcd

负责节点间的服务发现和配置共享。

kube-proxy

运行在每个计算节点上，负责Pod网络代理。定时从etcd获取到service信息来做相应的策略。

kubelet

运行在每个计算节点上，作为agent，接受分配该节点的Pods任务及管理容器，周期性获取容器状态，反馈给kube-apiserver。

DNS

一个可选的DNS服务，用于为每个Service对象创建DNS记录，这样所有的Pod就可以通过DNS访问服务了。

 

环境准备

系统:Centos7.4

docker v17.03

k8s 1.11.2

域名:k8s.domain.com

ip hostname role

10.79.167.26 k8s1 master

10.79.167.27 k8s2 master

10.79.167.28 k8s3 master

10.79.166.8 vip

升级内核

rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org

rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm

yum --enablerepo=elrepo-kernel install kernel-ml -y

sed -i s/saved/0/g /etc/default/grub

grub2-set-default "$(cat /boot/grub2/grub.cfg |grep menuentry|grep 'menuentry '|head -n 1|awk -F "'" '{print $2}')"

#查看默认启动版本

grub2-editenv list

grub2-mkconfig -o /boot/grub2/grub.cfg && reboot

 

 

docker安装

# 安装依赖包

yum install -y yum-utils device-mapper-persistent-data lvm2

# 添加docker源

yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo

# 安装docker

yum install -y --setopt=obsoletes=0 \

docker-ce-17.03.2.ce-1.el7.centos \

docker-ce-selinux-17.03.2.ce-1.el7.centos \

docker-compose

# 切换国内hub

curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://ef017c13.m.daocloud.io

cat >/etc/docker/daemon.json<<EOF

{"registry-mirrors": ["http://ef017c13.m.daocloud.io"],"live-restore": true,"storage-driver": "overlay2"}

EOF

# 启动

systemctl enable docker

systemctl start docker

 

 

安装kubeadm,kubelet,kubectl

#配置源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo

[kubernetes]

name=Kubernetes

baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64

enabled=1

gpgcheck=1

repo_gpgcheck=1

gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

EOF

 

#安装

yum install -y kubelet kubeadm kubectl ipvsadm

 

 

配置系统相关参数

sed -i 's/SELINUX=permissive/SELINUX=disabled/' /etc/sysconfig/selinux

setenforce 0

systemctl stop firewalld 

systemctl disable firewalld

swapoff -a && sysctl -w vm.swappiness=0

iptables -P FORWARD ACCEPT

# 配置转发相关参数，否则可能会出错

cat <<EOF >  /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables=1

net.bridge.bridge-nf-call-iptables=1

vm.swappiness=0

net.ipv4.ip_forward=1

EOF

sysctl --system

# /etc/fstab中swap相关的需要删除，否则会导致重启时kubelet启动失败

# 加载ipvs相关内核模块

cat >>/etc/profile<<EOF

modprobe ip_vs

modprobe ip_vs_rr

modprobe ip_vs_wrr

modprobe ip_vs_sh

modprobe nf_conntrack_ipv4

EOF

source /etc/profile

cat >>/etc/hosts <<EOF

10.79.167.26 k8s1.local

10.79.167.27 k8s2.local

10.79.167.28 k8s3.local

EOF

 

 

集群高可用

#安装Pacemake Corosync haproxy

yum install corosync pacemaker pcs fence-agents resource-agents -y

yum install haproxy -y

#启动pcsd

systemctl enable pcsd.service

systemctl start pcsd.service

#修改群集用户hacluster密码

echo centos | passwd --stdin hacluster

 

#其中一节点执行 k8s1

#创建、启动my_cluster集群

pcs cluster auth -u hacluster -p centos k8s1 k8s2 k8s3

pcs cluster setup --start --name my_cluster k8s1 k8s2 k8s3

pcs cluster enable --all #集群自启动

pcs cluster start  --all # 启动集群

pcs cluster status #集群状态

 

#检验

corosync-cfgtool -s             #验证corosync

corosync-cmapctl| grep members  #查看成员

pcs status corosync             #查看corosync状态

 

pcs property set stonith-enabled=false  #禁用STONITH

pcs property set no-quorum-policy=ignore   #无仲裁时，选择忽略

crm_verify -L -V  #检查配置

 

#创建 VIP 资源

pcs resource create vip ocf:heartbeat:IPaddr2 ip=10.79.166.8 cidr_netmask=21 op monitor interval=28s

 

#查看集群状态

pcs status

 

#haproxy配置

cat >/etc/haproxy/haproxy.cfg<<EOF

global

  log 127.0.0.1 local0 err

  maxconn 50000

  uid 99

  gid 99

  #daemon

  nbproc 1

  pidfile haproxy.pid

 

defaults

  mode http

  log 127.0.0.1 local0 err

  maxconn 50000

  retries 3

  timeout connect 5s

  timeout client 30s

  timeout server 30s

  timeout check 2s

 

listen admin_stats

  mode http

  bind 0.0.0.0:1080

  log 127.0.0.1 local0 err

  stats refresh 30s

  stats uri     /haproxy-status

  stats realm   Haproxy\ Statistics

  stats auth    admin:admin

  stats hide-version

  stats admin if TRUE

 

frontend k8s-https

  bind 0.0.0.0:8443

  mode tcp

  #maxconn 50000

  default_backend k8s-https

 

backend k8s-https

  mode tcp

  balance roundrobin

  server k8s1 10.79.167.26:6443 weight 1 maxconn 1000 check inter 2000 rise 2 fall 3

  server k8s2 10.79.167.27:6443 weight 1 maxconn 1000 check inter 2000 rise 2 fall 3

  server k8s3 10.79.167.28:6443 weight 1 maxconn 1000 check inter 2000 rise 2 fall 3

EOF

systemctl enable haproxy

systemctl start haproxy

# 监控haproxy运行状态

pcs resource create haproxy systemd:haproxy op monitor interval=5s --clone

 

 

 

配置kubelet

DOCKER_CGROUPS=$(docker info | grep 'Cgroup' | cut -d' ' -f3)

echo $DOCKER_CGROUPS

cat >/etc/sysconfig/kubelet<<EOF

KUBELET_EXTRA_ARGS="--cgroup-driver=$DOCKER_CGROUPS --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause-amd64:3.1"

EOF

 

# 启动

systemctl daemon-reload

systemctl enable kubelet && systemctl restart kubelet

 

 

ssh免密配置

# k8s1上

ssh-keygen

ssh-copy-id k8s2

ssh-copy-id k8s3

 

 

配置k8s1

# 生成配置文件

mkdir -p /opt/kubeadm-cfg && cd /opt/kubeadm-cfg

CP0_IP="10.79.167.26"

CP0_HOSTNAME="k8s1"

cat >kubeadm-master.config<<EOF

apiVersion: kubeadm.k8s.io/v1alpha2

kind: MasterConfiguration

kubernetesVersion: v1.11.1

imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers

 

apiServerCertSANs:

- "k8s1"

- "k8s2"

- "k8s3"

- "10.79.167.26"

- "10.79.167.27"

- "10.79.167.28"

- "10.79.166.8"

- "127.0.0.1"

 

api:

  advertiseAddress: $CP0_IP

  controlPlaneEndpoint: 10.79.166.8:8443

 

etcd:

  local:

    extraArgs:

      listen-client-urls: "https://127.0.0.1:2379,https://$CP0_IP:2379"

      advertise-client-urls: "https://$CP0_IP:2379"

      listen-peer-urls: "https://$CP0_IP:2380"

      initial-advertise-peer-urls: "https://$CP0_IP:2380"

      initial-cluster: "$CP0_HOSTNAME=https://$CP0_IP:2380"

    serverCertSANs:

      - $CP0_IP

    peerCertSANs:

      - $CP0_IP

 

controllerManagerExtraArgs:

  node-monitor-grace-period: 10s

  pod-eviction-timeout: 10s

 

networking:

  podSubnet: 10.244.0.0/16

  

kubeProxy:

  config:

    mode: ipvs

EOF

 

# 提前拉取镜像

# 如果执行失败 可以多次执行

kubeadm config images pull --config kubeadm-master.config

 

# 初始化

# 注意保存返回的 join 命令

kubeadm init --config kubeadm-master.config

mkdir -p $HOME/.kube

\cp /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

 

# 打包ca相关文件上传至其他master节点

cd /etc/kubernetes && tar cvzf k8s-key.tgz admin.conf pki/ca.* pki/sa.* pki/front-proxy-ca.* pki/etcd/ca.*

scp k8s-key.tgz k8s2:~/

scp k8s-key.tgz k8s3:~/

ssh k8s2 'tar xf k8s-key.tgz -C /etc/kubernetes/'

ssh k8s3 'tar xf k8s-key.tgz -C /etc/kubernetes/'

 

 

 

 

配置k8s2

mkdir -p /opt/kubeadm-cfg && cd /opt/kubeadm-cfg

# 生成配置文件

CP0_IP="10.79.167.26"

CP0_HOSTNAME="k8s1"

CP1_IP="10.79.167.27"

CP1_HOSTNAME="k8s2"

cat >kubeadm-master.config<<EOF

apiVersion: kubeadm.k8s.io/v1alpha2

kind: MasterConfiguration

kubernetesVersion: v1.11.1

imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers

 

apiServerCertSANs:

- "k8s1"

- "k8s2"

- "k8s3"

- "10.79.167.26"

- "10.79.167.27"

- "10.79.167.28"

- "10.79.166.8"

- "127.0.0.1"

 

api:

  advertiseAddress: $CP1_IP

  controlPlaneEndpoint: 10.79.166.8:8443

 

etcd:

  local:

    extraArgs:

      listen-client-urls: "https://127.0.0.1:2379,https://$CP1_IP:2379"

      advertise-client-urls: "https://$CP1_IP:2379"

      listen-peer-urls: "https://$CP1_IP:2380"

      initial-advertise-peer-urls: "https://$CP1_IP:2380"

      initial-cluster: "$CP0_HOSTNAME=https://$CP0_IP:2380,$CP1_HOSTNAME=https://$CP1_IP:2380"

      initial-cluster-state: existing

    serverCertSANs:

      - $CP1_HOSTNAME

      - $CP1_IP

    peerCertSANs:

      - $CP1_HOSTNAME

      - $CP1_IP

 

controllerManagerExtraArgs:

  node-monitor-grace-period: 10s

  pod-eviction-timeout: 10s

 

networking:

  podSubnet: 10.244.0.0/16

  

kubeProxy:

  config:

    mode: ipvs

EOF

 

# 配置kubelet

kubeadm alpha phase certs all --config kubeadm-master.config

kubeadm alpha phase kubelet config write-to-disk --config kubeadm-master.config

kubeadm alpha phase kubelet write-env-file --config kubeadm-master.config

kubeadm alpha phase kubeconfig kubelet --config kubeadm-master.config

systemctl restart kubelet

 

# 添加etcd到集群中

rm -rf $HOME/.kube/*

mkdir -p $HOME/.kube

\cp /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

KUBECONFIG=$HOME/.kube/config

kubectl exec -n kube-system etcd-${CP0_HOSTNAME} -- etcdctl --ca-file /etc/kubernetes/pki/etcd/ca.crt --cert-file /etc/kubernetes/pki/etcd/peer.crt --key-file /etc/kubernetes/pki/etcd/peer.key --endpoints=https://${CP0_IP}:2379 member add ${CP1_HOSTNAME} https://${CP1_IP}:2380

kubeadm alpha phase etcd local --config kubeadm-master.config

 

# 提前拉取镜像

# 如果执行失败 可以多次执行

kubeadm config images pull --config kubeadm-master.config

 

# 部署

kubeadm alpha phase kubeconfig all --config kubeadm-master.config

kubeadm alpha phase controlplane all --config kubeadm-master.config

kubeadm alpha phase mark-master --config kubeadm-master.config

 

 

 

配置k8s3

# 生成配置文件

mkdir /opt/kubeadm-cfg && cd /opt/kubeadm-cfg

CP0_IP="10.79.167.26"

CP0_HOSTNAME="k8s1"

CP1_IP="10.79.167.27"

CP1_HOSTNAME="k8s2"

CP2_IP="10.79.167.28"

CP2_HOSTNAME="k8s3"

cat >kubeadm-master.config<<EOF

apiVersion: kubeadm.k8s.io/v1alpha2

kind: MasterConfiguration

kubernetesVersion: v1.11.1

imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers

 

apiServerCertSANs:

- "k8s1"

- "k8s2"

- "k8s3"

- "10.79.167.26"

- "10.79.167.27"

- "10.79.167.28"

- "10.79.166.8"

- "127.0.0.1"

 

api:

  advertiseAddress: $CP2_IP

  controlPlaneEndpoint: 10.79.166.8:8443

 

etcd:

  local:

    extraArgs:

      listen-client-urls: "https://127.0.0.1:2379,https://$CP2_IP:2379"

      advertise-client-urls: "https://$CP2_IP:2379"

      listen-peer-urls: "https://$CP2_IP:2380"

      initial-advertise-peer-urls: "https://$CP2_IP:2380"

      initial-cluster: "$CP0_HOSTNAME=https://$CP0_IP:2380,$CP1_HOSTNAME=https://$CP1_IP:2380,$CP2_HOSTNAME=https://$CP2_IP:2380"

      initial-cluster-state: existing

    serverCertSANs:

      - $CP2_HOSTNAME

      - $CP2_IP

    peerCertSANs:

      - $CP2_HOSTNAME

      - $CP2_IP

 

controllerManagerExtraArgs:

  node-monitor-grace-period: 10s

  pod-eviction-timeout: 10s

 

networking:

  podSubnet: 10.244.0.0/16

  

kubeProxy:

  config:

    mode: ipvs

EOF

 

# 配置kubelet

kubeadm alpha phase certs all --config kubeadm-master.config

kubeadm alpha phase kubelet config write-to-disk --config kubeadm-master.config

kubeadm alpha phase kubelet write-env-file --config kubeadm-master.config

kubeadm alpha phase kubeconfig kubelet --config kubeadm-master.config

systemctl restart kubelet

 

# 添加etcd到集群中

rm -rf $HOME/.kube/*

mkdir -p $HOME/.kube

\cp /etc/kubernetes/admin.conf $HOME/.kube/config

chown $(id -u):$(id -g) $HOME/.kube/config

KUBECONFIG=$HOME/.kube/config

kubectl exec -n kube-system etcd-${CP0_HOSTNAME} -- etcdctl --ca-file /etc/kubernetes/pki/etcd/ca.crt --cert-file /etc/kubernetes/pki/etcd/peer.crt --key-file /etc/kubernetes/pki/etcd/peer.key --endpoints=https://${CP0_IP}:2379 member add ${CP2_HOSTNAME} https://${CP2_IP}:2380

kubeadm alpha phase etcd local --config kubeadm-master.config

 

# 提前拉取镜像

# 如果执行失败 可以多次执行

kubeadm config images pull --config kubeadm-master.config

 

# 部署

kubeadm alpha phase kubeconfig all --config kubeadm-master.config

kubeadm alpha phase controlplane all --config kubeadm-master.config

kubeadm alpha phase mark-master --config kubeadm-master.config

 

 

 

 

配置kubectl

 

# 查看node节点

kubectl get nodes

 

# 只有网络插件也安装配置完成之后，才能会显示为ready状态

# 设置master允许部署应用pod，参与工作负载

kubectl taint nodes --all node-role.kubernetes.io/master-

 

 

配置网络组件calico(任意一台master执行)

# 下载配置

cd /opt/kubeadm-cfg && mkdir kube-calico && cd kube-calico

wget https://docs.projectcalico.org/v3.2/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml

wget https://docs.projectcalico.org/v3.2/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml

sed -i 's/192.168/10.244/g' calico.yaml

kubectl apply -f .

 

# 查看

kubectl get pods --namespace kube-system

kubectl get svc --namespace kube-system

 

 

配置命令自动补全

yum install -y bash-completion

source /usr/share/bash-completion/bash_completion

source <(kubectl completion bash)

echo "source <(kubectl completion bash)" >> ~/.bashrc

 

 

配置前端负载均衡 traefik

参考文档

traefik可以两种方式运行，rs方式以及daemonset方式，这里选用daemonset方式，即每台node都运行一个traefik实例参与负载均衡

 

 

mkdir traefik-ds && cd traefik-ds

wget https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/traefik-rbac.yaml

wget https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/traefik-ds.yaml

wget https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/ui.yaml

 

 

修改ui.yaml中host地址为自己的domain地址

 

修改traefik-ds.yaml，添加nodeSelector,让traefik只部署在master节点

 

 

#标记master节点用于部署traefik

kubectl label nodes k8s1 zone=traefik

kubectl label nodes k8s2 zone=traefik

kubectl label nodes k8s3 zone=traefik

kubectl apply -f .

 

 

访问： http://10.79.167.26:8080 或traefik.domain.com（提前做好解析）

 

 

traefik实现https代理（全网最简单方式）

参考文章–kubernets Traefik 的HTTP 和HTTPS

参考文章–kubernetes使用traefik的https方式访问web应用

参考文章–kubernetes traefik配置https实践操作记录

提前进行：安装acme.sh并生成*.domain.com的泛域名证书

 

https这步踩了很多的坑，网上的教程大多只针对一个证书的情况，自己结合官方文档摸索出了绑定多证书的方法。。

 

#创建证书secret

kubectl -n kube-system create secret generic traefik-cert --from-file=/opt/certs/domain.com.cert --from-file=/opt/certs/domain.com.key

 

#开启https并绑定默认证书

cat >configmap.yaml<<EOF

apiVersion: v1

kind: ConfigMap

metadata:

  name: traefik-conf

  namespace: kube-system

data:

  traefik.toml: |

    defaultEntryPoints = ["http","https"]

    insecureSkipVerify = true

    [entryPoints]

      [entryPoints.http]

      address = ":80"

      [entryPoints.https]

      address = ":443"

        [entryPoints.https.tls]

          [[entryPoints.https.tls.certificates]]

          certFile = "/ssl/domain.com.cert"

          keyFile = "/ssl/domain.com.key"

EOF

#traefik-ds.yaml中添加443端口、挂载配置以及证书

 

 

 

 

#配置生效

kubectl apply -f

 

 

这里实现了单个证书的配置，想要实现多个证书的绑定，以下是全网最简单的方式

 

#创建tls类型的证书,注意不同namespace的证书secret不能通用，需要创建多个

kubectl create secret tls domain2-cert --cert=/opt/certs/domain2.com.cert --key=/opt/certs/domain2.com.key -n kube-system

 

 

需要绑定证书的ingress直接在配置文件中绑定对应的secret

 

 

 

dashboard部署

图形化界面更直观

 

cd /opt/kubeadm-cfg && mkdir kube-dashboard && cd kube-dashboard

wget https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/alternative/kubernetes-dashboard.yaml

#创建rbac授权

cat >rbac.yaml<<EOF

apiVersion: rbac.authorization.k8s.io/v1beta1

kind: ClusterRoleBinding

metadata:

  name: kubernetes-dashboard

  labels:

    k8s-app: kubernetes-dashboard

roleRef:

  apiGroup: rbac.authorization.k8s.io

  kind: ClusterRole

  name: cluster-admin

subjects:

- kind: ServiceAccount

  name: kubernetes-dashboard

  namespace: kube-system

EOF

#创建ingress

cat >ingress.yaml<EOF

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

  name: k8s-web-ui

  annotations:

    kubernetes.io/ingress.class: traefik

  namespace: kube-system

spec:

  rules:

  - host: k8s.domain.com

    http:

      paths:

      - path: /

        backend:

          serviceName: kubernetes-dashboard

          servicePort: 80

EOF

kubectl apply -f .

 

 

 

 

应用部署示例（jenkins）

以从私仓拉取部署jenkins为例

私仓地址为hub.domain.com

 

#创建secret用于登陆私仓

kubectl create secret docker-registry hubsecret --docker-server=hub.domain.com --docker-username=admin --docker-password=Harbor12345

#创建配置文件

cat >jenkins-templete.yaml<<EOF

---

apiVersion: apps/v1

kind: Deployment

metadata:

  name: jenkins

spec:

  replicas: 1

  strategy:

    type: RollingUpdate

    rollingUpdate:

      maxSurge: 2

      maxUnavailable: 0

  selector:

    matchLabels:

      app: jenkins

  template:

    metadata:

      labels:

        app: jenkins

    spec:

      imagePullSecrets:

      - name: hubsecret

      containers:

      - name: jenkins

        image: hub.domain.com/docker/jenkins:1.0

        imagePullPolicy: IfNotPresent

        ports:

        - containerPort: 8080

          name: web

          protocol: TCP

        - containerPort: 50000

          name: agent

          protocol: TCP

        volumeMounts:

        - name: jenkinshome

          mountPath: /var/jenkins_home

        env:

        - name: JAVA_OPTS

          value: "-Duser.timezone=Asia/Shanghai"

      volumes:

      - name: jenkinshome

        nfs:

          server: 10.79.163.29

          path: "/data/jenkins_home"

---

kind: Service

apiVersion: v1

metadata:

  labels:

      app: jenkins

  name: jenkins

spec:

  ports:

  - port: 8080

    targetPort: 8080

    name: web

  - port: 50000

    targetPort: 50000

    name: agent

  selector:

    app: jenkins

---

apiVersion: extensions/v1beta1

kind: Ingress

metadata:

  name: jenkins

  annotations:

    kubernetes.io/ingress.class: traefik

spec:

  rules:

  - host: jenkins.domain.com

    http:

      paths:

      - path: /

        backend:

          serviceName: jenkins

          servicePort: 8080

EOF

 

 

加入新node节点

升级内核

docker安装

安装kubeadm,kubelet,kubectl

配置系统相关参数

配置kubelet

 

#开放防火墙权限（或加入对应端口权限）

iptables -P INPUT ACCEPT

iptables -P OUTPUT ACCEPT

iptables -P FORWARD ACCEPT

#添加解析到hosts文件（所有k8s节点都要加）

cat >>/etc/hosts<<EOF

10.79.167.26  k8s1

10.79.167.27  k8s2

10.79.167.28  k8s3

10.79.160.50  node1

10.79.160.51  node2

10.79.160.65  node3

10.79.160.67  node4

EOF

 

 

# master节点查看node加入命令

kubeadm token create --print-join-command

# 得到的kubeadm join命令在node节点执行

# 提示token过期解决方法

token=$(kubeadm token generate)

kubeadm token create $token --print-join-command --ttl=0

 

 

troubleshoot

etcd集群排错