K8S learning 003 -- using kubeadm to deploy the installation cluster

Article directory


1: Environmental preparation

  • VMware software
  • One centos7 virtual machine as the master node, IP address:, minimum number of CPU cores: 2
  • One centos7 virtual machine as node01 node, IP address:
  • 1 centos7 virtual machine as node02 node, IP address:

2: Cluster deployment process

2.1: set system initialization

  • Modify the host names of three virtual machines

    [root@192 ~]# hostnamectl set-hostname k8s-master	'//In the same way, modify the node host names k8s-node01 and node02 '
    [root@192 ~]# su
    [root@k8s-master ~]# 
  • The three nodes can be associated with each other. The configuration in the master node is as follows

    [root@k8s-master ~]# vim /etc/hosts   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6 k8s-master k8s-node01 k8s-node02
    [root@k8s-master ~]# scp /etc/hosts root@k8s-node01:/etc/hosts	'//Copy it to node01 node '
        '//You need to enter the password of yes and node01 nodes during the period '
    [root@k8s-master ~]# scp /etc/hosts root@k8s-node01:/etc/hosts
2.1.1: three nodes initialize the system, showing only the operation of the master node
  • Install dependency package

    [root@k8s-master ~]# yum install -y \
    > conntrack \
    > ntpdate \
    > ntp \
    > ipvsadm \
    > ipset \
    > jq \
    > iptables \
    > curl \
    > sysstat \
    > libseccomp \
    > wget  \
    > vim net-tools git
  • Clear firewall rules

    [root@k8s-master ~]# systemctl stop firewalld && systemctl disable firewalld	'//Turn off the firewall and set it to not start automatically "
    [root@k8s-master ~]# yum -y install iptables-services && systemctl start iptables && systemctl enable iptables && iptables -F && service iptables save	'//Install iptables service, turn on iptables and set it to power on automatically, clear iptables rules and save iptables configuration '
  • Turn off the SELINUX and swap functions. If the pod is placed in the swap, the work efficiency will be reduced

    [root@k8s-master ~]# swapoff -a && sed -i '/swap/s/^\(.*\)$/#\1/g' /etc/fstab	'//Close the swap partition [virtual memory] and permanently close the virtual memory. '
    [root@k8s-master ~]# setenforce 0 && sed -i 's/^SELINUX=.*/SELINUX=disabled/' /etc/selinux/config	'//Close selinux '
  • Adjust kernel parameters

    [root@k8s-master ~]# cat > kubernetes.conf <<EOF
    > net.bridge.bridge-nf-call-iptables=1	'//Turn on bridge mode '
    > net.bridge.bridge-nf-call-ip6tables=1	'//Turn on bridge mode '
    > net.ipv4.ip_forward=1
    > net.ipv4.tcp_tw_recycle=0
    > vm.swappiness=0	'//Swap swap space is prohibited, only allowed when the system is OOM '
    > vm.overcommit_memory=1	'//Don't check if physical memory is sufficient '
    > vm.panic_on_oom=0 	'//Open OOM'
    > fs.inotify.max_user_instances=8192
    > fs.inotify.max_user_watches=1048576
    > fs.file-max=52706963
    > fs.nr_open=52706963
    > net.ipv6.conf.all.disable_ipv6=1	'//Close ipv6'
    > net.netfilter.nf_conntrack_max=2310720
    > EOF
    [root@k8s-master ~]# cp kubernetes.conf  /etc/sysctl.d/kubernetes.conf	'//Put the file in the sysctl.d directory and adjust these kernel parameters at startup. '
    [root@k8s-master ~]# modprobe br_netfilter
    [root@k8s-master ~]# sysctl -p /etc/sysctl.d/kubernetes.conf	'//Make it effective immediately '
    If prompted: sysctl:cannot stat /proc/sys/net/netfilter/nf_conntrack_max: There is no file or directory:##Because the kernel version is less than 4.0, you can update the kernel later
  • Adjust system time zone: if you choose Shanghai when installing CentOS 7, you do not need to change it

    #Set the system time zone to China / Shanghai
    timedatectl set-timezone Asia/Shanghai
    #Write the current UTC time to the hardware clock
    timedatectl set-local-rtc 0
    #Restart services that depend on system time
    systemctl restart rsyslog 
    systemctl restart crond
  • Shut down unwanted services

    [root@k8s-master ~]# systemctl stop postfix &&systemctl disable postfix	'//Turn off the mail service and turn it on automatically "
  • Set the log saving method to Journal

    '//Create journal log save directory and configuration file save directory '
    [root@k8s-master ~]# mkdir /var/log/journal
    [root@k8s-master ~]# mkdir /etc/systemd/journald.conf.d
    [root@k8s-master ~]# cat > /etc/systemd/journald.conf.d/99-prophet.conf <<EOF
    > [Journal]
    >  #Persist to disk
    > Storage=persistent
    >  #Compress history log
    > Compress=yes
    > SyncIntervalSec=5m
    > RatelimitInterval=30s
    > RatelimitBurst=1000
    > #Maximum occupancy: 10G
    > SystemMaxUse=10G
    > #Single log file maximum 200M
    > SystemMaxFileSize=200M
    > #Log retention time 2 weeks
    > MaxRetentionSec=2week
    > #Do not forward logs to syslog
    > ForwardToSyslog=no
    > EOF
    [root@k8s-master ~]# systemctl restart systemd-journald	'//Restart journal d '
  • Upgrade the system kernel to 4.4 to improve the running stability of k8s system

    [root@k8s-master ~]# rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm	'//Upgrade RPM package '
    [root@k8s-master ~]# yum --enablerepo=elrepo-kernel install -y kernel-lt	'//Install kernel '
    [root@k8s-master ~]# grub2-set-default "CentOS Linux (4.4.213-1.el7.elrepo.x86_64) 7 (Core)"	'//Set boot from new kernel "
    [root@k8s-master ~]# init 6	'//Restart
    [root@k8s-master ~]# uname -r	'//Check kernel version '

2.2: kubeadm deployment and installation

  • The precondition for Kube proxy to enable ipvs is that both master and slave do

    [root@k8s-master ~]# modprobe br_netfilter	'//Load netfilter module '
        	'//Boot module dependency '
    [root@k8s-master ~]# cat > /etc/sysconfig/modules/ipvs.modules <<EOF
    > #!/bin/bash
    > modprobe -- ip_vs
    > modprobe -- ip_vs_rr
    > modprobe -- ip_vs_wrr
    > modprobe -- ip_vs_sh
    > modprobe -- nf_conntrack_ipv4
    > EOF
    [root@k8s-master ~]# chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules &&lsmod | grep -e ip_vs -e nf_conntrack_ipv4
2.2.1: docker installation, both master and slave
  • Install docker dependency

    [root@k8s-master ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
  • Load Alibaba cloud docker image, YUM warehouse

    [root@k8s-master ~]# yum-config-manager \
    > --add-repo \
    > http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
  • Update and install docker: long time

    [root@k8s-master ~]# yum update -y && yum install -y docker-ce
  • Restart the system after installation

    [root@k8s-master ~]# grub2-set-default "CentOS Linux (4.4.213-1.el7.elrepo.x86_64) 7 (Core)" 
    [root@k8s-master ~]# init 6
    '//If the cursor does not stay on version 4.4, be sure to manually select version 4.4 '
  • Open docker

    [root@k8s-master ~]# systemctl status docker
  • Configure daemon

    [root@k8s-master ~]# cat > /etc/docker/daemon.json <<EOF
    > {
    > "exec-opts": ["native.cgroupdriver=systemd"],
    > "log-driver": "json-file",
    > "log-opts": {
    > "max-size": "100m"
    > }
    > }
    > EOF
    [root@k8s-master ~]# mkdir ‐p /etc/systemd/system/docker.service.d
    [root@k8s-master ~]# systemctl daemon-reload && systemctl restart docker && systemctl enable docker	'//Reload the daemon, restart the docker, and turn on the docker. "
2.2.2: kubeadm installation, both master and slave
  • Import Alibaba cloud K8Syum warehouse

    [root@k8s-master ~]# cat <<EOF > /etc/yum.repos.d/kubernetes.repo
    > [kubernetes]
    > name=Kubernetes
    > baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
    > enabled=1
    > gpgcheck=0
    > repo_ gpgcheck=0
    > gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
    > http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    > EOF
    [root@k8s-master ~]# yum -y install kubeadm-1.15.1 kubect1-1.15.1 kubelet-1.15.1
  • Start up kubelet

    [root@k8s-master ~]# systemctl enable kubelet.service
  • Unzip the kubeadm image

    [root@k8s-master ~]# mount.cifs // /mnt
    Password for root@//  
    [root@k8s-master ~]# cd /mnt/kubernetes/
    [root@k8s-master kubernetes]# mkdir /opt/k8s
    [root@k8s-master kubernetes]# tar zxvf kubeadm-basic.images.tar.gz -C /opt/k8s/
  • Copy the extracted image to the node node

    [root@k8s-master kubernetes]# scp -r /opt/k8s/ root@k8s-node01:/opt/k8s
    [root@k8s-master kubernetes]# scp -r /opt/k8s/ root@k8s-node02:/opt/k8s
  • Write a script to quickly import kubeadm

    [root@k8s-master ~]# vim daoru.sh	'//Edit script '
    ls /opt/k8s/kubeadm-basic.images > /opt/k.txt
    cd /opt/k8s/kubeadm-basic.images
    for i in `cat /opt/k.txt`
            docker load -i $i
    rm -rf /opt/k.txt
    [root@k8s-master ~]# chmod +x daoru.sh 	'//Add script execution permission '
    [root@k8s-master ~]# scp daoru.sh root@k8s-node01:	'//Copy to node '
    [root@k8s-master ~]# scp daoru.sh root@k8s-node02:
    [root@k8s-master ~]# ./daoru.sh 	'//Execute script for each node '
  • Initialize the master node, node node does not need to operate

    [root@k8s-master ~]# cd /opt
    [root@k8s-master opt]# kubeadm config print init-defaults > kubeadm-config.yaml
    [root@k8s-master opt]# vim kubeadm-config.yaml 
    '//Line 12 is changed to the current local IP, with a space after the colon '
    '//The current version of line 34 is v1.15.1 '
    kubernetesVersion: v1.15.1
    '//Add pod node segment below line 37 '
      podSubnet: ""	'//This IP address is the address segment automatically assigned by flannel '
    '//Insert the default scheduling method below line 39 as: ipvs'
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
    featureGates :
       SupportIPVSProxyMode: true
    mode: ipvs
  • Start initializing master

    [root@k8s-master opt]# kubeadm init --config=kubeadm-config.yaml --experimental-upload-certs | tee kubeadm-init.log
  • Check kubeadm-init.log and follow the prompts

    [root@k8s-master opt]# vim kubeadm-init.log 
    '//Lines 56, 57 and 58 should be copied as command input "
    mkdir -p $HOME/.kube
    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    sudo chown $(id -u):$(id -g) $HOME/.kube/config
    [root@k8s-master opt]# mkdir -p $HOME/.kube
    [root@k8s-master opt]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    [root@k8s-master opt]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
    [root@k8s-master opt]# kubectl get node	'//View node status'
    NAME         STATUS     ROLES    AGE     VERSION
    k8s-master   NotReady   master   4m57s   v1.15.1
  • Move important files

    [root@k8s-master opt]# mkdir k-install
    ...Omit the operation. After the file is moved, it is the lower tree view
    [root@k8s-master opt]# tree /opt/k-install/
    /opt/k-install/	'//Create k-install directory '
    ├── code	'//Create code directory under k-install '
    │   ├── kubeadm-config.yaml	'//Move these two files to the code directory '
    │   └── kubeadm-init.log
    └── plugin	'//Create plugin directory under k-install directory '
        └── flannel	'//Create the flannel directory under the plugin directory '
    3 directories, 2 files
  • Download and install the flannel component

    [root@k8s-master opt]# cd k-install/plugin/flannel/
    [root@k8s-master flannel]# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
    [root@k8s-master flannel]# kubectl create -f kube-flannel.yml
    [root@k8s-master flannel]# kubectl get pod -n kube-system
        '//I'll see later that flannel is in running state '
    [root@k8s-master flannel]# kubectl get node
    k8s-master   Ready    master   162m   v1.15.1	'//Found k8s in ready state '
  • Node node operation, adding to master node

    Master node kubeadm-init.log These two sentences are copied to node Perform automatic add to primary node in node
    [root@k8s-master flannel]# cd ../../code/
    [root@k8s-master code]# ls
    kubeadm-config.yaml  kubeadm-init.log
    [root@k8s-master code]# vim kubeadm-init.log 
    kubeadm join --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:fb07c549867486b9dc43dd6f00267f335ff250978476e9d177b43ce1e450c375
    '//Copy these two sentences to the node node and execute directly "
    [root@k8s-master code]# kubectl get pod -n kube-system -o wide	'//View more details'
    [root@k8s-master code]# kubectl get pod -n kube-system -w	'//Monitoring status'
        '//Other omissions, these two are initializing. We will monitor them until they are running '
    kube-flannel-ds-amd64-cvg8v          0/1     Init:0/1   0          2m35s
    kube-flannel-ds-amd64-k69bq          0/1     Init:0/1   0          2m38s
    [root@k8s-master code]# kubectl get node
    k8s-master   Ready    master   162m   v1.15.1
    k8s-node01   Ready    node     162m   v1.15.1
    k8s-node02   Ready    node     162m   v1.15.1
  • Successful cluster construction

2.3: the experiment is over and the cluster is built successfully

[root@k8s-master code]# kubectl get node
k8s-master   Ready    master   162m   v1.15.1
k8s-node01   Ready    node     162m   v1.15.1
k8s-node02   Ready    node     162m   v1.15.1
120 original articles published, 61 praised, 10000 visitors+
Private letter follow

Tags: Kubernetes Docker yum iptables

Posted on Fri, 14 Feb 2020 05:10:35 -0800 by stewart