使用kubeadm安装Kubernetes 1.6

2017-04-05 阅读: Kubernetes

环境准备

192.168.61.41 node1
192.168.61.42 node2
192.168.61.43 node3

安装Docker 1.12

Kubernetes 1.6还没有针对docker 1.13和最新的docker 17.03上做测试和验证,所以这里安装Kubernetes官方推荐的Docker 1.12版本。

yum install -y yum-utils

yum-config-manager \
    --add-repo \
    https://docs.docker.com/v1.13/engine/installation/linux/repo_files/centos/docker.repo
    
yum makecache fast

查看版本:

yum list docker-engine.x86_64  --showduplicates |sort -r
docker-engine.x86_64             1.13.1-1.el7.centos                 docker-main
docker-engine.x86_64             1.12.6-1.el7.centos                 docker-main
docker-engine.x86_64             1.11.2-1.el7.centos                 docker-main

安装1.12.6:

yum install -y docker-engine-1.12.6

systemctl start docker
systemctl enable docker

系统配置

根据官方文档Installing Kubernetes on Linux with kubeadm 中的Limitations小节中的内容,对各节点系统做如下设置:

创建/etc/sysctl.d/k8s.conf文件,添加如下内容:

net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

执行sysctl -p /etc/sysctl.d/k8s.conf使修改生效。

在/etc/hostname中修改各节点的hostname,在/etc/hosts中设置hostname对应非lo回环网卡ip:

192.168.61.41 node1
192.168.61.42 node2
192.168.61.43 node3

安装kubeadm和kubelet

下面在各节点安装kubeadm和kubelet:

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

测试地址http://yum.kubernetes.io/repos/kubernetes-el7-x86_64是否可用,如果不可用需要科学上网。

 curl http://yum.kubernetes.io/repos/kubernetes-el7-x86_64

查看kubeadm, kubelet, kubectl, kubernets-cni的最新版本:

yum list kubeadm  --showduplicates |sort -r
kubeadm.x86_64                        1.6.1-0                        kubernetes
kubeadm.x86_64                        1.6.0-0                        kubernetes

yum list kubelet  --showduplicates |sort -r
kubelet.x86_64                        1.6.1-0                        kubernetes
kubelet.x86_64                        1.6.0-0                         kubernetes
kubelet.x86_64                        1.5.4-0                         kubernetes


yum list kubectl  --showduplicates |sort -r
kubectl.x86_64                        1.6.1-0                        kubernetes
kubectl.x86_64                        1.6.0-0                         kubernetes
kubectl.x86_64                        1.5.4-0                         kubernetes


yum list kubernets-cni  --showduplicates |sort -r
kubernetes-cni              x86_64              0.5.1-0                    kubernetes

kubeadm和kubelet已经是1.6.1,就是我们要安装的版本,直接安装即可:

setenforce 0

yum install -y kubelet kubeadm kubectl kubernetes-cni
...
Installed:
  kubeadm.x86_64 0:1.6.1-0               kubectl.x86_64 0:1.6.1-0        kubelet.x86_64 0:1.6.1-0
  kubernetes-cni.x86_64 0:0.5.1-0

Dependency Installed:
  ebtables.x86_64 0:2.0.10-15.el7                       socat.x86_64 0:1.7.2.2-5.el7

Complete!
systemctl enable kubelet.service

初始化集群

接下来使用kubeadm初始化集群,选择node1作为Master Node,在node1上执行下面的命令:

kubeadm init --kubernetes-version=v1.6.1 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.61.41

注意到kubeadm init的--kubernetes-version--apiserver-advertise-address发生了变化,之前kuebadm 1.6.0-0.alpha是--use-kubernetes-version--api-advertise-addresses

因为我们选择flannel作为Pod网络插件,所以上面的命令指定--pod-network-cidr=10.244.0.0/16。 在集群初始化遇到问题,可以使用下面的命令进行清理后重新再初始化:

kubeadm reset
ifconfig cni0 down
ip link delete cni0
ifconfig flannel.1 down
ip link delete flannel.1
rm -rf /var/lib/cni/

kubeadm init执行成功后输出下面的信息:

kubeadm init --kubernetes-version=v1.6.1 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.61.41
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[init] Using Kubernetes version: v1.6.1
[init] Using Authorization mode: RBAC
[preflight] Running pre-flight checks
[preflight] Starting the kubelet service
[certificates] Generated CA certificate and key.
[certificates] Generated API server certificate and key.
[certificates] API Server serving cert is signed for DNS names [node0 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.61.41]
[certificates] Generated API server kubelet client certificate and key.
[certificates] Generated service account token signing key and public key.
[certificates] Generated front-proxy CA certificate and key.
[certificates] Generated front-proxy client certificate and key.
[certificates] Valid certificates and keys now exist in "/etc/kubernetes/pki"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/admin.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/controller-manager.conf"
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/scheduler.conf"
[apiclient] Created API client, waiting for the control plane to become ready
[apiclient] All control plane components are healthy after 14.583864 seconds
[apiclient] Waiting for at least one node to register
[apiclient] First node has registered after 6.008990 seconds
[token] Using token: e7986d.e440de5882342711
[apiconfig] Created RBAC rules
[addons] Created essential addon: kube-proxy
[addons] Created essential addon: kube-dns

Your Kubernetes master has initialized successfully!

To start using your cluster, you need to run (as a regular user):

  sudo cp /etc/kubernetes/admin.conf $HOME/
  sudo chown $(id -u):$(id -g) $HOME/admin.conf
  export KUBECONFIG=$HOME/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  http://kubernetes.io/docs/admin/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join --token e7986d.e440de5882342711 192.168.61.41:6443

Master Node初始化完成,使用kubeadm初始化的Kubernetes集群在Master节点上的核心组件:kube-apiserver, kube-scheduler, kube-controller-manager是以静态Pod的形式运行的。

ls /etc/kubernetes/manifests/
etcd.yaml  kube-apiserver.yaml  kube-controller-manager.yaml  kube-scheduler.yaml

在/etc/kubernetes/manifests/目录里可以看到kube-apiserver, kube-scheduler, kube-controller-manager的定义文件。另外集群持久化存储etcd也是以单点静态Pod的形式运行的,对于etcd后边我们会把它切换成etcd集群,这里暂且不表。

查看一下kube-apiserver.yaml的内容:

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    component: kube-apiserver
    tier: control-plane
  name: kube-apiserver
  namespace: kube-system
spec:
  containers:
  - command:
    - kube-apiserver
    .......
    - --insecure-port=0

注意到kube-apiserver的选项--insecure-port=0,也就是说kubeadm 1.6.0初始化的集群,kube-apiserver没有监听默认的http 8080端口。 所以我们使用kubectl get nodes会报The connection to the server localhost:8080 was refused - did you specify the right host or port?

查看kube-apiserver的监听端口可以看到只监听了https的6443端口,

netstat -nltp | grep apiserver
tcp6       0      0 :::6443                 :::*                    LISTEN      9831/kube-apiserver

为了使用kubectl访问apiserver,在~/.bash_profile中追加下面的环境变量:

export KUBECONFIG=/etc/kubernetes/admin.conf
source ~/.bash_profile

此时kubectl命令在master node上就好用了,查看一下当前机器中的Node:

kubectl get nodes
NAME      STATUS     AGE       VERSION
node0     NotReady   3m        v1.6.1

安装Pod Network

接下来安装flannel network add-on:

kubectl create -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel-rbac.yml
kubectl apply -f  https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
serviceaccount "flannel" created
configmap "kube-flannel-cfg" created
daemonset "kube-flannel-ds" created

如果Node有多个网卡的话,参考flannel issues 39701,目前需要在kube-flannel.yml中使用--iface参数指定集群主机内网网卡的名称,否则可能会出现dns无法解析。需要将kube-flannel.yml下载到本地,flanneld启动参数加上--iface=<iface-name>

......
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
......
containers:
      - name: kube-flannel
        image: quay.io/coreos/flannel:v0.7.0-amd64
        command: [ "/opt/bin/flanneld", "--ip-masq", "--kube-subnet-mgr", "--iface=eth1" ]
......

使用kubectl get pod --all-namespaces -o wide确保所有的Pod都处于Running状态。

kubectl get pod --all-namespaces -o wide

使master node参与工作负载

使用kubeadm初始化的集群,出于安全考虑Pod不会被调度到Master Node上,也就是说Master Node不参与工作负载。

这里搭建的是测试环境可以使用下面的命令使Master Node参与工作负载:

kubectl taint nodes --all  node-role.kubernetes.io/master-

测试DNS

kubectl run curl --image=radial/busyboxplus:curl -i --tty
Waiting for pod default/curl-2421989462-vldmp to be running, status is Pending, pod ready: false
Waiting for pod default/curl-2421989462-vldmp to be running, status is Pending, pod ready: false
If you don't see a command prompt, try pressing enter.
[ root@curl-2421989462-vldmp:/ ]$

进入后执行nslookup kubernetes.default确认解析正常。

[ root@curl-2421989462-vldmp:/ ]$ nslookup kubernetes.default
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes.default
Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

测试OK后,删除掉curl这个Pod。

kubectl delete deploy curl

向集群中添加节点

下面将node2和node3加入集群,分别在node2和node3上执行:

kubeadm join --token e7986d.e440de5882342711 192.168.61.41:6443
[kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters.
[preflight] Running pre-flight checks
[discovery] Trying to connect to API Server "192.168.61.41:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://192.168.61.41:6443"
[discovery] Cluster info signature and contents are valid, will use API Server "https://192.168.61.41:6443"
[discovery] Successfully established connection with API Server "192.168.61.41:6443"
[bootstrap] Detected server version: v1.6.1
[bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)
[csr] Created API client to obtain unique certificate for this node, generating keys and certificate signing request
[csr] Received signed certificate from the API server, generating KubeConfig...
[kubeconfig] Wrote KubeConfig file to disk: "/etc/kubernetes/kubelet.conf"

Node join complete:
* Certificate signing request sent to master and response
  received.
* Kubelet informed of new secure connection details.

Run 'kubectl get nodes' on the master to see this machine join.

查看集群中节点:

kubectl get nodes
NAME      STATUS    AGE       VERSION
node1     Ready     12m       v1.6.1
node2     Ready     4m        v1.6.1
node3     Ready     2m        v1.6.1

安装Dashboard插件

wget https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml

kubectl create -f kubernetes-dashboard.yaml

从http://NodeIp:NodePort访问dashboard,浏览器显示下面的错误:

User "system:serviceaccount:kube-system:default" cannot list statefulsets.apps in the namespace "default". (get statefulsets.apps)

这是因为Kubernetes 1.6开始API Server启用了RBAC授权,当前的kubernetes-dashboard.yaml没有定义授权的ServiceAccount,所以访问API Server时被拒绝了。

根据https://github.com/kubernetes/dashboard/issues/1803中的内容临时授予system:serviceaccount:kube-system:default cluster_admin的角色,临时解决一下。

创建dashboard-rbac.yaml,定义system:serviceaccount:kube-system:default和ClusterRole cluster-admin绑定:

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: dashboard-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin 
subjects:
- kind: ServiceAccount
  name: default
  namespace: kube-system
 kubectl create -f dashboard-rbac.yml

在集群中运行Heapster

下面安装Heapster为集群添加使用统计和监控功能,为Dashboard添加仪表盘。

下载最新的Heapster到集群中的某个Node上。

wget https://github.com/kubernetes/heapster/archive/v1.3.0.tar.gz

使用InfluxDB做为Heapster的后端存储,开始部署,中间会pull相关镜像,包含gcr.io/google_containers/heapster_grafana:v2.6.0-2

tar -zxvf v1.3.0.tar.gz
cd heapster-1.3.0/deploy/kube-config/influxdb

kubectl create -f ./
deployment "monitoring-grafana" created
service "monitoring-grafana" created
deployment "heapster" created
service "heapster" created
deployment "monitoring-influxdb" created
service "monitoring-influxdb" created

最后确认所有的pod都处于running状态,打开Dashboard,集群的使用统计会以仪表盘的形式显示出来。

参考

标题:使用kubeadm安装Kubernetes 1.6
本文链接:https://blog.frognew.com/2017/04/kubeadm-install-kubernetes-1.6.html
转载请注明出处。

目录