使用Helm安装Cilium 1.13

2023-07-11

Cilium是一个基于eBPF的数据平面的网络、观测和安全解决方案。它提供了一个简单的扁平化第三层网络，能够以本地路由或覆盖模式跨多个集群进行部署。 Cilium的基础是名为eBPF的Linux内核技术，它支持在Linux内核的各个集成点（如网络IO、应用程序套接字和跟踪点tracepoints）动态插入eBPF字节码，以实现安全、网络和可见性逻辑。eBPF具有高效和灵活的特性。本文将介绍如何使用Helm安装Cilium。与Cilium快速安装相比，这需要进行一些额外的步骤，并要求你手动选择最适合你特定环境的数据路径(datapath)和IPAM模式(IPAM mode)。

1.系统要求和环境信息 #

安装Cilium的系统要求:

Kubernetes必须配置为使用CNI（请参阅网络插件要求）
Linux内核 >= 4.9.17
有些Linux的发行版需要配置为不管理外部路由，例如，在Ubuntu 22.04中就是这种情况(Cilium GitHu的ISSUE 18706中有讨论)。对于Ubuntu 22.04在/etc/systemd/networkd.conf中配置如下:
```
1[Network]
2ManageForeignRoutes=no
3ManageForeignRoutingPolicyRules=no
```
配置修改后，执行systemctl restart systemd-networkd使配置生效。

更多信息可查看System Requirements以获取有关系统要求的更多详细信息。

本文使用的环境信息:

1kubectl get node
2NAME    STATUS   ROLES                AGE   VERSION
3node1   Ready    control-plane,edge   36d   v1.27.2
4node2   Ready    <none>               36d   v1.27.2
5node3   Ready    <none>               36d   v1.27.2

Kubernetes的版本为1.27，在安装Cilium前使用的CNI网络插件为Calico。

node1~node3的操作系统为Ubuntu 22.04, 内核版本为5.15.0-73-generic。

2.安装Cilium #

设置Helm Repository:

1helm repo add cilium https://helm.cilium.io/

使用Helm安装时的默认配置:

Datapath	IPAM	Datastore
Encapsulation	Cluster Pool	Kubernetes CRD

使用Helm安装cilium的命令如下:

1helm install cilium cilium/cilium --version 1.13.4 \
2  --namespace kube-system

查看Pod状态:

1kubectl get po -n kube-system -o wide | grep cilium
2cilium-9kfsz                            1/1     Running   0              3m25s   192.168.96.153   node3   <none>           <none>
3cilium-crvwn                            1/1     Running   0              3m25s   192.168.96.152   node2   <none>           <none>
4cilium-operator-d5f57588-pb7st          1/1     Running   0              3m25s   192.168.96.153   node3   <none>           <none>
5cilium-operator-d5f57588-sk9lr          1/1     Running   0              3m25s   192.168.96.152   node2   <none>           <none>
6cilium-z8tl2                            1/1     Running   0              3m25s   192.168.96.151   node1   <none>           <none>

3.验证安装 #

3.1 使用Cilium CLI查看状态 #

下面将安装最新版本的Cilium CLI，可以使用它检查Cilium安装的状态，并启用/禁用各种功能（例如clustermesh、Hubble）。

1CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/master/stable.txt)
2CLI_ARCH=amd64
3if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
4curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
5sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
6sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
7rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

使用cilium status查看当前Cilium的状态:

 1cilium status --wait
 2    /¯¯\
 3 /¯¯\__/¯¯\    Cilium:             OK
 4 \__/¯¯\__/    Operator:           OK
 5 /¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
 6 \__/¯¯\__/    Hubble Relay:       disabled
 7    \__/       ClusterMesh:        disabled
 8
 9Deployment             cilium-operator    Desired: 2, Ready: 2/2, Available: 2/2
10DaemonSet              cilium             Desired: 3, Ready: 3/3, Available: 3/3
11Containers:            cilium             Running: 3
12                       cilium-operator    Running: 2
13Cluster Pods:          2/5 managed by Cilium
14Helm chart version:    1.13.4
15Image versions         cilium             quay.io/cilium/cilium:v1.13.4@sha256:bde8800d61aaad8b8451b10e247ac7bdeb7af187bb698f83d40ad75a38c1ee6b: 3
16                       cilium-operator    quay.io/cilium/operator-generic:v1.13.4@sha256:09ab77d324ef4d31f7d341f97ec5a2a4860910076046d57a2d61494d426c6301: 2

从cilium status的输出中可以看出，默认安装了cilium-operator这个Deployment，和cilium这个DaemonSet，它们对应的Pod全部正常运行。Cluster Pods: 2/5 managed by Cilium说明当前集群中的Pod有5个，其中2个Pod的网络被Cilium管理，另外3个还在被Calico管理。

这2个被Cilium接管网络的Pod是CoreDNS，在安装完cilium后会自动将cordns的Pod切换为cilium网络:

1get po -o wide -A | grep -v 192
2NAMESPACE       NAME                                        READY   STATUS    RESTARTS       AGE   IP               NODE    NOMINATED NODE   READINESS GATES
3calico-system   calico-kube-controllers-b474bd8d5-cn2f5     1/1     Running   0              20m   10.244.135.7     node3   <none>           <none>
4kube-system     coredns-d888b99fd-jp4bz                     1/1     Running   0              11m   10.0.0.21        node2   <none>           <none>
5kube-system     coredns-d888b99fd-vzwvs                     1/1     Running   0              11m   10.0.2.225       node3   <none>           <none>
6kube-system     kubernetes-dashboard-787d9d478d-8bgxq       2/2     Running   0              21m   10.244.135.8     node3   <none>           <none>
7kube-system     metrics-server-767457f446-xshkf             1/1     Running   0              21m   10.244.135.4     node3   <none>           <none>

从上面的命令输出可以看出，coredns的Pod的IP已经切换为Cilium网络Pod默认的CIDR 10.0.0.0/8。其他的Pod还是Calico网络。

3.2 重启未被Cilium管理的Pod #

如果没有使用带有taintnode.cilium.io/agent-not-ready的Node创建集群，那么未管理的Pod需要手动重新启动。重新启动所有已经在非主机网络模式(host-networking mode)下运行的Pod，以确保Cilium开始管理它们。

1kubectl get pods --all-namespaces -o custom-columns=NAMESPACE:.metadata.namespace,NAME:.metadata.name,HOSTNETWORK:.spec.hostNetwork --no-headers=true | grep '<none>' | awk '{print "-n "$1" "$2}' | xargs -L 1 -r kubectl delete pod
2
3pod "calico-kube-controllers-b474bd8d5-cn2f5" deleted
4pod "coredns-d888b99fd-jp4bz" deleted
5pod "coredns-d888b99fd-vzwvs" deleted
6pod "kubernetes-dashboard-787d9d478d-8bgxq" deleted
7pod "metrics-server-767457f446-xshkf" deleted

Pod重启后，确认其PodIP都切换成到10.0.0.0/8

1kubectl get po -o wide -A | grep -v 192
2calico-system   calico-kube-controllers-b474bd8d5-7hwrb     1/1     Running   0              13m   10.0.0.70        node2   <none>           <none>
3kube-system     coredns-d888b99fd-9gz2l                     1/1     Running   0              13m   10.0.0.38        node2   <none>           <none>
4kube-system     coredns-d888b99fd-lhw9d                     1/1     Running   0              13m   10.0.2.81        node3   <none>           <none>
5kube-system     kubernetes-dashboard-787d9d478d-dnknx       2/2     Running   0              13m   10.0.0.183       node2   <none>           <none>
6kube-system     metrics-server-767457f446-b9hn5             1/1     Running   0              13m   10.0.2.37        node3   <none>           <none>

再次使用cilium status查看当前Cilium的状态，确认Cluster Pods:显示所有的Pod都被Cilium管理:

 1cilium status
 2    /¯¯\
 3 /¯¯\__/¯¯\    Cilium:             OK
 4 \__/¯¯\__/    Operator:           OK
 5 /¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
 6 \__/¯¯\__/    Hubble Relay:       disabled
 7    \__/       ClusterMesh:        disabled
 8
 9Deployment             cilium-operator    Desired: 2, Ready: 2/2, Available: 2/2
10DaemonSet              cilium             Desired: 3, Ready: 3/3, Available: 3/3
11Containers:            cilium             Running: 3
12                       cilium-operator    Running: 2
13Cluster Pods:          5/5 managed by Cilium
14Helm chart version:    1.13.4
15Image versions         cilium             quay.io/cilium/cilium:v1.13.4@sha256:bde8800d61aaad8b8451b10e247ac7bdeb7af187bb698f83d40ad75a38c1ee6b: 3
16                       cilium-operator    quay.io/cilium/operator-generic:v1.13.4@sha256:09ab77d324ef4d31f7d341f97ec5a2a4860910076046d57a2d61494d426c6301: 2