以下步骤运行在 oracle cloud ARM Ampere A1 实例上，实例配置为：4C24G，操作系统为：ubuntu 22.04 LTS
一、安装 k3s
使用以下命令启动 k3s 集群，不安装 cni， 同时设置 cluster cidr 和 service cidr 防止和本机地址冲突( –cluster-cidr=‘10.42.0.0/16’ –service-cidr=‘10.253.0.0/16’ )
1
2
3
4
5
6
|
curl -sfL -sfL https://get.k3s.io | \
INSTALL_K3S_EXEC="--flannel-backend=none \
--disable-network-policy \
--disable-kube-proxy \
--prefer-bundled-bin \
--disable=traefik,servicelb,local-storage" sh -s -
|
二、安装 Cilium
1. 操作系统准备
禁用外部路由管理
一些发行版需要配置不管理外部路由，例如 Ubuntu 22.04，详细信息请查看GitHub issue 18706，编辑 /etc/systemd/networkd.conf 添加以下设置：
1
2
3
|
[Network]
ManageForeignRoutes=no
ManageForeignRoutingPolicyRules=no
|
重启systemd-networkd服务
1
2
|
systemctl daemon-reload
systemctl restart systemd-networkd
|
2. 安装 Cilium CLI
1
2
3
4
5
6
7
8
9
10
|
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
#查看当前安装版本
cilium version --client
|
3. 安装 Cilium
同时安装 Hubble & Hubble UI
为确保 Cilium 不干扰 Istio，必须使用 –config bpf-lb-sock-hostns-only=true cilium CLI 标志或 socketLB.hostNamespaceOnly=true 来部署 Cilium。同时 ipam.operator.clusterPoolIPv4PodCIDRList 指定 k3s 默认的 pod CIDR。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
API_SERVER_IP='10.0.0.214' # 服务器 ip 地址
API_SERVER_PORT=6443
cilium install --set hubble.relay.enabled=true \
--set hubble.ui.enabled=true \
--set operator.replicas=1 \
--set kubeProxyReplacement=strict \
--set ipam.operator.clusterPoolIPv4PodCIDRList="10.42.0.0/16" \
--set socketLB.hostNamespaceOnly=true \
--set k8sServiceHost=${API_SERVER_IP} \
--set k8sServicePort=${API_SERVER_PORT} \
--set tunnel=disabled \
--set autoDirectNodeRoutes=true \
--set ipv4NativeRoutingCIDR="10.42.0.0/16" \
--set bpf.masquerade=true --version 1.14.1
|
NATIVE ROUTING：
1
2
3
|
--set tunnel=disabled\
--set autoDirectNodeRoutes=true\
--set ipv4NativeRoutingCIDR="10.42.0.0/16"
|
IP 地址伪装(Masquerading):
1
|
--set bpf.masquerade=true
|
4. 验证 Cilium 安装
运行以下命令验证 cilium 时候正确安装
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
# cilium status --wait
/¯¯\
/¯¯\__/¯¯\ Cilium: OK
\__/¯¯\__/ Operator: OK
/¯¯\__/¯¯\ Envoy DaemonSet: disabled (using embedded mode)
\__/¯¯\__/ Hubble Relay: OK
\__/ ClusterMesh: disabled
DaemonSet cilium Desired: 1, Ready: 1/1, Available: 1/1
Deployment hubble-relay Desired: 1, Ready: 1/1, Available: 1/1
Deployment hubble-ui Desired: 1, Ready: 1/1, Available: 1/1
Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1
Containers: cilium-operator Running: 1
cilium Running: 1
hubble-relay Running: 1
hubble-ui Running: 1
Cluster Pods: 6/6 managed by Cilium
Helm chart version: 1.14.0
Image versions cilium quay.io/cilium/cilium:v1.14.0@sha256:5a94b561f4651fcfd85970a50bc78b201cfbd6e2ab1a03848eab25a82832653a: 1
hubble-relay quay.io/cilium/hubble-relay:v1.14.0@sha256:bfe6ef86a1c0f1c3e8b105735aa31db64bcea97dd4732db6d0448c55a3c8e70c: 1
hubble-ui quay.io/cilium/hubble-ui:v0.12.0@sha256:1c876cfa1d5e35bc91e1025c9314f922041592a88b03313c22c1f97a5d2ba88f: 1
hubble-ui quay.io/cilium/hubble-ui-backend:v0.12.0@sha256:8a79a1aad4fc9c2aa2b3e4379af0af872a89fcec9d99e117188190671c66fc2e: 1
cilium-operator quay.io/cilium/operator-generic:v1.14.0@sha256:3014d4bcb8352f0ddef90fa3b5eb1bbf179b91024813a90a0066eb4517ba93c9: 1
|
运行以下命令验证集群网络的连通性
1
2
3
4
5
6
|
$ cilium connectivity test
ℹ️ Single-node environment detected, enabling single-node connectivity test
ℹ️ Monitor aggregation detected, will skip some flow validation steps
........
✅ All 42 tests (184 actions) successful, 13 tests skipped, 0 scenarios skipped.
|
三、安装 Istio
1. 下载 Istio
使用以下脚本地址下载 Istio
1
|
curl -L https://istio.io/downloadIstio | sh -
|
转到 Istio 包目录。例如，如果包是istio-1.18.2：
安装目录包含：
将istioctl客户端添加到路径：
1
|
cp bin/istioctl /usr/local/bin/
|
2. 使用预设文件部署 istio
采用demo配置组合, 选择它是因为它包含了一组专为测试准备的功能集合，另外还有用于生产或性能测试的配置组合。
1
2
3
4
5
6
7
8
|
# export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
# istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete
Making this installation the default for injection and validation
|
命名空间添加标签，指示 Istio 在部署应用的时候，自动注入 Envoy 边车代理
1
2
|
$ kubectl label namespace default istio-injection=enabled
namespace/default labeled
|
四、卸载
当 k3s 环境出现问题，或者需要重建，需要使用以下方法：
1
2
3
4
5
6
7
8
9
10
|
ip link delete cilium_host
ip link delete cilium_net
ip link delete cilium_vxlan
iptables-save | grep -iv cilium | iptables-restore
ip6tables-save | grep -iv cilium | ip6tables-restore
/usr/local/bin/k3s-killall.sh
/usr/local/bin/k3s-uninstall.sh
|
至此完成 cilium + Istio 实验环境搭建，后续将进行 cilium 和 Istio 相关的实验。
