以下步骤运行在 oracle cloud ARM Ampere A1 实例上，实例配置为：4C24G，操作系统为：ubuntu 22.04 LTS

一、安装 k3s

使用以下命令启动 k3s 集群，不安装 cni， ~~同时设置 cluster cidr 和 service cidr 防止和本机地址冲突(~~ –cluster-cidr=‘10.42.0.0/16’ –service-cidr=‘10.253.0.0/16’ )

1
2
3
4
5
6


curl -sfL -sfL https://get.k3s.io | \
INSTALL_K3S_EXEC="--flannel-backend=none \
                 --disable-network-policy \
                 --disable-kube-proxy \
                 --prefer-bundled-bin \
                 --disable=traefik,servicelb,local-storage" sh -s -

二、安装 Cilium

1. 操作系统准备

禁用外部路由管理

一些发行版需要配置不管理外部路由，例如 Ubuntu 22.04，详细信息请查看GitHub issue 18706，编辑 /etc/systemd/networkd.conf 添加以下设置：

1
2
3


[Network]
ManageForeignRoutes=no
ManageForeignRoutingPolicyRules=no

重启systemd-networkd服务

1
2


systemctl daemon-reload
systemctl restart systemd-networkd

2. 安装 Cilium CLI

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
CLI_ARCH=amd64
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}

#查看当前安装版本
cilium version --client

3. 安装 Cilium

同时安装 Hubble & Hubble UI

为确保 Cilium 不干扰 Istio，必须使用 –config bpf-lb-sock-hostns-only=true cilium CLI 标志或 socketLB.hostNamespaceOnly=true 来部署 Cilium。同时 ipam.operator.clusterPoolIPv4PodCIDRList 指定 k3s 默认的 pod CIDR。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
API_SERVER_IP='10.0.0.214'  # 服务器 ip 地址
API_SERVER_PORT=6443
cilium install --set hubble.relay.enabled=true \
              --set hubble.ui.enabled=true \
              --set operator.replicas=1 \
              --set kubeProxyReplacement=strict \
              --set ipam.operator.clusterPoolIPv4PodCIDRList="10.42.0.0/16" \
              --set socketLB.hostNamespaceOnly=true \
              --set k8sServiceHost=${API_SERVER_IP} \
              --set k8sServicePort=${API_SERVER_PORT} \
              --set tunnel=disabled \
              --set autoDirectNodeRoutes=true \
              --set ipv4NativeRoutingCIDR="10.42.0.0/16" \
              --set bpf.masquerade=true --version 1.14.1

NATIVE ROUTING：

1
2
3


--set tunnel=disabled\
--set autoDirectNodeRoutes=true\
--set ipv4NativeRoutingCIDR="10.42.0.0/16"

IP 地址伪装(Masquerading):

1

--set bpf.masquerade=true

4. 验证 Cilium 安装

运行以下命令验证 cilium 时候正确安装

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23


# cilium status --wait
   /¯¯\
/¯¯\__/¯¯\    Cilium:             OK
\__/¯¯\__/    Operator:           OK
/¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
\__/¯¯\__/    Hubble Relay:       OK
   \__/       ClusterMesh:        disabled

DaemonSet              cilium             Desired: 1, Ready: 1/1, Available: 1/1
Deployment             hubble-relay       Desired: 1, Ready: 1/1, Available: 1/1
Deployment             hubble-ui          Desired: 1, Ready: 1/1, Available: 1/1
Deployment             cilium-operator    Desired: 1, Ready: 1/1, Available: 1/1
Containers:            cilium-operator    Running: 1
                      cilium             Running: 1
                      hubble-relay       Running: 1
                      hubble-ui          Running: 1
Cluster Pods:          6/6 managed by Cilium
Helm chart version:    1.14.0
Image versions         cilium             quay.io/cilium/cilium:v1.14.0@sha256:5a94b561f4651fcfd85970a50bc78b201cfbd6e2ab1a03848eab25a82832653a: 1
                      hubble-relay       quay.io/cilium/hubble-relay:v1.14.0@sha256:bfe6ef86a1c0f1c3e8b105735aa31db64bcea97dd4732db6d0448c55a3c8e70c: 1
                      hubble-ui          quay.io/cilium/hubble-ui:v0.12.0@sha256:1c876cfa1d5e35bc91e1025c9314f922041592a88b03313c22c1f97a5d2ba88f: 1
                      hubble-ui          quay.io/cilium/hubble-ui-backend:v0.12.0@sha256:8a79a1aad4fc9c2aa2b3e4379af0af872a89fcec9d99e117188190671c66fc2e: 1
                      cilium-operator    quay.io/cilium/operator-generic:v1.14.0@sha256:3014d4bcb8352f0ddef90fa3b5eb1bbf179b91024813a90a0066eb4517ba93c9: 1

运行以下命令验证集群网络的连通性

1
2
3
4
5
6


$ cilium connectivity test
ℹ️  Single-node environment detected, enabling single-node connectivity test
ℹ️  Monitor aggregation detected, will skip some flow validation steps
........

✅ All 42 tests (184 actions) successful, 13 tests skipped, 0 scenarios skipped.

三、安装 Istio

1. 下载 Istio

使用以下脚本地址下载 Istio

1

curl -L https://istio.io/downloadIstio | sh -

转到 Istio 包目录。例如，如果包是istio-1.18.2：

1

$ cd istio-1.18.2

安装目录包含：

samples/目录下的示例应用程序
bin/目录下的[istioctl](https://istio.io/latest/zh/docs/reference/commands/istioctl)客户端二进制文件。

将istioctl客户端添加到路径：

1

cp bin/istioctl /usr/local/bin/

2. 使用预设文件部署 istio

采用demo配置组合, 选择它是因为它包含了一组专为测试准备的功能集合，另外还有用于生产或性能测试的配置组合。

1
2
3
4
5
6
7
8


# export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
# istioctl install --set profile=demo -y
✔ Istio core installed
✔ Istiod installed
✔ Egress gateways installed
✔ Ingress gateways installed
✔ Installation complete
Making this installation the default for injection and validation

命名空间添加标签，指示 Istio 在部署应用的时候，自动注入 Envoy 边车代理

1
2


$ kubectl label namespace default istio-injection=enabled
namespace/default labeled

四、卸载

当 k3s 环境出现问题，或者需要重建，需要使用以下方法：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


ip link delete cilium_host
ip link delete cilium_net
ip link delete cilium_vxlan

iptables-save | grep -iv cilium | iptables-restore
ip6tables-save | grep -iv cilium | ip6tables-restore


/usr/local/bin/k3s-killall.sh
/usr/local/bin/k3s-uninstall.sh