关于 SR-IOV 本文就不再介绍了,具体可以查看 Intel® 82599 SR-IOV Driver Companion Guide。
一、系统环境
操作系统: RHEL 7.2
OpenStack版本: OpenStack Mitaka Allinone
网卡型号: Intel Corporation 82599ES
SR-IVO网卡名: ens1f0, ens1f0
二、服务器配置
在服务器 BIOS 中开启 VT-d 和 SR-IOV
三、操作系统配置
1. 编辑 /etc/default/grub 文件,加入以下内容
# vim /etc/default/grub |
需要说明的是: ixgbe.max_vfs 参数已经废弃,故没有加入到内核参数中。
2. 重新生成 grub 文件
# grub2-mkconfig -o /boot/grub2/grub.cfg |
3. 配置 SR-IOV 的网卡开机自启
编辑网卡配置文件,修改以下内容
BOOTPROTO=none |
4. 设置开机自动创建 VF(计算节点)
# vim /etc/rc.d/rc.local |
5. 重启服务器
6. 验证 VF 已经创建,并且是 up 状态
# lspci | grep Ethernet |
四、OpenStack 配置
安装 sr-iov agent(计算节点)
控制节点配置
1. 修改 nova 调度,启用 PciPassthrough Filter
# vim /etc/nova/nova.conf |
2. 在 ML2 中加载 sriovnicswitch mechanism driver,并设置网络绑定
# vim /etc/neutron/plugins/ml2/ml2_conf.ini |
3. 增加支持的 PCI 厂家的 VF 设备
- 查看id
# lspci -nn | grep -i ethernet
02:00.0 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
02:00.1 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
02:00.2 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
02:00.3 Ethernet controller [0200]: Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)
05:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
05:00.1 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 01)
05:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.3 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.4 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.5 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:10.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.1 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.3 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.4 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.5 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
05:11.7 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01) - 配置设备ID
# vim /etc/neutron/plugins/ml2/ml2_conf_sriov.ini
[ml2_sriov]
supported_pci_vendor_devs = 8086:10ed
4. 修改 neutron-server 启动文件,加载 ml2_conf_sriov.ini 文件
# vim /usr/lib/systemd/system/neutron-server.service |
5. 重启服务
计算节点配置
1. 配置 PCI 设备白名单
# vim /etc/nova/nova.conf |
2. 配置 SR-IOV neutron agent
# vim /etc/neutron/plugins/ml2/sriov_agent.ini |
3. 启动/重启服务
五、测试
1. 创建网络
创建网络 |
2. 创建 port
# neutron port-create net1 --binding:vnic-type direct |
3. 创建虚拟机
# neutron port-list |
4. 查看虚拟机网卡
- 速率
- 型号
5. 查看 VF
# ip link show ens1f0 |
参考文章
SR-IOV-Passthrough-For-Networking
OpenStack Networking Guide
OpenStack Mitaka Neutron SR-IOV配置