openstack 使用 nova docker driver

一、安装docker并修改使用阿里云的镜像加速

以下操作在controller节点和compute节点进行(controller节点安装docker是为了方便下载docker镜像直接导入glance)

  1. 创建yum repo文件(这里使用阿里云的源)
1
2
3
4
5
6
7
8
# tee /etc/yum.repos.d/docker.repo <<-'EOF'
[dockerrepo]
name=Docker Repository
baseurl=http://mirrors.aliyun.com/docker-engine/yum/repo/main/centos/7/
enabled=1
gpgcheck=1
gpgkey=http://mirrors.aliyun.com/docker-engine/yum/gpg
EOF
  1. 安装docker
1
# yum install docker-engine
  1. 修改docker使用阿里云镜像加速
1
2
3
# cp -n /lib/systemd/system/docker.service /etc/systemd/system/docker.service
# sed -i "s|ExecStart=/usr/bin/dockerd|ExecStart=/usr/bin/dockerd --registry-mirror=https://dhxb****.mirror.aliyuncs.com|g" /etc/systemd/system/docker.service
# systemctl daemon-reload

上文 https://dhxb****.mirror.aliyuncs.com 是我的加速器地址,获取自己加速地址请参考阿里云:https://cr.console.aliyun.com/#/accelerator

  1. 启动docker并设置开机自启
1
2
# systemctl enable docker
# systemctl start docker

二、在compute节点安装并配置novadocker

  1. 安装novadocker
1
2
3
4
5
# usermod -aG docker nova
# yum -y install git python-pip
# pip install -e git+https://github.com/openstack/nova-docker#egg=novadocker
# cd src/novadocker/
# python setup.py install
  1. 配置 /etc/nova/nova.conf 使用docker driver
1
2
3
4
5
6
7
[DEFAULT]
compute_driver = novadocker.virt.docker.DockerDriver

[docker]
# Commented out. Uncomment these if you'd like to customize:
## vif_driver=novadocker.virt.docker.vifs.DockerGenericVIFDriver
## snapshots_directory=/var/tmp/my-snapshot-tempdir

将/src/novadocker/etc/nova/rootwrap.d/docker.filters文件拷贝到/etc/nova/rootwrap.d/docker.filters,并修改rootwrap.d的访问权限,然后启动nova-compute服务

1
2
# cp -R /src/novadocker/etc/nova/rootwrap.d /etc/nova/
# chown -R root:nova /etc/nova/rootwrap.d # systemctl restart openstack-nova-compute

三、上传镜像到glacne

  1. 在glance的配置文件中启用driver
1
2
3
# vim /etc/glance/glance-api.conf
[image_format]
container_formats = ami,ari,aki,bare,ovf,docker
  1. 重启glance-api服务
1
# openstack-sevice restart glance
  1. 获取docker镜像,并上传到glance中
1
2
# docker pull cirros
# docker save cirros | glance image-create --container-format=docker --disk-format=raw --name cirros

四、创建docker instance

  1. 创建实例
1
# nova boot --image cirros --flavor m1.tiny --nic net-id=59cc6a1d-0cc1-44c7-8b0a-9dc071fde397 cirros-docker
  1. 使用docker命令查看容器
1
2
3
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dc6e1c21887d cirros "/sbin/init" 47 minutes ago Up 47 minutes nova-bfeeb788-7fdf-476f-904a-8cc8ee3eb81c

注:dashboard上控制台无法使用

遇到的一些问题

修改使用docker driver后nova-compute的日志可以在 /var/log/message查看

1. 重启nova-conpute服务失败

1
2
3
4
……
Aug 08 12:14:51 compute2 nova-compute[21233]: 2016-08-08 12:14:51.388 21233 ERROR nova.virt.driver File "/usr/lib/python2.7/site-packages/oslo_config
Aug 08 12:14:51 compute2 nova-compute[21233]: 2016-08-08 12:14:51.388 21233 ERROR nova.virt.driver __import__(module_str)
Aug 08 12:14:51 compute2 nova-compute[21233]: 2016-08-08 12:14:51.388 21233 ERROR nova.virt.driver ImportError: No module named conf.netconf

解决方法:

1
2
3
# cd src/novadocker/
# git checkout -b stable/liberty origin/stable/liberty
# python setup.py install

然后即可正常启动nova-compute服务

2. 创建虚拟机的时候提示报错

1
404 Client Error: Not Found ("No such image: cirros-docker")]

解决方法:上传image的时候image name必须和docker image的名字一致,否则在创建instance的时候就是有上述错误

3. 启动虚拟机的时候报命名空间权限错误

1
2
3
4
Aug 8 14:12:59 compute2 nova-compute: 2016-08-08 14:12:59.200 12444 ERROR nova.compute.manager [instance: 3608b187-fe0c-4554-aa96-d5ed630042bc] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf ip netns exec ee27f11ab9dc265ad864dbcb8b9a800693fd9517f0bcfa166e3ccae66c300843 ip link set lo up
Aug 8 14:12:59 compute2 nova-compute: 2016-08-08 14:12:59.200 12444 ERROR nova.compute.manager [instance: 3608b187-fe0c-4554-aa96-d5ed630042bc] Exit code: 1
Aug 8 14:12:59 compute2 nova-compute: 2016-08-08 14:12:59.200 12444 ERROR nova.compute.manager [instance: 3608b187-fe0c-4554-aa96-d5ed630042bc] Stdout: u''
Aug 8 14:12:59 compute2 nova-compute: 2016-08-08 14:12:59.200 12444 ERROR nova.compute.manager [instance: 3608b187-fe0c-4554-aa96-d5ed630042bc] Stderr: u'Cannot open network namespace "ee27f11ab9dc265ad864dbcb8b9a800693fd9517f0bcfa166e3ccae66c300843": Permission denied\n'

解决方法:关闭selinux

1
2
# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
# reboot

参考文章

OpenStack里使用Nova Docker Driver需要注意的细节
https://github.com/openstack/nova-docker
http://heavenkong.blogspot.com/2016/07/resolved-mitaka-novadocker-error.html

一个默默无闻的工程师的日常
Built with Hugo
主题 StackJimmy 设计