2017年5月

devstack dashboard 开启开发者选项 和 OpenStack Profiler

在ocata的版本中,引入了一个新的“openstack profiler”的面板,启用openstack profiler可以方便的看到访问horizon页面时的API调用情况。如下图所示:
image
下面介绍如何启用 openstack profiler,首先需要一个正常运行的devstack环境,启用方法如下

安装mongoDB

Horizon会将API调用过程的数据都保存到mongodb中,mongodb可以安装在本机,也可以在本机能够访问的任意一台机器上。

  1. 安装软件包

    # yum install mongodb-server mongodb -y
  2. 编辑文件 /etc/mongod.conf 并完成如下动作:
    • 配置 bind_ip 使用本机 ip 或者 0.0.0.0。
      bind_ip = 192.168.3.222
    • 默认情况下,MongoDB会在/var/lib/mongodb/journal 目录下创建几个 1 GB 大小的日志文件。如果你想将每个日志文件大小减小到128MB并且限制日志文件占用的总空间为512MB,配置 smallfiles 的值:
      smallfiles = true
  3. 启动MongoDB 并配置它随系统启动
    # systemctl enable mongod.service
    # systemctl start mongod.service

配置 horizon

  1. 复制文件
    $ cd /opt/stack/horizon
    $ cp openstack_dashboard/contrib/developer/enabled/_9001_developer.py openstack_dashboard/local/enabled/
    $ cp openstack_dashboard/contrib/developer/enabled/_9030_profiler.py openstack_dashboard/local/enabled/
    $ cp openstack_dashboard/contrib/developer/enabled/_9010_preview.py openstack_dashboard/local/enabled/
    $ cp openstack_dashboard/local/local_settings.d/_9030_profiler_settings.py.example openstack_dashboard/local/local_settings.d/_9030_profiler_settings.py
  2. 编辑 _9030_profiler_settings.py 文件,修改 mongoDB 相关配置
    修改 OPENSTACK_HOST 为mongoDB所在地址

    $ vim openstack_dashboard/local/local_settings.d/_9030_profiler_settings.py
    
    OPENSTACK_PROFILER.update({
      'enabled': True,
      'keys': ['SECRET_KEY'],
      'notifier_connection_string': 'mongodb://192.168.3.222:27017',
      'receiver_connection_string': 'mongodb://192.168.3.222:27017'
    })
  3. 重启 horizon,重新登录 dashboard ,会发现右上角有一个 Profile 下拉菜单,如下图:
    image
    如果要获取当前页面的API调用数据,点击 Profile Current Page 会重新刷新页面,加载完成后,到 Developer 下面的 OpenStack Profiler 页面就会看到页面加载过程的详细数据。

参考文章:
孔令贤-OpenStack Horizon Profiling
OpenStack Installation Guide for Red Hat Enterprise Linux and CentOS

ssh 无密码访问的问题

ssh 无密码登录失败

虚拟机 resize 需要配置计算节点之间 nova 用户无密码访问,但是在配置过程中有一台始终不能用密钥登录,对比了正常可以无密码登录的日志如下。

# 正常登录
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /var/lib/nova/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg ssh-rsa blen 279
# 异常报错
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /var/lib/nova/.ssh/id_rsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /var/lib/nova/.ssh/id_dsa
debug3: no such identity: /var/lib/nova/.ssh/id_dsa: No such file or directory
debug1: Trying private key: /var/lib/nova/.ssh/id_ecdsa
debug3: no such identity: /var/lib/nova/.ssh/id_ecdsa: No such file or directory
debug1: Trying private key: /var/lib/nova/.ssh/id_ed25519
debug3: no such identity: /var/lib/nova/.ssh/id_ed25519: No such file or directory
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password

分析问题

  1. 找个一个类似报错的 CentOS SSH公钥登录问题 ,文中是由于seliunx导致的,我查看了本地的selinux发现已经关闭,不适用我的情况

  2. 使用 journalctl _COMM=sshd 命令查看日志,发现如下权限问题
May 10 17:11:11 compute01 sshd[26498]: pam_systemd(sshd:session): Failed to release session: Interrupted system call
May 10 17:11:11 compute01 sshd[26498]: pam_unix(sshd:session): session closed for user root
May 10 17:12:28 compute01 sshd[2297]: Authentication refused: bad ownership or modes for directory /var/lib/nova
May 10 17:13:09 compute01 sshd[2297]: Connection closed by 192.168.101.105 [preauth]
May 10 17:13:33 compute01 sshd[4103]: Authentication refused: bad ownership or modes for directory /var/lib/nova
May 10 17:25:21 compute01 sshd[23157]: Authentication refused: bad ownership or modes for directory /var/lib/nova
May 10 17:25:25 compute01 sshd[23157]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=compute02  user=nova
  1. 对比无问题主机的 /var/lib/nova 权限
正常主机
drwxr-xr-x   8 nova    nova     118 May 10 16:59 nova
异常主机
drwxrwxrwx. 11 nova           nova            4096 May 10 17:07 nova
  1. 解决办法
    修改 /var/lib/nova 目录权限为 755 后,可以正常无密码登录
# chmod -R 755 /var/lib/nova/

openstack HA模式下控制台无法访问的问题

控制台无法访问,多次刷新才能访问,nova有如下报错

2017-02-09 17:09:51.311 57467 INFO nova.console.websocketproxy [-] 192.168.170.41 - - [09/Feb/2017 17:09:51] "GET /websockify HTTP/1.1" 101 -
2017-02-09 17:09:51.312 57467 INFO nova.console.websocketproxy [-] 192.168.170.41 - - [09/Feb/2017 17:09:51] 192.168.170.41: Plain non-SSL (ws://) WebSocket connection
2017-02-09 17:09:51.313 57467 INFO nova.console.websocketproxy [-] 192.168.170.41 - - [09/Feb/2017 17:09:51] 192.168.170.41: Version hybi-13, base64: 'False'
2017-02-09 17:09:51.313 57467 INFO nova.console.websocketproxy [-] 192.168.170.41 - - [09/Feb/2017 17:09:51] 192.168.170.41: Path: '/websockify'
2017-02-09 17:09:51.382 57467 INFO nova.console.websocketproxy [req-f51929d9-8c9b-4df0-abeb-247ce6ef5d65 - - - - -] handler exception: The token '1dfc9af9-8a49-44b3-a955-5196197bc8f7' is invalid or has expired

原因分析

When running a multi node environment with HA between two or more controller nodes(or controller plane service nodes), nova consoleauthservice must be configured with memcached.  
If not, no more than one consoleauth service can berunning in active state, since it need to save the state of the sessions. Whenmemcached is not used, you can check that can connect to the vnc console only afew times when you refresh the page. If that occurs means that the connectionis handled by the consoleauth service that currently is issuing sessions.    
To solve your issue, configure memcached as backend tonova-consoleauth service.  
To solve your issue add this line to nova.conf:  
memcached_servers = 192.168.100.2:11211,192.168.100.3:11211  
This should work to solve your issue.  

解决

M版在增加memcached_servers选项

# vim /etc/nova/nova.conf

[DEFAULT]
# "memcached_servers" opt is deprecated in Mitaka. In Newton release oslo.cache
# config options should be used as this option will be removed. Please add a
# [cache] group in your nova.conf file and add "enable" and "memcache_servers"
# option in this section. (list value)
memcached_servers=controller01:11211,controller02:11211,controller03:11211

如果是N版的话,memcached_servers已经废弃,需要按照如下修改

[cache]
enabled=true
backend=oslo_cache.memcache_pool
memcache_servers=controller01:11211,controller02:11211,controller03:11211