使用 pacemaker 配置 rabbitmq 高可用集群

OpenStack 高可用集群各个服务组件之间使用消息队列进行通信，消息队列系统的高可用是 OpenStack 集群能够提供高可用服务的核心基础。在 OpenStack 中 Rabbitmq 是使用最多的高级消息队列系统。下文介绍如何使用 pacemaker 集群配置Rabbitmq 的高可用。

安装和配置组件

安装软件包

1

# yum install rabbitmq-server -y

防止网络分区

1
2
3
4
5


# vim /etc/rabbitmq/rabbitmq.config
修改：
%%  {cluster_partition_handling, ignore},
为：
  {cluster_partition_handling, pause_minority}

修改监听地址

1

# echo "NODE_IP_ADDRESS=${管理ip}" > /etc/rabbitmq/rabbitmq-env.conf

rabbitmq集群配置

创建自动创建用户设置权限的脚本

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


Vim /usr/local/sbin/set_rabbitmq_policy

# This script is called by rabbitmq-server-ha.ocf during RabbitMQ
# cluster start up. It is a convenient place to set your cluster
# policy here, for example:
#${OCF_RESKEY_ctl} set_policy ha-all "." '{"ha-mode":"all", "ha-sync-mode":"automatic"}' --apply-to all --priority 0
${OCF_RESKEY_ctl} set_policy ha-all "." '{"ha-mode":"all", "ha-sync-mode":"automatic"}' --apply-to all --priority 0
#add user usage:user passwd
${OCF_RESKEY_ctl} add_user openstack 7136fb0c6ac12946e06f
${OCF_RESKEY_ctl} set_permissions openstack ".*" ".*" ".*"

为脚本增加执行权限

1

# chmod 755 /usr/local/sbin/set_rabbitmq_policy

创建 rabbitmq 高可用集群

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


# pcs resource create --force --master rabbitmq-cluster ocf:rabbitmq:rabbitmq-server-ha \
    erlang_cookie=DPMDALGUKEOMPTHWPYKC node_port=5672 \
    op monitor interval=30 timeout=120 \
    op monitor interval=27 role=Master timeout=120 \
    op monitor interval=30 role=Slave timeout=120 OCF_CHECK_LEVEL=30 \
    op start interval=0 timeout=360 \
    op stop interval=0 timeout=120 \
    op promote interval=0 timeout=120 \
    op demote interval=0 timeout=120 \
    op notify interval=0 timeout=180 \
    meta notify=true ordered=false interleave=false master-max=1 master-node-max=1

验证

确认集群状态

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


  # rabbitmqctl cluster_status
  Cluster status of node rabbit@controller01 ...
  [{nodes,[{disc,[rabbit@controller01]},
  		{ram,[rabbit@controller03,rabbit@controller02]}]},
  {running_nodes,[rabbit@controller02,rabbit@controller03,rabbit@controller01]},
  {cluster_name,<<"rabbit@controller01">>},
  {partitions,[]},
  {alarms,[{rabbit@controller02,[]},
  		{rabbit@controller03,[]},
  		{rabbit@controller01,[]}]}]

确认高可用状态

1
2
3
4
5
6
7
8
9


# pcs resource 
 vip	(ocf::heartbeat:IPaddr2):	Started controller01
 Clone Set: haproxy-clone [haproxy]
     Started: [ controller01 controller02 controller03 ]
 Clone Set: memcached-clone [memcached]
     Started: [ controller01 controller02 controller03 ]
 Master/Slave Set: rabbitmq-cluster-master [rabbitmq-cluster]
     Masters: [ controller03 ]
     Slaves: [ controller01 controller02 ]

确认 rabbitmq 集群策略

1
2
3


# rabbitmqctl  list_policies
Listing policies ...
/	ha-all	all	.	{"ha-mode":"all","ha-sync-mode":"automatic"}	0

确认 rabbitmq 用户是否正确创建

1
2
3
4


# rabbitmqctl list_users
Listing users ...
openstack	[]
guest	[administrator]

确认 openstack 用户访问权限

1
2
3
4


# rabbitmqctl list_permissions 
Listing permissions in vhost "/" ...
guest	.*	.*	.*
openstack	.*	.*	.*