使用 pacemaker 配置 rabbitmq 高可用集群

OpenStack 高可用集群各个服务组件之间使用消息队列进行通信，消息队列系统的高可用是 OpenStack 集群能够提供高可用服务的核心基础。在 OpenStack 中 Rabbitmq 是使用最多的高级消息队列系统。下文介绍如何使用 pacemaker 集群配置Rabbitmq 的高可用。

安装和配置组件

安装软件包

1

# yum install rabbitmq-server -y

防止网络分区

1
2
3
4
5


# vim /etc/rabbitmq/rabbitmq.config
修改：
%%  {cluster_partition_handling, ignore},
为：
  {cluster_partition_handling, pause_minority}

修改监听地址

1

# echo "NODE_IP_ADDRESS=${管理ip}" > /etc/rabbitmq/rabbitmq-env.conf

rabbitmq集群配置

创建自动创建用户设置权限的脚本

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


Vim /usr/local/sbin/set_rabbitmq_policy

# This script is called by rabbitmq-server-ha.ocf during RabbitMQ
# cluster start up. It is a convenient place to set your cluster
# policy here, for example:
#${OCF_RESKEY_ctl} set_policy ha-all "." '{"ha-mode":"all", "ha-sync-mode":"automatic"}' --apply-to all --priority 0
${OCF_RESKEY_ctl} set_policy ha-all "." '{"ha-mode":"all", "ha-sync-mode":"automatic"}' --apply-to all --priority 0
#add user usage:user passwd
${OCF_RESKEY_ctl} add_user openstack 7136fb0c6ac12946e06f
${OCF_RESKEY_ctl} set_permissions openstack ".*" ".*" ".*"

为脚本增加执行权限

1

# chmod 755 /usr/local/sbin/set_rabbitmq_policy

创建 rabbitmq 高可用集群

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


# pcs resource create --force --master rabbitmq-cluster ocf:rabbitmq:rabbitmq-server-ha \
    erlang_cookie=DPMDALGUKEOMPTHWPYKC node_port=5672 \
    op monitor interval=30 timeout=120 \
    op monitor interval=27 role=Master timeout=120 \
    op monitor interval=30 role=Slave timeout=120 OCF_CHECK_LEVEL=30 \
    op start interval=0 timeout=360 \
    op stop interval=0 timeout=120 \
    op promote interval=0 timeout=120 \
    op demote interval=0 timeout=120 \
    op notify interval=0 timeout=180 \
    meta notify=true ordered=false interleave=false master-max=1 master-node-max=1

验证

确认集群状态

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


  # rabbitmqctl cluster_status
  Cluster status of node rabbit@controller01 ...
  [{nodes,[{disc,[rabbit@controller01]},
  		{ram,[rabbit@controller03,rabbit@controller02]}]},
  {running_nodes,[rabbit@controller02,rabbit@controller03,rabbit@controller01]},
  {cluster_name,<<"rabbit@controller01">>},
  {partitions,[]},
  {alarms,[{rabbit@controller02,[]},
  		{rabbit@controller03,[]},
  		{rabbit@controller01,[]}]}]

确认高可用状态

1
2
3
4
5
6
7
8
9


# pcs resource 
 vip	(ocf::heartbeat:IPaddr2):	Started controller01
 Clone Set: haproxy-clone [haproxy]
     Started: [ controller01 controller02 controller03 ]
 Clone Set: memcached-clone [memcached]
     Started: [ controller01 controller02 controller03 ]
 Master/Slave Set: rabbitmq-cluster-master [rabbitmq-cluster]
     Masters: [ controller03 ]
     Slaves: [ controller01 controller02 ]

确认 rabbitmq 集群策略

1
2
3


# rabbitmqctl  list_policies
Listing policies ...
/	ha-all	all	.	{"ha-mode":"all","ha-sync-mode":"automatic"}	0

确认 rabbitmq 用户是否正确创建

1
2
3
4


# rabbitmqctl list_users
Listing users ...
openstack	[]
guest	[administrator]

确认 openstack 用户访问权限

1
2
3
4


# rabbitmqctl list_permissions 
Listing permissions in vhost "/" ...
guest	.*	.*	.*
openstack	.*	.*	.*

Nickname

Website

0/500

OωO
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
（╯‵□′）╯︵┴─┴
￣﹃￣
(/ω＼)
∠( ᐛ 」∠)＿
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ｀)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ(￣∇￣o)
ヾ(´･･｀｡)ノ"
( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
(ó﹏ò｡)
Σ(っ °Д °;)っ
( ,,´･ω･)ﾉ"(´っω･｀｡)
╮(╯▽╰)╭
o(*////▽////*)q
＞﹏＜
( ๑´•ω•) "(ㆆᴗㆆ)

😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣

颜文字
Emoji
Bilibili

1 comments

geekspeng 2019-04-10

你好，我按照上述步骤，进行检查，发现都ok，但是我执行 systemctl status rabbitmq-server 发现rabbitmq并没有启动起来
[root@node3 ~]# systemctl status rabbitmq-server
rabbitmq-server.service - RabbitMQ broker
Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
Active: inactive (dead)

Windows 7

Chrome 73.0.3683.103

tanmx 2019-04-10

Reply @geekspeng :

@geekspeng , 进程启动的不是通过服务启动，ps命令查看进程即可

macOS Mojave

Chrome 73.0.3683.86

geekspeng 2019-04-10

Reply @tanmx :

@tanmx , 已经看到了，非常感谢。

那我应该使用rabbitmq-server

Windows 7

Chrome 73.0.3683.103

geekspeng 2019-04-10

Reply @geekspeng :

@geekspeng , rabbit://node1:5672,node2:5672,node3:5672

Windows 7

Chrome 73.0.3683.103

geekspeng 2019-04-10

Reply @geekspeng :

@geekspeng , http://ip:25672访问web管理界面了，访问不了啊？
ps 查看端口为 25672？

[root@node3 ~]# ps aux | grep rabbitmq
rabbitmq 29819 0.0 0.0 48984 736 ? S 06:25 0:00 /usr/lib64/erlang/erts-7.3.1.2/bin/epmd -daemon
root 55061 0.0 0.2 88260 2320 ? S 06:36 0:00 su rabbitmq -s /bin/sh -c /usr/sbin/rabbitmq-server >> “/var/log/rabbitmq/startup_log” 2>/dev/null
rabbitmq 55065 0.0 0.1 9556 1156 ? Ss 06:36 0:00 sh -c /usr/sbin/rabbitmq-server >> “/var/log/rabbitmq/startup_log” 2>/dev/null
rabbitmq 55067 0.0 0.1 9560 1488 ? S 06:36 0:00 /bin/sh -e /usr/lib/rabbitmq/bin/rabbitmq-server
rabbitmq 55243 0.6 4.8 647328 48768 ? Sl 06:36 0:04 /usr/lib64/erlang/erts-7.3.1.2/bin/beam -W w -A 64 -P 1048576 -t 5000000 -stbt db -K true -B i – -root /usr/lib64/erlang -progname erl – -home /var/lib/rabbitmq – -pa /usr/lib/rabbitmq/lib/rabbitmq_server-3.6.5/ebin -noshell -noinput -sname rabbit@node3 -boot start_sasl -config /etc/rabbitmq/rabbitmq -kernel inet_default_connect_options [{nodelay,true}] -rabbit tcp_listeners [{“auto”,5672}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,”/var/log/rabbitmq/rabbit@node3.log“} -rabbit sasl_error_logger {file,”/var/log/rabbitmq/rabbit@node3-sasl.log“} -rabbit enabled_plugins_file “/etc/rabbitmq/enabled_plugins” -rabbit plugins_dir “/usr/lib/rabbitmq/lib/rabbitmq_server-3.6.5/plugins” -rabbit plugins_expand_dir “/var/lib/rabbitmq/mnesia/rabbit@node3-plugins-expand” -os_mon start_cpu_sup false -os_mon start_disksup false -os_mon start_memsup false -mnesia dir “/var/lib/rabbitmq/mnesia/rabbit@node3” -kernel inet_dist_listen_min 25672 -kernel inet_dist_listen_max 25672
rabbitmq 55617 0.0 0.0 11588 448 ? Ss 06:36 0:00 inet_gethost 4
rabbitmq 55618 0.0 0.0 13712 680 ? S 06:36 0:00 inet_gethost 4
root 81385 0.0 0.0 112704 968 pts/0 S+ 06:49 0:00 grep –color=auto rabbitmq

Windows 7

Chrome 73.0.3683.103

tanmx 2019-04-10

Reply @geekspeng :

@geekspeng , web管理页面要手动打开，打开方法百度一下就有了

iOS 12.1

Safari 12.0