1 MHA 工作原理和架构
MHA集群架构
MHA工作原理
的master保存二进制日志事件(binlog events)识别含有最新更新的slave应用差异的中继日志(relay log)到其他的slave应用从master保存的二进制日志事件(binlog events)提升一个slave为新的master使其他的slave连接新的master进行复制
MHA软件
MHA软件由两部分组成,Manager工具包和Node工具包
Manager工具包主要包括以下几个工具:
masterha_check_ssh | 检查MHA的SSH配置状况 |
masterha_check_repl | 检查MySQL复制状况 |
masterha_manger | 启动MHA |
masterha_check_status | 检测当前MHA运行状态 |
masterha_master_monitor | 检测master是否宕机 |
masterha_master_switch | 故障转移(自动或手动) |
masterha_conf_host | 添加或删除配置的server信息 |
masterha_stop --conf=app1.cnf | 停止MHA |
masterha_secondary_check | 两个或多个网络线路检查MySQL主服务器的可用 |
**Node工具包:**这些工具通常由MHA Manager的脚本触发,无需人为操作)主要包括以下几个工具:
save_binary_logs | #保存和复制master的二进制日志 |
apply_diff_relay_logs | #识别差异的中继日志事件并将其差异的事件应用于其他的slave |
filter_mysqlbinlog | #去除不必要的ROLLBACK事件(MHA已不再使用此工具) |
purge_relay_logs | #清除中继日志(不会阻塞SQL线程) |
MHA自定义扩展:
secondary_check_script | #通过多条网络路由检测master的可用性 |
master_ip_ailover_script | #更新Application使用的masterip |
shutdown_script | #强制关闭master节点 |
report_script | #发送报告 |
init_conf_load_script | #加载初始配置参数 |
master_ip_online_change_script | #更新master节点ip地址 |
MHA配置文件:
global配置,为各application提供默认配置,默认文件路径 /etc/masterha_default.cnf
application配置:为每个主从复制集群
2 实现 MHA 实战案例
环境:四台主机
192.168.100.120 CentOS7 MHA管理端192.168.100.104 CentOS8 MySQL8.0 Master192.168.100.105 CentOS8 MySQL8.0 Slave1192.168.100.106 CentOS8 MySQL8.0 Slave22.1 在管理节点上安装两个包mha4mysql-manager和mha4mysql-node
mha4mysql-manager和mha4mysql-node安装包:
链接:https://pan.baidu.com/s/1BvaQBQEdx-9N3w1fy3YnTA?pwd=grc0 提取码:grc0
[root@mha-manager ~]#yum -y install mha4mysql-manager-0.58-0.el7.centos.noarch.rpm[root@mha-manager ~]#yum -y install mha4mysql-node-0.58-0.el7.centos.noarch.rpm
2.2 在所有MySQL服务器上安装mha4mysql-node包
[root@master ~]#yum -y install mha4mysql-node-0.58-0.el7.centos.noarch.rpm
2.3 在所有节点实现相互之间ssh key验证
[root@mha-manager ~]#ssh-keygen[root@mha-manager ~]#ssh-copy-id 127.0.0.1[root@mha-manager ~]#rsync -av .ssh 192.168.100.104:/root/[root@mha-manager ~]#rsync -av .ssh 192.168.100.105:/root/[root@mha-manager ~]#rsync -av .ssh 192.168.100.106:/root/
2.4 在管理节点建立配置文件
[root@mha-manager ~]#mkdir /etc/mastermha/[root@mha-manager ~]#vim /etc/mastermha/app1.cnf[server default]user=mhauserpassword=magedumanager_workdir=/data/mastermha/app1/manager_log=/data/mastermha/app1/manager.logremote_workdir=/data/mastermha/app1/ssh_user=rootrepl_user=repluserrepl_password=123456ping_interval=1master_ip_failover_script=/usr/local/bin/master_ip_failoverreport_script=/usr/local/bin/sendmail.shcheck_repl_delay=0master_binlog_dir=/data/mysql/[server1]hostname=192.168.100.104candidate_master=1[server2]hostname=192.168.100.105candidate_master=1[server3]hostname=192.168.100.106
2.5 相关脚本
实现邮件通信
yum -y install postfix mailx systemctl enable --now postfix
vim /etc/mail.rc#在下面加上set from=邮箱set smtp=smtp.qq.comset smtp-auth-user=邮箱set smtp-auth-password=授权码
获取授权码:
再邮箱里面打开IMAP/SMTP服务,获取授权码!
[root@mha-manager ~]#vim /usr/local/bin/sendmail.sh#!/bin/bashecho "MySQL is down" | mail -s "MHA Warning" 邮箱[root@mha-manager ~]#chmod +x /usr/local/bin/sendmail.sh
[root@mha-manager ~]#vim /usr/local/bin/master_ip_failover#!/usr/bin/env perluse strict;use warnings FATAL => 'all';use Getopt::Long;my ($command, $ssh_user, $orig_master_host, $orig_master_ip,$orig_master_port, $new_master_host, $new_master_ip, $new_master_port);#执行时必须删除下面三行注释my $vip = '192.168.100.150/24';#设置Virtual IPmy $gateway = '192.168.100.2';#网关Gateway IPmy $interface = 'eth0'; #指定VIP所在网卡my $key = "1";my $ssh_start_vip = "/sbin/ifconfig $interface:$key $vip;/sbin/arping -I $interface -c 3 -s $vip $gateway >/dev/null 2>&1";my $ssh_stop_vip = "/sbin/ifconfig $interface:$key down";GetOptions('command=s' => \$command,'ssh_user=s' => \$ssh_user,'orig_master_host=s' => \$orig_master_host,'orig_master_ip=s' => \$orig_master_ip,'orig_master_port=i' => \$orig_master_port,'new_master_host=s' => \$new_master_host,'new_master_ip=s' => \$new_master_ip,'new_master_port=i' => \$new_master_port,);exit &main();sub main {print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";if ( $command eq "stop" || $command eq "stopssh" ) {# $orig_master_host, $orig_master_ip, $orig_master_port are passed.# If you manage master ip address at global catalog database,# invalidate orig_master_ip here.my $exit_code = 1;eval {print "Disabling the VIP on old master: $orig_master_host \n";&stop_vip();$exit_code = 0;};if ($@) {warn "Got Error: $@\n";exit $exit_code;}exit $exit_code;}elsif ( $command eq "start" ) {# all arguments are passed.# If you manage master ip address at global catalog database,# activate new_master_ip here.# You can also grant write access (create user, set read_only=0, etc) here.my $exit_code = 10;eval {print "Enabling the VIP - $vip on the new master - $new_master_host \n";&start_vip();$exit_code = 0;};if ($@) {warn $@;exit $exit_code;}exit $exit_code;}elsif ( $command eq "status" ) {print "Checking the Status of the script.. OK \n";`ssh $ssh_user\@$orig_master_host \" $ssh_start_vip \"`;exit 0;}else {&usage();exit 1;}}#A simple system call that enable the VIP on the new mastersub start_vip() {`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;}# A simple system call that disable the VIP on the old_mastersub stop_vip() {`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;}sub usage {print"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";}
授予执行权限:
[root@mha-manager ~]#chmod +x /usr/local/bin/master_ip_failover
2.6 实现Master
[root@master ~]#mkdir /data/mysql/[root@master ~]#chown mysql.mysql /data/mysql/[root@master ~]#dnf -y install mysql-server[root@master ~]#vim /etc/my.cnf[mysqld]server_id=1log-bin=/data/mysql/mysql-binskip_name_resolve=1general_log #观察结果,非必须项,生产无需启用[root@master ~]#systemctl enable --now mysqldmysql>show master logs;mysql>create user repluser@'192.168.100.%' identified by '123456';mysql>grant replication slave on *.* to repluser@'192.168.100.%' ;mysql>create user mhauser@'192.168.100.%' identified by '123456';mysql>grant all on *.* to mhauser@'192.168.100.%' ;#配置VIP[root@master ~]#ifconfig eth0:1 192.168.100.150/24
2.7 实现slave
[root@slave ~]#mkdir /data/mysql[root@slave ~]#chown mysql.mysql /data/mysql/[root@slave ~]#dnf -y install mysql-server[root@slave ~]#vim /etc/my.cnf[mysqld]server_id=2 #不同节点此值各不相同log-bin=/data/mysql/mysql-binread_onlyrelay_log_purge=0skip_name_resolve=1 #禁止反向解析general_log #方便观察的设置,生产无需启用[root@slave ~]#systemctl enable --now mysqldmysql>CHANGE MASTER TO MASTER_HOST='MASTER_IP', MASTER_USER='repluser', MASTER_PASSWORD='123456', MASTER_LOG_FILE='mysql-bin.000001', MASTER_LOG_POS=245;mysql>START SLAVE;
2.8 检查MHA的环境
#检查环境
[root@mha-manager ~]#masterha_check_ssh --conf=/etc/mastermha/app1.cnf
[root@mha-manager ~]#masterha_check_repl --conf=/etc/mastermha/app1.cnf
#查看状态
[root@mha-manager ~]#masterha_check_status --conf=/etc/mastermha/app1.cnf
2.9 启动MHA
#开启MHA,默认是前台运行,生产环境一般为后台执行nohup masterha_manager --conf=/etc/mastermha/app1.cnf &> /dev/null
#查看状态masterha_check_status --conf=/etc/mastermha/app1.cnf
2.10 排错日志
tail /data/mastermha/app1/manager.log
2.11 模拟故障检验
#当 master down机后,mha管理程序自动退出
通过查看日志可以看到主节点(192.168.100.104)已经切换为(192.168.100.105)
tail -f /data/mastermha/app1/manager.log
#查看状态masterha_check_status --conf=/etc/mastermha/app1.cnf
在192.168.100.105查看开启的进程
可以看到192.168.100.105已经成为主节点
在192.168.100.106查看从节点状态信息
可以看到192.168.100.106的主节点变成了192.168.100.106