CentOS Shell 腳本監控 MySQL 主從複制：自動檢測、異常日誌、自動修復跳過錯誤

前言

MySQL 主從複制偶爾因網路抖動、大事務、DDL 等導致中斷。本文提供兩個 Shell 腳本：基礎版（每 10 分鐘檢測、記錄日誌、嘗試 SKIP COUNTER=1 修復）與進階版（自動獲取 Master Binlog 位置、重新建立複制）。

一、基礎監控腳本 /home/crontab/check_mysql_slave.sh

#!/bin/sh
ip=eth0
mysql_binfile=/usr/local/mysql/bin/mysql
mysql_user=root
mysql_pass=123456
mysql_sockfile=/tmp/mysql.sock
datetime=`date +"%Y-%m-%d/%H:%M:%S"`
mysql_slave_logfile=/home/logs/check_mysql_slave.log

slave_ip=`ifconfig $ip | grep "inet addr" | awk -F[:" "]+ '{print $4}'`
status=$($mysql_binfile -u$mysql_user -p$mysql_pass -S $mysql_sockfile -e "SHOW SLAVE STATUS\G" | grep -i "running")

Slave_IO_Running=`echo $status | grep Slave_IO_Running | awk '{print $2}'`
Slave_SQL_Running=`echo $status | grep Slave_SQL_Running | awk '{print $2}'`

if [ "$Slave_IO_Running" = "Yes" -a "$Slave_SQL_Running" = "Yes" ]; then
  echo "Slave is Running!"
else
  echo " $datetime $slave_ip Slave is not running!" >> $mysql_slave_logfile
  $mysql_binfile -u$mysql_user -p$mysql_pass -S $mysql_sockfile -e "SLAVE STOP;"
  $mysql_binfile -u$mysql_user -p$mysql_pass -S $mysql_sockfile -e "SET GLOBAL SQL_SLAVE_SKIP_COUNTER=1;"
  $mysql_binfile -u$mysql_user -p$mysql_pass -S $mysql_sockfile -e "SLAVE START;"
  $mysql_binfile -u$mysql_user -p$mysql_pass -S $mysql_sockfile -e "EXIT"
fi

部署步驟

chmod +x /home/crontab/check_mysql_slave.sh
mkdir -p /home/logs
vim /etc/crontab
# 每 10 分鐘執行
*/10 * * * * root /home/crontab/check_mysql_slave.sh
/etc/rc.d/init.d/crond restart
chkconfig crond on
service crond start

二、進階自動重建複制腳本 slave_repl.sh

#!/bin/sh
mstool="/usr/local/mysql-3307/bin/mysql -h 192.168.1.106 -uroot -pw!zl7POg27 -P 3307"
sltool="/usr/local/mysql-3307/bin/mysql -h 192.168.1.107 -uroot -pw!zl7POg27 -P 3307"

declare -a slave_stat
slave_stat=($($sltool -e "SHOW SLAVE STATUS\G" | grep Running | awk '{print $2}'))

if [ "${slave_stat[0]}" = "Yes" -a "${slave_stat[1]}" = "Yes" ]; then
  echo "OK slave is running"; exit 0
else
  echo "Critical slave is error"
  echo "*********************************************************"
  echo "Now Starting replication with Master Mysql!"
  
  file=`$mstool -e "SHOW MASTER STATUS\G" | grep "File" | awk '{print $2}'`
  pos=`$mstool -e "SHOW MASTER STATUS\G" | grep "Pos" | awk '{print $2}'`
  
  $sltool -e "SLAVE STOP; CHANGE MASTER TO MASTER_HOST='192.168.1.106', MASTER_PORT=3307, MASTER_USER='repl', MASTER_PASSWORD='w!zl7POg27', MASTER_LOG_FILE='$file', MASTER_LOG_POS=$pos; SLAVE START;"
  sleep 3
  $sltool -e "SHOW SLAVE STATUS\G;" | grep Running
  echo "Now Replication is Finished!"
  echo "**********************************************************"
  exit 2
fi

三、執行效果演示

# ./slave_stop3307.sh
*******************************
Now stop Slave Replication!
Slave_IO_Running: No
Slave_SQL_Running: No
*******************************

# ./slave_repl3307.sh
Critical slave is error
*********************************************************
Now Starting replication with Master Mysql!
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Now Replication is Finished!
**********************************************************

四、生產環境建議增強

• 加入告警通知（郵件、釘釘、企業微信）
• 記錄詳細錯誤資訊：Last_Error, Last_SQL_Error
• 避免無腦 SKIP COUNTER：關鍵業務需人工介入，防止數據不一致
• GTID 模式下簡化為 CHANGE MASTER TO MASTER_AUTO_POSITION=1
• 監控 Seconds_Behind_Master 延遲指標，超閾值告警
• 使用 pt-heartbeat 或 orchestrator 進行更專業的複制管理