Home / server / Questions / 642763
Accepted
Navern
Asked: 2014-11-08 07:36:22 +0800 CST

MySQL replication shuts down after rotate of binlog

  • 772

I have following issue. I had successfull setup of master-replica(row-based). It has been running for 2-3 hours. After bin-log was rotated -> mysql-bin.index file got corrupted on master.

Here is show master logs;

show master logs\G

*************************** 1. row ***************************
 Log_name: mysql-bin.000001
File_size: 27682022

 *************************** 2. row ***************************
 Log_name: mysqld  Ver 5.1.73-5-beget-log for debian-linux-gnu on x86_64 ((LTD BeGet))
 File_size: 0

 *************************** 3. row ***************************
 Log_name: or its affiliates
 File_size: 0

 *************************** 4. row ***************************
 Log_name: or its affiliates. All rights reserved.
 File_size: 0

 4 rows in set (0.00 sec)

Configuration file of mysql on master:

cat /etc/mysql/my.cnf  | grep -v -E "^$" | grep -v -E "^#"
[mysqld]
event_scheduler = 1
interactive_timeout = 30
set-variable        = wait_timeout=30
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /home/mysql
tmpdir          = /home/tmpfs
server_id = 1
log-bin = mysql-bin
binlog-do-db = somedb
binlog-format = row
skip-external-locking
bind-address                = 0.0.0.0
default-character-set   = UTF8
innodb_file_per_table = 1
max_allowed_packet      = 128M
thread_stack            = 512K
thread_cache_size       = 64
myisam-recover          = BACKUP
thread_concurrency      = 48
max_user_connections    = 100
max_connections         = 200
back_log                = 500
join_buffer_size        = 4M
read_buffer_size        = 4M
read_rnd_buffer_size    = 8M
sort_buffer_size        = 4M
myisam_sort_buffer_size = 512M
query_cache_limit       = 4M
query_cache_size        = 128M
tmp_table_size          = 5000M
max_heap_table_size     = 5000M
key_buffer              = 128M
max_connect_errors      = 65000
max_error_count         = 65000
key_buffer_size         = 102M
open_files_limit        = 1040000
table_open_cache        = 10000
table_definition_cache  = 100000

mysql-bin.index file contains output of mysqld --help

show slave status has following errors:

Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 4
Relay_Log_File: mysqld-relay-bin.000003
Relay_Log_Pos: 152
Relay_Master_Log_File: mysql-bin.000002
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Could not find first log file name in binary log index file'
Last_SQL_Errno: 0
Last_SQL_Error:

Version of mysql:

mysql -V
mysql  Ver 14.14 Distrib 5.1.73, for debian-linux-gnu (x86_64) using readline 5.1

I haven't found anything strange in error log. Actually i don't know how to debug this. Any help will be appreciated.

linux

1 Answers

  1. Best Answer

    Ok, answering my own question. It's a bug we've discovered in MySQL server. Making it simple we had nagios check which looked like

    mysqld --verbose --help 2>&1 >&-

    Ok here is the problem, stdout is closed. And mysql prints output to index file which is opened before.

    We've opened bug for this case: https://bugs.launchpad.net/percona-server/+bug/1401547

    Fix was easy, just remove >&-, so we are not closing stdout, but it's still strange behaviour.

    • 0