Skip to content

Commit ac1b464

Browse files
author
Alfranio Correia
committed
BUG#43929 binlog corruption when max_binlog_cache_size is exceeded
Large transactions and statements may corrupt the binary log if the size of the cache, which is set by the max_binlog_cache_size, is not enough to store the the changes. In a nutshell, to fix the bug, we save the position of the next character in the cache before starting processing a statement. If there is a problem, we simply restore the position thus removing any effect of the statement from the cache. Unfortunately, to avoid corrupting the binary log, we may end up loosing changes on non-transactional tables if they do not fit in the cache. In such cases, we store an Incident_log_event in order to stop the slave and alert users that some changes were not logged. Precisely, for every non-transactional changes that do not fit into the cache, we do the following: a) the statement is *not* logged b) an incident event is logged after committing/rolling back the transaction, if any. Note that if a failure happens before writing the incident event to the binary log, the slave will not stop and the master will not have reported any error. c) its respective statement gives an error For transactional changes that do not fit into the cache, we do the following: a) the statement is *not* logged b) its respective statement gives an error To work properly, this patch requires two additional things. Firstly, callers to MYSQL_BIN_LOG::write and THD::binlog_query must handle any error returned and take the appropriate actions such as undoing the effects of a statement. We already changed some calls in the sql_insert.cc, sql_update.cc and sql_insert.cc modules but the remaining calls spread all over the code should be handled in BUG#37148. Secondly, statements must be either classified as DDL or DML because DDLs that do not get into the cache must generate an incident event since they cannot be rolled back.
1 parent de713f7 commit ac1b464

10 files changed

+661
-43
lines changed

‎mysql-test/include/commit.inc

+2-6
Original file line numberDiff line numberDiff line change
@@ -669,13 +669,9 @@ call p_verify_status_increment(1, 0, 1, 0);
669669
insert t1 set a=3;
670670
call p_verify_status_increment(2, 2, 2, 2);
671671
savepoint a;
672-
call p_verify_status_increment(0, 0, 0, 0);
672+
call p_verify_status_increment(1, 0, 0, 0);
673673
insert t1 set a=4;
674-
--echo # Binlog does not register itself this time for other than the 1st
675-
--echo # statement of the transaction with MIXED/STATEMENT binlog_format.
676-
--echo # It needs registering with the ROW format. Therefore 1,0,2,2 are
677-
--echo # the correct arguments to this test after bug#40221 fixed.
678-
call p_verify_status_increment(1, 0, 2, 2);
674+
call p_verify_status_increment(2, 2, 2, 2);
679675
release savepoint a;
680676
rollback;
681677
call p_verify_status_increment(0, 0, 0, 0);

‎mysql-test/r/commit_1innodb.result

+2-6
Original file line numberDiff line numberDiff line change
@@ -766,15 +766,11 @@ call p_verify_status_increment(2, 2, 2, 2);
766766
SUCCESS
767767

768768
savepoint a;
769-
call p_verify_status_increment(0, 0, 0, 0);
769+
call p_verify_status_increment(1, 0, 0, 0);
770770
SUCCESS
771771

772772
insert t1 set a=4;
773-
# Binlog does not register itself this time for other than the 1st
774-
# statement of the transaction with MIXED/STATEMENT binlog_format.
775-
# It needs registering with the ROW format. Therefore 1,0,2,2 are
776-
# the correct arguments to this test after bug#40221 fixed.
777-
call p_verify_status_increment(1, 0, 2, 2);
773+
call p_verify_status_increment(2, 2, 2, 2);
778774
SUCCESS
779775

780776
release savepoint a;

‎mysql-test/suite/rpl/r/rpl_binlog_max_cache_size.result

+135
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
stop slave;
2+
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
3+
reset master;
4+
reset slave;
5+
drop table if exists t1,t2,t3,t4,t5,t6,t7,t8,t9;
6+
start slave;
7+
CREATE TABLE t1(a INT PRIMARY KEY, data VARCHAR(30000)) ENGINE=Innodb;
8+
CREATE TABLE t2(a INT PRIMARY KEY, data VARCHAR(30000)) ENGINE=MyIsam;
9+
CREATE TABLE t3(a INT PRIMARY KEY, data VARCHAR(30000)) ENGINE=Innodb;
10+
########################################################################################
11+
# 1 - SINGLE STATEMENT
12+
########################################################################################
13+
*** Single statement on transactional table ***
14+
Got one of the listed errors
15+
*** Single statement on non-transactional table ***
16+
*** After WL#2687 the difference between STATEMENT/MIXED and ROW will not exist. ***
17+
Got one of the listed errors
18+
*** Single statement on both transactional and non-transactional tables. ***
19+
*** After WL#2687 we will be able to change the order of the tables. ***
20+
Got one of the listed errors
21+
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
22+
START SLAVE SQL_THREAD;
23+
TRUNCATE TABLE t1;
24+
TRUNCATE TABLE t2;
25+
TRUNCATE TABLE t3;
26+
BEGIN;
27+
Got one of the listed errors
28+
Got one of the listed errors
29+
Got one of the listed errors
30+
BEGIN;
31+
Got one of the listed errors
32+
Got one of the listed errors
33+
Got one of the listed errors
34+
BEGIN;
35+
Got one of the listed errors
36+
Got one of the listed errors
37+
source include/diff_master_slave.inc;
38+
########################################################################################
39+
# 3 - BEGIN - COMMIT
40+
########################################################################################
41+
TRUNCATE TABLE t1;
42+
TRUNCATE TABLE t2;
43+
TRUNCATE TABLE t3;
44+
BEGIN;
45+
Got one of the listed errors
46+
Got one of the listed errors
47+
Got one of the listed errors
48+
COMMIT;
49+
source include/diff_master_slave.inc;
50+
########################################################################################
51+
# 4 - BEGIN - ROLLBACK
52+
########################################################################################
53+
TRUNCATE TABLE t1;
54+
TRUNCATE TABLE t2;
55+
TRUNCATE TABLE t3;
56+
BEGIN;
57+
Got one of the listed errors
58+
Got one of the listed errors
59+
Got one of the listed errors
60+
ROLLBACK;
61+
Warnings:
62+
Warning 1196 Some non-transactional changed tables couldn't be rolled back
63+
source include/diff_master_slave.inc;
64+
########################################################################################
65+
# 5 - PROCEDURE
66+
########################################################################################
67+
TRUNCATE TABLE t1;
68+
TRUNCATE TABLE t2;
69+
TRUNCATE TABLE t3;
70+
CREATE PROCEDURE p1(pd VARCHAR(30000))
71+
BEGIN
72+
INSERT INTO t1 (a, data) VALUES (1, pd);
73+
INSERT INTO t1 (a, data) VALUES (2, pd);
74+
INSERT INTO t1 (a, data) VALUES (3, pd);
75+
INSERT INTO t1 (a, data) VALUES (4, pd);
76+
INSERT INTO t1 (a, data) VALUES (5, 's');
77+
END//
78+
TRUNCATE TABLE t1;
79+
TRUNCATE TABLE t1;
80+
BEGIN;
81+
Got one of the listed errors
82+
COMMIT;
83+
TRUNCATE TABLE t1;
84+
BEGIN;
85+
Got one of the listed errors
86+
ROLLBACK;
87+
source include/diff_master_slave.inc;
88+
########################################################################################
89+
# 6 - XID
90+
########################################################################################
91+
TRUNCATE TABLE t1;
92+
TRUNCATE TABLE t2;
93+
TRUNCATE TABLE t3;
94+
BEGIN;
95+
Got one of the listed errors
96+
Got one of the listed errors
97+
Got one of the listed errors
98+
ROLLBACK TO sv;
99+
Warnings:
100+
Warning 1196 Some non-transactional changed tables couldn't be rolled back
101+
COMMIT;
102+
source include/diff_master_slave.inc;
103+
########################################################################################
104+
# 7 - NON-TRANS TABLE
105+
########################################################################################
106+
TRUNCATE TABLE t1;
107+
TRUNCATE TABLE t2;
108+
TRUNCATE TABLE t3;
109+
BEGIN;
110+
Got one of the listed errors
111+
Got one of the listed errors
112+
Got one of the listed errors
113+
Got one of the listed errors
114+
Got one of the listed errors
115+
COMMIT;
116+
BEGIN;
117+
Got one of the listed errors
118+
COMMIT;
119+
########################################################################################
120+
# CLEAN
121+
########################################################################################
122+
DROP TABLE t1;
123+
DROP TABLE t2;
124+
DROP TABLE t3;
125+
DROP TABLE IF EXISTS t4;
126+
DROP TABLE IF EXISTS t5;
127+
DROP TABLE IF EXISTS t6;
128+
DROP PROCEDURE p1;
129+
DROP TABLE t1;
130+
DROP TABLE t2;
131+
DROP TABLE t3;
132+
DROP TABLE IF EXISTS t4;
133+
DROP TABLE IF EXISTS t5;
134+
DROP TABLE IF EXISTS t6;
135+
DROP PROCEDURE p1;

‎mysql-test/suite/rpl/t/rpl_binlog_max_cache_size-master.opt

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
--binlog_cache_size=4096 --max_binlog_cache_size=7680

0 commit comments

Comments
 (0)