Skip to content

Commit fdc0314

Browse files
author
Joao Gramacho
committed
BUG#27638268: GTID AUTO SKIP DOES NOT WORK FOR QUERIES WITH PARSER ERRORS
Problem ======= Replicating from older to newer MySQL server versions may lead to ER_PARSE_ERROR on replication applier threads because of, for example, the workload is creating a table using a name that on the older server version is now a reserved word. One common workaround fix for this issue DBAs/operators do is to apply the offending transaction manually (changing it enough to be applied without errors) and making the replication applier thread to skip the offending transaction already present on slaves relay log. When GTID_MODE is ON, the recommended way of making the replication applier to skip a transaction is to commit its GTID with an empty transactions (or with the replacement transaction), but GTID auto skip was not working for statements leading to ER_PARSE_ERROR. Analysis ======== GTID auto skip functionality depends on MySQL parser for: - Not skip "innocent" statements. Here, "innocent" includes SET, SHOW, SELECT, DO, USE, and empty query, unless it invokes a stored function. The reason we allow innocent statements to execute even within skipped transactions is that they can be useful for testing and debugging. E.g., we want to run SELECT @@session.gtid_owned to check that there is not a bug that makes the server acquire ownership. - Keep track of transaction boundaries. When applying a workload dumped from mysqlbinlog client program, even skipping statements from already applied transactions, the client session applying the workload must known if a statement is a DDL or a DML (if it is supposed to be wrapped with "BEGIN/COMMIT" or not). Fix === This patch will make the replication applier thread to ignore ER_PARSE_ERROR when applying query events if the transaction being applied is also being GTID auto skipped. Making workloads dumped from mysqlbinlog to also work with GTID auto skip when some statements lead to ER_PARSE_ERROR is too risk, as the GTID enforcement logic could not be guaranteed without having detailed information about the offending statement.
1 parent f8da281 commit fdc0314

File tree

4 files changed

+146
-0
lines changed

4 files changed

+146
-0
lines changed

‎mysql-test/std_data/binlog_56_gtid_reserved_word.000001

1.39 KB
Binary file not shown.

‎mysql-test/suite/rpl/r/rpl_gtid_skip_with_parser_error.result

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
include/master-slave.inc
2+
Warnings:
3+
Note #### Sending passwords in plain text without SSL/TLS is extremely insecure.
4+
Note #### Storing MySQL user name or password information in the master info repository is not secure and is therefore not recommended. Please consider using the USER and PASSWORD connection options for START SLAVE; see the 'START SLAVE Syntax' in the MySQL Manual for more information.
5+
[connection master]
6+
include/rpl_stop_server.inc [server_number=1]
7+
include/rpl_start_server.inc [server_number=1]
8+
include/assert.inc [All transactions are assumed as applied on master]
9+
[connection slave]
10+
include/start_slave_io.inc
11+
[connection master]
12+
include/sync_slave_io_with_master.inc
13+
START SLAVE SQL_THREAD;
14+
include/wait_for_slave_sql_error.inc [errno=1064]
15+
include/assert.inc [Only 1st transaction is applied on slave]
16+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:2';
17+
BEGIN;
18+
COMMIT;
19+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:3';
20+
BEGIN;
21+
COMMIT;
22+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:4';
23+
BEGIN;
24+
COMMIT;
25+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:6';
26+
BEGIN;
27+
COMMIT;
28+
SET GTID_NEXT=AUTOMATIC;
29+
include/start_slave_sql.inc
30+
include/sync_slave_sql_with_io.inc
31+
CALL mtr.add_suppression("The slave coordinator and worker threads are stopped");
32+
include/rpl_end.inc

‎mysql-test/suite/rpl/t/rpl_gtid_skip_with_parser_error.test

+107
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
# ==== Purpose ====
2+
#
3+
# Test that GTID auto skip works even when replication applier is applying
4+
# events that would lead to ER_PARSE_ERROR.
5+
#
6+
# For this test case a binary log file was generated in MySQL 5.6 creating
7+
# a table with a name that is a reserved word in MySQL 5.7+.
8+
#
9+
# Replicating the binary log to a 5.7+ slave shall make the applier thread
10+
# to stop once reaching the offending transaction.
11+
#
12+
# Supposing a DBA/operator fixed the issue manually and the GTID of the
13+
# offending transaction was committed so the replicated one shall be skipped
14+
# by GTID auto skip functionality, restarting the slave applier thread shall
15+
# not hit errors for this transaction anymore.
16+
#
17+
# The binary log file generated in MySQL 5.6 has the following content:
18+
#
19+
#+-------------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
20+
#| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
21+
#+-------------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
22+
#| master-bin.000001 | 4 | Format_desc | 1 | 120 | Server ver: 5.6.41-debug-log, Binlog ver: 4 |
23+
#| master-bin.000001 | 120 | Previous_gtids | 1 | 151 | |
24+
#| master-bin.000001 | 151 | Gtid | 1 | 199 | SET @@SESSION.GTID_NEXT= '11111111-1111-1111-1111-111111111111:1' |
25+
#| master-bin.000001 | 199 | Query | 1 | 308 | use `test`; CREATE TABLE t1 (a int PRIMARY KEY) |
26+
#| master-bin.000001 | 308 | Gtid | 1 | 356 | SET @@SESSION.GTID_NEXT= '11111111-1111-1111-1111-111111111111:2' |
27+
#| master-bin.000001 | 356 | Query | 1 | 458 | use `test`; CREATE TABLE GENERATED LIKE test.t1 |
28+
#| master-bin.000001 | 458 | Gtid | 1 | 506 | SET @@SESSION.GTID_NEXT= '11111111-1111-1111-1111-111111111111:3' |
29+
#| master-bin.000001 | 506 | Query | 1 | 585 | BEGIN |
30+
#| master-bin.000001 | 585 | Query | 1 | 701 | use `test`; INSERT INTO GENERATED VALUES (1), (2), (3) |
31+
#| master-bin.000001 | 701 | Query | 1 | 781 | COMMIT |
32+
#| master-bin.000001 | 781 | Gtid | 1 | 829 | SET @@SESSION.GTID_NEXT= '11111111-1111-1111-1111-111111111111:4' |
33+
#| master-bin.000001 | 829 | Query | 1 | 901 | BEGIN |
34+
#| master-bin.000001 | 901 | Table_map | 1 | 953 | table_id: 75 (test.GENERATED) |
35+
#| master-bin.000001 | 953 | Write_rows | 1 | 993 | table_id: 75 flags: STMT_END_F |
36+
#| master-bin.000001 | 993 | Query | 1 | 1066 | COMMIT |
37+
#| master-bin.000001 | 1066 | Gtid | 1 | 1114 | SET @@SESSION.GTID_NEXT= '11111111-1111-1111-1111-111111111111:5' |
38+
#| master-bin.000001 | 1114 | Query | 1 | 1229 | use `test`; DROP TABLE `t1` /* generated by server */ |
39+
#| master-bin.000001 | 1229 | Gtid | 1 | 1277 | SET @@SESSION.GTID_NEXT= '11111111-1111-1111-1111-111111111111:6' |
40+
#| master-bin.000001 | 1277 | Query | 1 | 1399 | use `test`; DROP TABLE `GENERATED` /* generated by server */ |
41+
#+-------------------+------+----------------+-----------+-------------+-------------------------------------------------------------------+
42+
#
43+
# ==== Related Bugs and Worklogs ====
44+
#
45+
# BUG#27638268 GTID AUTO SKIP DOES NOT WORK FOR QUERIES WITH PARSER ERRORS
46+
#
47+
48+
# This test case is not compatible with SBR.
49+
--source include/have_binlog_format_mixed_or_row.inc
50+
--source include/have_gtid.inc
51+
--let $rpl_skip_start_slave = 1
52+
--source include/master-slave.inc
53+
54+
--let $MASTER_DATADIR= `select @@datadir`
55+
56+
# Restart the master to make it use the 5.6 binary log file as its own.
57+
--let $rpl_server_number= 1
58+
--source include/rpl_stop_server.inc
59+
--remove_file $MASTER_DATADIR/master-bin.000001
60+
--copy_file std_data/binlog_56_gtid_reserved_word.000001 $MASTER_DATADIR/master-bin.000001
61+
--source include/rpl_start_server.inc
62+
63+
# Assert that all 6 transactions are assumed as already applied.
64+
--let $assert_text= All transactions are assumed as applied on master
65+
--let $assert_cond= "[SELECT @@GLOBAL.gtid_executed]" = "11111111-1111-1111-1111-111111111111:1-6"
66+
--source include/assert.inc
67+
68+
# Start and sync slave's receiver thread
69+
--source include/rpl_connection_slave.inc
70+
--source include/start_slave_io.inc
71+
--source include/rpl_connection_master.inc
72+
--source include/sync_slave_io_with_master.inc
73+
74+
# Start slave's applier thread. It should stop with ER_PARSE_ERROR.
75+
START SLAVE SQL_THREAD;
76+
--let $slave_sql_errno= convert_error(ER_PARSE_ERROR)
77+
--source include/wait_for_slave_sql_error.inc
78+
79+
# Assert that only 1st transaction is applied.
80+
--let $assert_text= Only 1st transaction is applied on slave
81+
--let $assert_cond= "[SELECT @@GLOBAL.gtid_executed]" = "11111111-1111-1111-1111-111111111111:1"
82+
--source include/assert.inc
83+
84+
# Prepare GTID auto skip of offending transactions.
85+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:2';
86+
BEGIN;
87+
COMMIT;
88+
89+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:3';
90+
BEGIN;
91+
COMMIT;
92+
93+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:4';
94+
BEGIN;
95+
COMMIT;
96+
97+
SET GTID_NEXT='11111111-1111-1111-1111-111111111111:6';
98+
BEGIN;
99+
COMMIT;
100+
101+
SET GTID_NEXT=AUTOMATIC;
102+
103+
# Restart applier thread that shall sync without issues.
104+
--source include/start_slave_sql.inc
105+
--source include/sync_slave_sql_with_io.inc
106+
CALL mtr.add_suppression("The slave coordinator and worker threads are stopped");
107+
--source include/rpl_end.inc

‎sql/log_event.cc

+7
Original file line numberDiff line numberDiff line change
@@ -4867,6 +4867,13 @@ int Query_log_event::do_apply_event(Relay_log_info const *rli,
48674867
}
48684868

48694869
compare_errors:
4870+
/* Parser errors shall be ignored when (GTID) skipping statements */
4871+
if (thd->is_error() &&
4872+
thd->get_stmt_da()->mysql_errno() == ER_PARSE_ERROR &&
4873+
gtid_pre_statement_checks(thd) == GTID_STATEMENT_SKIP)
4874+
{
4875+
thd->get_stmt_da()->reset_diagnostics_area();
4876+
}
48704877
/*
48714878
In the slave thread, we may sometimes execute some DROP / * 40005
48724879
TEMPORARY * / TABLE that come from parts of binlogs (likely if we

0 commit comments

Comments
 (0)