Writing in the front
MySQL is one of the most used relational databases in the Internet industry, and MySQL is open source. In-depth study of MySQL can deepen our understanding of the principle of database. Mykit-data can’t parse MySQL8’s binlog correctly. Mykit-data can parse mysql5.x’s binlog without any problems, and can parse the result data correctly. However, parsing mysql8.x’s binlog always fails due to a 12-bit discrepancy with the binlog number.
The article has been included:
Github.com/sunshinelyz…
Gitee.com/binghe001/t…
The problem to repair
It is too late today, I am still studying the source code of MySQL 8.0.20, and I will write a detailed article later to share with my friends. Here, I’ll go straight to how I solved the problem.
X/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x/mysql8.x
, I looked through the MySQL almost all the official documents, the analytical binlog mykit – data on the function of the log to write again, analytical MySQL5. X is no problem, analytical MySQL8. X or dislocation.
What went wrong? X binlog: mysql8.x binlog: mysql8.x binlog Then I will read out the extra bits of binlog data, directly ignore, so that the subsequent binlog parsing operations aligned.
To give it a try, I added the following code to myKit-Data framework source code.
The above code is a wrapper for the checksum reading of MySQL’s binlog bits. When the number of binlog bits read is not the limit of the number of digits read, the data is read until the number of binlog bits read is the limit of the number of digits read. For specific internal logic, you can read myKit-data source code.
After adding this logic to the MySQL 8.x database, verify the test and parse the binlog successfully!! The problem that troubled me for a few days is so inadvertently solved!!
X = mysql8.x; x = mysql5.x; x = mysql8.x; x = mysql5.x; The goal is to align the parsing bits of the binlog so that the next event is properly parsed. This processing does not affect the parsing results.
A lot of times is like this, when you are struggling to solve a problem, unable to find a solution and at a loss, in a casual moment, will inadvertently solve this tricky problem, but the premise is that you need to deeply understand its principle and try all kinds of ways and methods to solve it!
About mykit – data
Mykit-data is a fully open source data heterogeneity middleware, supporting plug-in, visual data heterogeneity framework, support MySQL to MySQL, MySQL to Oracle, Oracle to MySQL, Oracle to Oracle full, real-time/timed incremental data synchronization. Full plug-in, visual operation. Logs are used to minimize data loss during synchronization. Retry failures, manual intervention, and view synchronized data and detailed log information.
Mysql5.x, mysql8.x, Oracle 11g and above are currently supported. More heterogeneous data sources will be supported in the form of plug-ins.
The open source address of mykit-data is as follows:
GitHub:github.com/sunshinelyz…
Gitee:gitee.com/binghe001/m…
Finally, everyone click a Star for this open source project!!
Ok, that’s all for today, I’m Glacier, if you have any questions, you can leave a comment below, you can also add my wechat: sun_shine_LYz, exchange technology together, advance together, together awesome ~~