The original address: blog. Jcole. Us / 2013/01/02 /… MySQL source code related to some comments and interpretation of ideas, as well as with the picture
I’ve been using InnoDB for about a decade, and I know it well enough to do most of the things I want to do with it. However, in order to implement some of the more efficient solutions, I found that I needed to have a deeper understanding of its principles and ideas. Unfortunately, InnoDB documentation lacks a clear and unambiguous explanation of InnoDB’s internal data structures, and reading the source code is the only way to get a deeper understanding of how it works.
However, in the process of reading the source code, I soon found that the internal structure and connection of the code were very complex, which made it difficult for me to understand the connection just by reading the code. Hopefully, if you’re reading this article, you’ll be able to figure this out by just reading the code (personally, I’ve had a lot of misconceptions along the way).
One approach I’ve taken for a long time to understand something complex and poorly documented involves the following three steps:
- Read existing documentation and existing code until basic understanding is reached. During this step, serious misunderstandings or incorrect disassembly often occur.
- Write my own implementation, even a very basic and rudimentary one. It’s better to write in a completely different language (thus avoiding the lazy tendency to cut and paste) to change the code based on what works and what doesn’t, and to correct my understanding.
- Summarize new documents and diagrams based on my new understanding. Refactoring my implementation as needed (reviewing the code while summarizing the documentation often reveals problems with the implementation at the time and optimizes the original design). Correct documentation based on refactored code. Repeat this process until it is correct.
InnoDB disk data structure implementation
I started the Innodb_Ruby project to implement InnoDB disk data structures in Ruby. I chose Ruby because it’s flexible, fast for prototyping, and it’s my favorite language by far. Any language is fine, and performance is not an issue (although we don’t want it to be, since it makes testing annoying)
After the project started, I implemented a very basic FIL header resolution within a few minutes (this part is the same for all InnoDB page types). It then took a few more hours to implement the INDEX page header and be able to answer some very basic questions, such as how many records are in each INDEX page, which is useful.
I went on to implement each of the key data structures I needed in the order I wanted to learn more about InnoDB’s storage, and Davi was involved in writing some of the details, such as dealing with variable width field types in records.
We have now basically implemented the read-only implementation of InnoDB’s main data structures.
Record the InnoDB disk data structure
Once I had solved enough of InnoDB’s secrets, I felt I could start drawing more accurate pictures to better illustrate InnoDB’s storage principles, so I began to build easy-to-understand diagrams of all the major InnoDB disk data structures. I started the project Innodb_Diagrams and used the mapping software OmniGraffle. (555~ Can you take care of those of us who can’t afford a MacBook?)
This is because most disk storage formats (ibdataX and *.ibd files) for table space files in documents are based on Barracuda row format storage (COMPACT row format records). Most documentation needs to be added for the Antelope line format (REDUNDANT line format records). The log file is also currently required for supplemental documentation.
Use code and diagrams
So far we have the code to implement the interactive presentation, as well as pictures that can be a good source of support, and I plan to write a few more articles on some of the more interesting but undocumented ones. Please stay tuned ~~