preface

The most incomprehensible thing about the world is that it is comprehensible.

The most incomprehensible thing about the world is that it is comprehensible at all. — Albert Einstein

Open Source has created a thriving software industry. Open source allows developers around the world to work together to create great tool-like projects, known as “wheels,” that benefit individual companies large and small while also showcasing the technical prowess of the creator or contributor. Nowadays, many developers are using open source projects as third-party libraries or dependencies for their projects to accomplish development tasks faster and more efficiently.

This writer is no exception. When I was recently refactoring the Crawlab front end with Vue 3, I used an updated version of ElementUI developed by the Element team, the new UI framework refactoring from Vue 3, Element Plus. The Element team fully refactored the project in Vue 3 at Element Plus, embracing TypeScript in its entirety; Some components are richer than the previous Vue 2 version; The overall style and use of the previous version consistent; Some apis have also become more streamlined in use. Therefore, the author did not encounter too many obstacles in the initial process of reconstructing the Front-end of Crawlab. Combined with the previous writing experience, the author seemed to be very skilled in the development process. However, the good times did not last long. With the continuous development of the project, the author encountered some technical difficulties. More specifically, the implementation of some of the more complex features was limited by the Element Plus framework itself. I eventually managed to fix the problem, but I was also impressed by the difficulty of Hacking open source project source code. Therefore, I also hope to take this opportunity to share my experience in mastering open source code with readers.

This article will start with the experience of solving Element Plus problems, discuss the issues of open source projects or open source frameworks step by step, further discuss the methods and skills of harnessing the source code of open source projects, and share my thoughts on reading, understanding and changing the source code. This article is primarily a discussion of methodology, not too much technical detail, and should be read by readers of any professional background.

Hard on Element Plus

Introduction of framework

If you’re a front-end engineer who’s used Vue, you’ve probably heard of ElementUI. This is a UI framework, which means it is a toolclass framework that helps you build Web projects. It contains a lot of common components, layouts, themes, etc. Early ElementUI was written in Vue 2 and is the most popular UI framework in Vue, with 49K stars on Github, more than double the number two iView (24K). With Vue author Yu Yuxi’s release of Vue 3, announcing the full embrace of TS, the original Element team developed a new version of Vue 3, Element Plus, the protagonist of the story. If you don’t know anything about Vue 3 or even Vue, you can read the previous technical article on this blog, “HOW TS-enhanced Vue 3 Can Help You Easily build enterprise front-end Applications.”

Try a glimpse

Element Plus is as simple to use as in previous Vue 2 releases, and the documentation style is the same as in previous Vue 2 releases. It is very comprehensive. As encouraged by the official documentation, the author also tried to optimize the old Crawlab components, such as tables, in a componentized way. In addition to simple encapsulation of Column, Data, Pagination, etc., I also hope to add some new utility functions, such as Filter and Sort of custom table. Tables allow users to Customize the columns to be displayed and adjust the order of the columns, and so on. While the Element Plus framework has sorting and filtering capabilities, I personally find it too basic, and users generally need simple and user-friendly UI components.

The following figure is a screenshot of the front end of the screening table, which is similar to the display and operation of Excel.

The problem of implementing this function is not very big and can be solved according to the official documentation. It mainly uses the header Slot of the el-table-column in the column component and adds a Popover into it. So far So good! The next step is the customization of adding columns, which seems to be coming to a close.

Problem of the initial

The perfect way to customize a column is to have the user click on an action button, which brings up a Dialog that allows the user to select the columns to display, sort them by drag and drop, and then click OK. Then I quickly found a new component, Transfer, on Element Plus’s official website. The effect is shown in the picture below. In theory, I could control the array of displayed columns by referring to the popup in the dialog, such as putting the columns to be displayed in “Listing 2”. I! So far everything seems to be going very well.

However, after perusing the documentation, I found that the component does not support drag and drop, so column sorting is currently not possible. It’s a little frustrating, but it doesn’t matter. The main thing we want to implement is column selection. Sorting is also important, but let’s put it on hold until we implement component selection!

However, the reality is always unexpected. In further testing, I found that there seemed to be a major Bug with this component: when all candidate elements were selected in the shuttle box, it was impossible to select or deselect (move left or right)! Quickly debug carefully to find the cause, and repeatedly confirm whether I wrote my own code problems. Then it was found that oldChildren was null under the patchBlockChildren method in Runtime-core.esm-Builder.js when Vue updated the component. After further reading the source code for Element Plus’s shuttle box component, it was suspected that the transfer-panel.vue child had a problem updating the render without the el-checkbox-group, thus causing an update error. I then submitted the Bug Issue in Element Plus’s Github repository. However, the maintainer did not consider the problem I reported to be a Bug because there seemed to be no problem on CodePen. However, this Bug does exist objectively in my project and can be reproduced in the crawlab-frontend history commit.

Again meet trouble

But under, ponder by oneself along while fruitless after, decisive decision makes wheel oneself. Soon write a similar shuttle box from scratch components, components effect as shown below. The newly written component looks very similar to the previous one, but has a lot of freedom, so it also supports drag and drop. It looks as if the problem is about to be solved.

However, when I was fully prepared to test the drag-and-drop sort feature, something unexpected happened: after the column order data was changed, the table columns didn’t change at all! Changing the order of the column components el-table-column has no effect on the display! In the official document definition, the order of the el-table-column determines the actual display order of the column. After repeated experiments, the author had to admit that there were problems with the original implementation. Simply changing the order of the column components in the template does not affect the display on the interface.

The author thought of a “cong (Yu) Ming (Chun)” method. I tried to change the column order by forcing a refresh of the table component to re-render its data. However, after testing, it was found that such a violent way has a big performance problem, each time after clicking “Apply” will be delayed for nearly a second.

Thus, the author fell into a desperate situation.

The whole

In this dilemma, I stopped to think. Do I not know enough about Element Plus’s framework? How is the table component itself implemented? What are its limitations and disadvantages? These questions pushed me further to deconstruct the Element Plus framework itself by reading its source code to understand the code logic and how the components themselves work. So I cloned the Element Plus repository locally.

Fortunately, the code quality of the Element Plus project is very high, and the code structure and naming are very clear. Although there are relatively few comments, its clear logical structure and good naming conventions make it very readable. While marveling at the professional quality of engineers in Big factory, I also quickly located the source location of table components packages/table. The entire Element Plus project is managed in MonoRepo style, which is simply a Git repository with multiple NPM projects. Administration tools use Lerna. Below is the code organization structure of the NPM project for the EL-Table component.

. ├ ─ ─ __tests__ │ └ ─ ─ table. The spec. The ts ├ ─ ─ but ts ├ ─ ─ package. The json └ ─ ─ the SRC ├ ─ ─ config. Ts ├ ─ ─ the filter - panel. Vue ├ ─ ─ ├── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── ── Tree. Ts │ └ ─ ─ watcher. Ts ├ ─ ─ table │ ├ ─ ─ style - helper. Ts │ └ ─ ─ utils - helper. Ts ├ ─ ─ the table - body │ ├ ─ ─ events - helper. Ts │ ├ ─ ─ index. Ts │ ├ ─ ─ render - helper. Ts │ ├ ─ ─ styles - helper. Ts │ └ ─ ─ the table - body. Which s ├ ─ ─ the table - column │ ├ ─ ─ but ts │ ├ ─ ─ Render - helper. Ts │ └ ─ ─ watcher - helper. Ts ├ ─ ─ the table - footer │ ├ ─ ─ but ts │ ├ ─ ─ mapState - helper. Ts │ └ ─ ─ style - helper. Ts ├ ─ ─ the table - the header │ ├ ─ ─ the event - helper. Ts │ ├ ─ ─ but ts │ ├ ─ ─ style.css. Helper. Ts │ ├ ─ ─ the table - the header. Which s │ └ ─ ─ Utils - helper. Ts ├ ─ ─ the table - layout. Ts ├ ─ ─ table. The ts ├ ─ ─ table. The vue ├ ─ ─ the tableColumn. Ts └ ─ ─ util. TsCopy the code

From this structure, we can see that the entire el-Table component consists of several sub-components, such as table-body, table-column, table-footer, and so on. The whole project has only one component, table.vue. Based on my experience with Vue development, I quickly realized that this was the entry point to the entire component. Let’s have a look first!

The entire table.vue file has 469 lines of code, which is quite a large file, but I won’t explain it here for space reasons. The most important discovery was a variable called store. A closer look reveals that this is a state manager created by Vuex. Boy! The table component uses Vuex to manage data. That’s clear. Once we get the Vuex part out of the way, we should be able to solve the rest of the problem.

Don’t say a word, plate it!

Problem solving

I kept trying to find column-related code and started searching globally for “columns” and similar keywords. After using the insights of detective Conan, I came to notice the useStore method in the SRC /store/index.ts file, which is the key to the whole problem! The key is the internal state variable store.states._columns, which is the key to rendering column data. But it is only assigned when the column is initialized or added or deleted. Adjusting the order of the el-table-column does not change this variable at all!

Find the cause and the solution is simple. I added the setColumns Mutation method to make it easier to set up the column array. The specific implementation process is limited to space reasons do not go into detail. Interested friends can see Crawlab Frontend source code.

Below is a rendering of the complete table’s custom columns. And you’re done!

The pros and cons of open source frameworks

Nothing is perfect, and a good open source framework is no exception. When I was Hacking source code, I realized this. I think it’s important to discuss issues related to using and developing open source frameworks or projects. First let’s look at its advantages.

advantage

Many well-known open source projects, such as Nginx and Redis, have become core technologies for software development. So why use a good open source framework? The author believes that open source frameworks have the following advantages:

  1. Free of charge. Who can resist something for free, and many free open source frameworks are already good enough;
  2. Transparent. All source code for open source frameworks is available for anyone to see;
  3. Can be changed. Most open source projects are MIT or BSD open source with a lot of freedom and can be customized;
  4. But the collaboration. Github is the largest open source project platform where developers around the world can participate in iterative open source projects.
  5. Influence. A good open source project can increase the visibility and influence of an author or contributor within the industry.

disadvantage

Although open source projects have many outstanding advantages, they often encounter various problems when using open source projects. I think there are risks in using open source frameworks. The author believes that open source frameworks have the following disadvantages:

  1. Security risks. Although many excellent open source projects are developed and maintained by enterprises or senior experts, contributors tend to neglect security because they are not completely used by themselves. There are numerous examples of security vulnerabilities in well-known open source projects. Examples include OpenSSL Heartbleed, Fastjson Remote code bug, Antd Christmas Eggs, etc.
  2. The good and the bad are mixed. Open source project developers, contributors, and maintainers can be anyone with different experiences and professional backgrounds, which inevitably leads to some differences in the quality of code or open source projects. While Coding standards can circumvent some of these problems, there are few good projects. Look at the NPM or Maven common repositories that host millions of projects.
  3. Cost of learning. I acknowledge that some of the best open source frameworks have mature and well-documented systems, but most lack effective documentation and tutorials. Even with detailed documentation, it takes a lot of time to read and learn; Most paid products include professional support, which can save developers time.
  4. Persistent problem. Good programmers can develop very high quality open source projects, but because open source projects themselves don’t generate cash, many authors are reluctant to invest in them for a long time, resulting in a lack of continuous maintenance and iteration.
  5. Unknown risks. Again good frameworks risks, due to the open source framework didn’t experience too many actual business early testing, many problems don’t get timely repair, therefore in the process of using open source framework more or less will encounter some unexpected problems, solve them will spend a lot of time, even some problems to solve.

The problem’s analyse

The unknown risks and learning costs are particularly salient in my previous experience with open source frameworks. When I adopted Element Plus’s Transfer component, I took it for granted that it could effectively meet my needs. However, it didn’t support the column sorting requirements I planned, and it had a major Bug that made it unusable. These issues led me to abandon the component and look for a costly solution to rebuild the wheel.

Later, in the process of hard eating Element Plus table component, I did not give up using the component, but by reading the source code of the component, I determined the location of the core code and the possible cause of the problem according to logic and experience, and made targeted adjustments, and finally solved the problem smoothly. The decision to read the source code and tweak it saved me a lot of time and effort, because it wasn’t cost-effective to duplicate a complex wheel like the table component.

Above is a screenshot of one of the components in the official documentation. In fact, Element Plus documentation is very complete, with detailed information on how to use components, examples, apis, etc. Whenever you encounter uncertainty in development, you can go directly to the official documentation. However, ElementUI was originally created by Ele. me, so its components are primarily designed to support ele. me’s delivery business, so it has some limitations. An updated version of ElementUI, Element Plus is the product of the same team, but its functionality and APIS have not changed much from the older version. In contrast, Ant Design, the UI framework created by Ant Financial, is relatively flexible and free, because it supports a wide range of business lines, while the Medium platform system promoted by Ali also requires the UI framework to be designed in a general way. Thus, Element Plus’s limitations most likely stem from its development team background. The Crawlab front-end framework chose Element Plus mainly because the older version was ElementUI, which was relatively cheap to migrate, but it also introduced some problems due to the immature framework. In fact, it’s important to understand that even good open source projects have problems. In this regard, it is important to understand the strengths and weaknesses of the open source framework itself for proper use. In order for open source frameworks to work for us, we need to know how to properly harness the source code of open source projects.

Next, I will combine my own project experience to show readers how to effectively harness the source code in open source projects.

Source Code Manipulation Tips

If an ancient man had traveled back in time to the present, he would have marveled at the various technological products. What they did not know, however, was that these so-called “magic” were created using science and technology, and the principles behind them were concluded through scientific experiments, not some mysterious witchcraft. The same applies to software.

Programmers use code to create magical Internet societies that are mysterious to non-technical people. They know it’s not real magic, but they have no idea how it works or what the principles are behind it. Good open source projects are a quick way for programmers to prototype, validate, and even develop production-ready systems. However, if programmers just use open source products as tools without any understanding of how they work, it’s like using magic, and you don’t know why it works. This will get you into trouble. To use an open source framework properly, you must master it; To master it, you must understand how it works; To understand how it works, you have to read the source code; To make source code do what you want, you must learn how to harness it, that is, how to modify and optimize source code.

Here are some useful ways to help you navigate source code.

Overcome frustration

Readers may be surprised by this seemingly useless advice. However, I have seen many friends, including myself many years ago, give up when they want to learn an open source project because it is too complicated. Part of it is frustration. I admit that it is unrealistic to understand a large open source project in the short term. As a result, many friends are scared away by the numerous technical difficulties, leaving a sentence “as long as you know how to use it”, and then continue to move bricks and turn screws. In fact, they underestimate their own potential. I do not object to programmer pragmatism, but it is important to note that too much pragmatism leads to utilitarianism, and you end up standing still.

One of the best things about open source frameworks is that they are completely transparent, so anyone can see the code, including the underlying implementation logic, code organization, project deployment, and so on. Isn’t this a great opportunity to improve your mastery of source code? By reading the source code of the open source framework, you can not only understand how it works, but also learn better ways to program and improve your professional skills. I’m not really advocating the “get started and drop the series” joke, you should at least suggest to yourself, “I don’t understand this because I don’t have the basics. I’ll be able to handle it when I improve the basics.” After gnawing through some complex open source framework source code, I found that reading the source code is actually not that difficult. It’s like exercise. After you get through this hill, this sprint, it’s very easy, and your physical fitness has improved.

Overcoming frustration is therefore very important in navigating the source code of an open source project, and it is important to suggest to yourself that reading the source code is not difficult.

Write your own open source projects

One of the reasons people don’t read source code is because they don’t know it. It’s like when you don’t know someone you like, you think she’s a perfect goddess. But when you really pursue her, when you go out with her and talk about your life, you will find that she is just an ordinary girl. Therefore, you can try to get your hands dirty and start writing an open source project yourself. This will help you understand the thinking of the authors of open source projects. Again, this isn’t a practical tip of any kind, but if you actually start doing it, you’ll improve quickly. Learn by doing!

Of course, it’s not easy to start an open source project. You might get stuck in what to write about in the first place. Here, I recommend the two articles I wrote before about the development of open source projects “How to build a Github Project with thousands of stars” and “Harvest the first 5K Star open source Project in my life, and share the experience and lessons with you”. It explains in detail the author’s experience in developing, maintaining and promoting Crawlab, an open source crawler management platform. It also introduces how to locate pain points, investigate users, promote products and manage projects, etc. Interested readers can read further.

Location entry file

Finally, this is a bit of dry advice. To read the source code, you must first find the Entry File for the project. The so-called entry file is a public channel that the project module or system exposes to the external system. The operation, call and execution of the whole project start from this file. Other code that implements specific logic is referenced in the entry file or in other files. For front-end projects, this is typically main.ts; Python projects are typically app.py; Golang projects are usually main.go, etc. If you compare large, seemingly unmanageable open source projects to Achilles, then entry files are his Achilles heel.

In fact, not only when we read open source projects, but also when we read code written by other colleagues at work, we start with entry files. Once you know the internal logic of the entry file, you can see the other submodules it references, and then you can follow that lead to the other core modules and move on. Once the entire project has been walked through the entry file, you will be able to understand the logical structure of the code more easily, which will help you to understand how it works.

In the previous example of solving the Element Plus table problem, the author located the entry file table.vue, then further found the key location of the column data, and finally solved the problem easily. Look! Harnessing the source code isn’t really that hard, is it?

Use global search

Again, this is a technical suggestion. Since large projects are usually split into many files, it can take a lot of time to exhaustively traverse all the files in this case. Therefore, you can try to search globally for possible keywords to quickly locate the core code. Global search is very powerful in mainstream ides, often supporting regular expressions, fuzzy matching, exact matching, case, and so on. Below is a screenshot of global search results from JetBrains’ WebStorm IDE.

For statically typed languages like TS, it is also possible to Find the invocation position of a method or variable as Java, C#, or Golang do. In WebStorms, it is “Find lead “. The name may be different in other ides, but they all mean the same thing. These techniques can help you read and understand the source code.

Read technical documentation

Good documentation includes the architecture, principles, concepts, etc. of the project to help developers quickly understand the code structure of the framework. However, not all open source frameworks are well documented, and this approach is rare. However, if technical documentation is included in an open source project, be sure to take the time to review it.

Of course, you can also try to contact the open source project author. NPM project information usually includes the author’s email address; And if you’re looking at an open source project on Github, you can make an Issue of it.

conclusion

This article discusses the advantages and disadvantages of open source projects and concludes that there are also problems with good open source frameworks by introducing my experience in the development of Element Plus, the open source UI framework. Then, based on my project experience, I propose five tips and methods for navigating open source projects, including overcoming frustration, writing your own open source projects, locating entry files, using global search, and reading technical documentation. Today’s software development can’t be done without good open source projects, but it’s important to realize that open source projects are not for you and therefore have limitations. In order to make the best use of an open source framework and make it compatible with your complex needs, in many cases you have to read its source code carefully, which incurs a certain learning cost.

We often say that programmers need to improve their technical skills and “write” good code; However, given the growing importance of teamwork, I believe that a more important skill for programmers is to be able to “read” other people’s code, good or bad. In the process of writing code to ensure the correct and reasonable naming conventions, as far as possible to add comments to facilitate others to understand, this is the ability to write code; When working with other people, the ability to read code carefully, to try to understand what they’re thinking, to think about whether they’re implementing it the right way, etc., is the ability to read code. Of course, writing and reading is only a very basic part, the ability to control code is not only “writing” and “reading”, should also include thinking, logic, planning and other skills, such as system architecture, algorithm principle, scalability design and so on.

So don’t get upset or angry when you complain about your colleague’s bad code. It’s probably because you don’t read code well enough to understand what your colleague is thinking. Similarly, our code might look awful to our colleagues; Or is it the pot calling the kettle black?

community

If you are interested in the author’s article, you can add the author’s wechat tikazyQ1 and mark “code of Tao”, and the author will pull you into the “code of Tao” communication group.