TiDB Hackathon 2020 TiDB Hackathon 2020 TiDB Hackathon 2020 TiDB Hackathon 2020 TiDB Hackathon 2020 This year is the fourth time that TiDB Hackathon has been held. The scale of participating teams is the largest ever, with a total of 45 teams from all over the world signing up, realizing global linkage for the first time. After 2 days of extreme challenge, many exciting projects emerged in the competition. In order to let more friends know the stories behind these participating teams, we will start the TiDB Hackathon 2020 Excellent Project sharing series. This article will introduce the wonderful stories behind the competition of B.A.D team.

Community developers involved in the development of TiDB need a relatively long period of early learning, in addition to the complex programming language, another biggest difficulty mainly comes from the complexity of the project itself. As a distributed project with multi-language and multi-runtime components, the development and debugging threshold of TiDB is much higher than some conventional software projects like mobile App, Web server and game client.

TiDE is a TiDB plugin based on the most popular Code editor today, Visual Studio Code, which can develop and debug TiDB clusters locally/remotely. It makes it easy for developers to start test clusters locally and debug authoring in real time without having to understand the internal mechanics of TiDB, greatly improving the experience of developers developing and debugging TiDB projects. ** This project won the second prize and CTO Special Award in this TiDB Hackathon 2020.

Project background: Everything started from an idea of CTO

One week before the competition, Huang Dongxu, CTO of our company, suddenly said in the group that he had a great idea. This idea originated from a few weeks ago. Dongxu originally wanted to add functions to TiDB, but the code of TiDB was too complicated. So he wanted to create a VSCode plugin, which is kind of like scaffolding. When developers want to add a feature, they can help them to generate template code directly in TiDB/TiKV/PD components with one click, just need to fill in the logic code.

Li Yilong, one of the team members, was inspired by this idea. He found that in daily development of TiDB/TiKV/PD, the developer had to manually replace the binary on the server and modify the component configuration, and then manually retrieve the log to see the result after the restart. The process was very tedious. It is also complicated to step through local debugging of major components simultaneously. He wondered if he could automate all of this using VSCode plug-ins. A k8s-like VSCode plug-in that allows you to manage a K8S cluster without a line of code. So r & D engineer Li Yilong, front-end engineer Huang Baoling and K8S expert Wu Yelei hit it off, all think this idea is too good, hit the pain point!

Through the discussion, the three people roughly determined the direction of the project, to develop a one-stop IDE for TiDB developers, to improve the efficiency of the artifact!

Project design: It’s so rewarding to do a project that benefits all TiDB developers

At the beginning of the project development, it was completely ecological development mode. Everyone realized what they were good at and most wanted TODO while recording their ideas as TODO. The initial division of labor is to be responsible for the implementation of scaffolding and cluster management functions; Lei is responsible for the implementation of debugging function; As the team leader, Xiao Long was responsible for the realization of the product and some functions of K8S. At the same time, he was also the inspiration of the team. He not only output project inspiration, but also named the team B.A.D according to the initials of the three people on GitHub.

After a few days of tension and enrichment of the hair, TiDE finally began to see the prototype, it mainly achieved the following several important functions:

Scaffolding function

Add a new function to one or several components of the TiDB ecosystem with one click. Currently, we have the ability to add an APP to the TiDB Dashboard with one click. After the addition, users only need to modify the default implementation to quickly add their desired functions to the TiDB ecosystem.

Add a new page for TiDB Dashboard and preview it live

Local development

TiDB ecosystem is composed of multiple projects, many functions need to be developed and debugged across multiple projects. TiDE is able to compile and package the code of each component in the working directory and deploy or update it to a local cluster with one click. It also provides the ability to view logs, topology management, configuration management and make local development silk-smooth.

Start a local cluster from code in the working directory

Remote development

TiDB as a distributed system, remote development in distributed environment is also an essential capability in many scenarios. TiDE provides a range of capabilities for remote clustering, including starting and stopping clusters, modifying configurations, replacing locally compiled binaries to remote nodes, viewing logs, and one-click SSH to remote nodes.

Operating a Remote Cluster

Manage remote cluster configuration

Debug the TiDB cluster

This includes debugging local and remote clusters. Following the same idea that TiDB typically runs in a cluster, TiDE enables single-step breakpoint debugging for all components in a cluster with one click, as well as for individual components or processes.

Single-step breakpoint debugging the local cluster

“TiDE really works, you know it”

In the process of project development, the strategy of promotion at the same time was adopted. Team members also expressed that one of the intentions of participating in the competition was to promote TiDE, so as to get more feedbacks from seed users, and then optimize the function. During this process, several unexpected interesting things happened. At the Hackathon competition site, a developer of the friendly team encountered difficulties in using TiUP. With the enthusiastic help of Xiaolong, the developer solved the permission problem of TiUP deployment by using TiDE, and accidentally got a user on the spot. “TiDE improves my development efficiency and frees up my mind. It’s easy to use and I know all about it.”

Several members even revealed the plot of the first Demo to several colleagues before the defense of the project. Unexpectedly, everyone was shocked and shouted “How cool is this?” “And immediately started asking team members,” Hey, could this reboot support just restarting a single instance or a single component at a time?” “Hey, what about Bench?” Well, it makes sense, you can have it, add it to TODO! The three students of B.A.D were very happy to see the recognition, because they saw that their project was really valuable. It was a great sense of achievement to make a project that benefited all TiDB developers!

Project significance: achievements in the present, benefits in the future

“We are here to help, but we will continue to benefit,” is how Huang describes TiDE, especially for TiDB, an open source project with a large and active community. It can be reflected not only in a certain product, but also in all future tiDB-related products, which has great social value. The significance of the specific project can be summarized as follows:

Lower the threshold of TiDB development and debugging, so it is no longer difficult to get on TiDB. TiDB code is very complex, so that many people who want to learn or join the open source community are discouraged, but with TiDE, the magic device, greatly reduce the threshold of TiDB development, so that more people can quickly use TiDB, into the community. TiDE, he said, is expected to replace the traditional “documentation” model of open source projects, where people learn about projects by putting examples and documents on the front page. Instead, TiDE allows users to get started quickly and turn dead documents into interactive displays. Quickly and intuitively let users understand the various functions of TiDB, so that they are interested in joining the community.

Improve development efficiency and user experience, truly enjoy the process and results of programming. With TiDE, we don’t have to spend 90% of our time building environments anymore, and we allow developers to get back to their roots and enjoy writing code and the results of programming. Alei agrees that the biggest significance of TiDE is getting more people to focus on improving the developer experience. “In the tech community, groups dedicated to improving the developer experience tend to be less critical and less sexy, and even I used to have that stereotype myself. But TiDE struck me as an example of how big the space is and how much fun it can be to improve the developer experience — and I think that’s something everyone who’s used TiDE can relate to.”

Future direction: Just a Beginning

Although this session of Hackathon has come to an end, it is just the beginning for TiDE. The TODO LIST is still very long, and the future can be roughly divided into the following development and optimization directions:

  • Cluster deployment: One-click deployment of various types of clusters for development testing

  • Machine management: Manages physical machines or VMS used to deploy clusters

,

  • TiUP Automatic installation: Automatically installs and upgrades TiUP

  • Optimized TiDB development experience on K8S: One-click update of native code to K8S cluster and debugging, making development process on K8S as simple and easy as it is locally

  • Interworking UDF: In conjunction with the WASM UDF feature, which won the first prize in Hackathon, it provides scaffolding and debugging functions to easily develop TiDB UDF and make customizing TiDB functions a breeze

  • Enter TiDB Incubator Program: This versatile tool project certainly hopes that all TiDB community partners will participate in this project. We know our pain points best

TiDE ended the Hackathon trip with the second prize and CTO Special Award. When talking about the feelings of participating in the Hackathon, the three members also said that they had gained a lot:

Huang Baoling: “I am very happy to have a Hackathon, especially our project has been recognized by everyone, helping everyone to solve the pain point. It opened my eyes. I felt like everyone was too strong. It was a special surprise for us to win both awards.”

Wu Ye Lei: “Excited, shocked and satisfied. I was excited by the recognition and praise my group’s products received, shocked by all kinds of bold and ingenious ideas and Demo shows, and satisfied with two days of profound memories and a group of new friends (and of course, two awards!). .”

Li Yilong: “the harvest is full of idea and everyone’s enthusiasm, is really immortal fight. The depth and breadth of technology demonstrated in the competition is amazing, and I think your Company deserves to be called a Hackathon Driven Company. This tradition must continue, after all, there are few Hackathon competitions that are so hardcore in the Chinese developer community. We all enjoyed the race very much and felt it was more of a party than a game. I met a lot of interesting people, and that was enough.”

Finally, github.com/dragonly/ti… “Has already received 46 stars on GitHub and nearly 40 downloads on the VSCode store.

TiDE is just beginning, the future is still very long, let us look forward to together