“Live up to the time, the creation of non-stop, this article is participating in 2021 year-end summary essay competition”
Today is December 13, 26 years old. It has been five years since 2017, when I briefly summarized my experiences of the year.
2017 Year-end Review 2018 Year-end Review 2019 Year-end Review 2020 Year-end review
Looking back on the past five years, from junior year to graduation this year, I seem to have done a lot of things and nothing at all. Also mixed to master’s graduation, lucky hair ACL, and then lucky into byte engaged in a new direction of research. The only thing that hasn’t changed over the past five years is that I’ve been playing league of Legends, still the same guys I played with, still familiar.
2021 is the year to officially become a social person, from an old guard on campus to a new one in the workplace. To sum up, the first half of the year was busy with graduation, and the second half was busy developing LightSeq.
This year, I spent my first Spring Festival away from home in Shanghai. First, I had to go back to Shanghai for nucleic acid treatment due to the epidemic. Second, I had to hand in my graduation thesis soon. Even make up with gather together, spent nearly a month to drive out a decent graduation thesis. The graduation process went unexpectedly fast, and the tutor did not modify it much. The defense was finished in just a few weeks, as if the school expected us to leave school and enter the society as soon as possible. After graduating with honors, thesis and national scholarship, the lab had a dinner party and then graduated.
Seven years of normal university, youth never graduate
That’s all I have to do in school. I’m still focused on the company. In the first half of this year, I further improved LightSeq inference engine, wrote a lot of articles to introduce LightSeq and promote LightSeq, and popularized the basic knowledge for many students who want to get started with CUDA and model acceleration.
After a few all-nighters, I wrote a beginner’s code for CUDA
With just two lines of code, I made Transformer reasoning 10 times faster
Also published a paper about LightSeq inference engine related technology on NAACL 2021, carried out the technology precipitation, inference engine development is also come to an end.
LightSeq: A High Performance Inference Library for Transformers
The most important work of this year was LightSeq training engine. From April to June, I developed the industry’s first Transformer full-process training acceleration engine with my colleagues for three months. I participated in the whole process of training engine from 0 to 1. During this period, I had to learn CUDA and model acceleration by myself in order to quickly integrate into the development of the project, which I did not have access to before in school. After the project was open source, I also submitted a paper. Although I did not win the project, I also accumulated a lot of experience and understood the writing method of some conference papers in the field of system. Leader also arranged me to go to QCon to share and publicized a wave of LightSeq technology.
LightSeq source
Training speed up 3 times! Bytedance launches the industry’s first full process acceleration engine for NLP models
LightSeq2: Accelerated Training for Transformer- Based Models on GPUs
At present, LightSeq quantitative reasoning is being developed, and we will meet formally in the near future, and the speed of model reasoning will also step up to a new level.
In terms of life, I started cooking again in the second half of the year. On the one hand, take-out food is not sanitary, and on the other hand, I am tired of eating it. However, the kitchen of the rented house is really too small. It takes a long time to clean up every meal, and often I don’t want to do it. Limited to space, here only a few pictures, more can go to my moments to watch.
Although the house is small, at the end of October, we still have a new member of the family, I took back a puppet cat from my elder brother’s home. Having a cat has added a lot of fun to my life. I can masturbate when I’m free. But also brought a lot of trouble, the cat often disorderly pull, sleep at night noisy, every day to shovel excrement feed food, and take a child. Maybe I gave him too good food, only about 6 months has grown to 10 catties.
The emotional side remains the same. There have been hiccups, but none of them are appropriate.
The biggest feeling for me this year is that after working, time gets faster and life gets monotonous. Especially after the abolition of the week, every week goes by so fast that it will be the next weekend before I have lapsed. This may also have something to do with my indooriness. I play video games every weekend. They still sleep late, and they keep destroying their bodies.
For 2022, my biggest wish is to do a good job and improve the model quantification first, so as to strive for promotion as soon as possible. I hope LightSeq’s STAR can reach 3000 as soon as possible. Then continue to cook, raise good cats, happy games every day. The article updated recently has not been frequent before, must be diligent a little bit.