Making the transition from finance to big data has been a rewarding journey.

The road to big data advancement is very long, but please rest assured that it is very easy to transfer Java to big data, and it is also very easy to learn big data from zero foundation. I will accompany you to do it together, and we will be done.

The article is a little long, it is all my true feelings. It is divided into seven parts: beginning, turning point, growth, New Flag, about this number, recommendation and summary.

I majored in software engineering in college and will graduate in 2018. According to statistics, the major with the highest average salary among recent graduates is software engineering! Fortunately, I did not lower the average level and went to a financial factory (Hangzhou) for internship with my own efforts and luck after graduation.

The department I work in is the most profitable — asset management. Our system covers all financial services except insurance. After completing normal tasks every day, I also learned a lot of financial business. I advise you not to enter the market if you do not understand the stock. In this business, the water is too deep. I won’t talk about the specifics.

Although the salary of the financial industry is good, but limited to old-fashioned technology, and even use Delphi. After doing for a period of time, I found that I was not suitable for myself, so I had the idea of leaving. Last year the economic situation is not good, so also dare not easily naked resignation. The mood at this point is: unwilling to go on like this.

At that time is not sure after what direction to go, see my public number of renaming records will know, but the brain is really in a mess!

On March 5, 2018, I set a Flag for myself. The following is the target of half a year in the knowledge planet of code farming. Morale is high.

As it turns out, four months of spare time to learn big data is not enough, stick to it, the time is more than expected. Hadoop ecology and Spark ecology as well as data warehousing, data modeling, machine learning and more. Some materials come from the Internet, others from educational institutions, and the rest can be found on official websites. The study notes are all written on the Language sparrow platform, which will be slowly transferred to the public account.

After reading large Web Architecture and Deep Understanding of the Java Virtual Machine, ILLUSTRATED HTTP was delayed. I also took my girlfriend to the graduation trip, but instead of Sichuan, I went to Beijing, Tianjin, Jinan and Qingdao.

For books on big data, I read Offline and Real-time Big Data Development, Data Warehouse Structure Design and Implementation, Spark Technology Insider, recommendation System Practice. Pick the key part to see, combined with their own learning technology for digestion.

What I listened to most during this process was Beyond. If you can’t hold on, turn on the music and listen to it for a while. Day and night, I have doubted myself.

Brief introduction of big data technology

What is a data warehouse? Talk about my understanding

PageRank algorithm, the key technology of search engine

Sqoop is not fully operational

Flume principle, analysis, architecture

Kafka introduction, architecture, installation

Kafka meets Spark Streaming

Data loss and repeated consumption in Kafka

HBase Architecture Analysis

HBase RowKey design

HBase data model, architecture, and component function description

Hbase table design in microblog, part

Zookeeper recovery mode, broadcast mode, and election process

Hadoop HA in-depth anatomy

Spark Tuning Integration – Summary (Long article)

Spark’s data localization, providing optimal compute nodes, is finally getting started

The Spark tuning glance | shuffle tuning

Shuffle file addressing process of Spark

From data collection, cluster analysis data, BI presentation

Flink dry incoming | Flink Forward China 2018 assembly data compilation

Here are 20 articles on big data technology written in 2018.

I have set myself several goals for 2019, one of which is the number and frequency of updates. Each article is as short and useful as possible. It is a long way to go to make big data clear to everyone.

The blog, which currently focuses on “big data” technology, also posts a weekly article about tools to improve efficiency. For new readers, click on the “Big Data” menu and the “Recommendation of the Week” menu below to see the corresponding series of articles. You can also click “Hide levels” to find out more, you know.

In 2018, the turning point. In 2019, transformation.

Three principles: self-motivation, self-awareness, and self-drive.

Join the right group, such as “Code farmer Turn Over” and “Shuai Zhang and his friends”.

Leave your comfort zone and you can’t avoid pain.

I have been in Shanghai for a week and interviewed two companies. One of them offered an annual salary of 23W, which is less than the previous financial position for me as a fresh graduate, but it is acceptable. I quite like that company. There is more room for development. The data platform should be created from scratch, and the department should be integrated into the big data department in Taiwan.

I have not been hired yet, and I plan to continue interviewing.

Future plans, “adhere to”, “attentively” to complete each article, can help everyone the best.