Big data has been popular for several years, but this year it seems to enter the era of big data for all people. In line with scientific research, I began to study big data by myself at the beginning of 2017. After systematic and comprehensive learning, I job-hopped to my current company this month.
Now the project data has been 50T and the average daily data has increased by 20G, probably because of the demand problem. The salary has doubled compared to the previous one (this is the key point).
Of course, my share is not to let you promotion, salary increase job-hopping, but for technical people, skills do not pressure body, the following personal experience to share, do not like spray.
ZHUANG now I’m going to start (B) :
I will mainly talk about the following points:
1. Career planning direction of big data
Second, big data development engineers need to get skills
Three, how to learn systematically
Iv. Data sharing (video, e-book, interview materials, project source code)
1. Career planning direction of big data
The purpose of the study is to apply it to work. The following are the three positions in the direction of big data, and the salary package is attached.
1. Big data pre-sales engineer, Big data operation and maintenance Engineer (HCNA) 10-15K
2. Big data Development Engineer, Big Data Platform Development Engineer (HCNP) 15K-20K
3. Big Data Scientist, Big Data Architect, Senior Analyst of Big Data (HCIE) 25K-50K
4. Others (Data mining engineer, warehouse acquisition and development engineer) 12K-25K
Second, big data development engineers need to get skills
Attached is an authoritative skill chart of big data engineers.
The summary is as follows:
10 required skills:
01.Linux Basic Operations
02. Introduction to Python & crawlers
03. Hadoop distributed centralized
04. Offline data processing
05. Real-time data processing
06.ETL is related to environment
07. Big Data Application and Data Mining
08. Java Advanced (Virtual Machines, Concurrent)
09.. HBase (JavaAPI operation +Phoenix)
10. Kafka
11. Spark (Core+ SparkSQL +Spark Streaming)
12. CDH cluster
This chart is probably a more comprehensive picture of skill mastery.
Three, how to learn systematically
I began to study by myself at the beginning of 2017, and I learned the basic operation of Linux by myself. It is easy to learn, and I can read books when I can understand it, but I can watch videos repeatedly when I can’t. It can be done in about a week.
Then I learned CDH cluster building, Python crawler, I am Java back-end development, these online to find some information to read books, very easy to learn, get started quickly.
Here, I recommend “In-depth Understanding of Java Virtual Machine” and “Java High-concurrency Practice” must see books.
Then I started learning Hadoop, hbase, Hive, Scala, Spark, Python, Kafka. When I started learning Hive, I got a little bit giddiness. I bought relevant books such as HBase Authoritative Guide, Hive Development Guide, Learn Scala Quickly and Spark Fast Big Data Analysis, and then visited many people’s blogs. I don’t know if self-taught people have the same trouble as me. I feel I can understand the contents of the books, but I don’t know how to use them after reading them. And then they don’t assemble and apply what they’ve learned.
Then I will stay at this stage, stopped more than a month of learning plans, leaving behind the office job-hopping quarrel with his girlfriend broke up, and so on have some things, I again remind of, or to their own career planning, conveniently to an online school, met a good teacher, the certificate of completion of 3 months even if the learning cycle. The plan is to finish the New Year before jumping, job-hopping people know, gold three silver four ~
Here is a reminder that if you feel that your reading efficiency is too slow, or easy to think illogically, you can collect some courses on the Internet, follow the course is OK. It all depends on your situation. If you can’t read efficiently, take online classes. If not, read on your own. If you have no self-control, you have nothing to recommend.
Anyway, I need to master and learn the knowledge point to tell you, you have time and energy, it is good to study the corresponding.
Iv. Data sharing (video, e-book, interview materials, project source code)
I admit very want to share the net school video that I sign up for, but now the school all abnormal condition, I paid money all can see on 2 computers only, all video encrypted… Can’t share
So I can only give you part of the information I sorted out, remember to pull it to the end…
Last but not least: keep an eye on new developments in technology, and keep learning and improving yourself.