The field of artificial intelligence research has a certain threshold. For beginners, general typically direct purchase of some popular books, such as “watermelon”, “flower”, “xx days from entry to master”, “xx days from entry to give up” and so on, but most of the books are the foundation of knowledge, a bit dull and boring, in addition content is too esoteric, beginners can see for a period of time, want to give up. In this paper, I will share my two-year research experience with you as an experienced person. I hope it will be helpful to those who are going to be engaged in artificial intelligence research.
start
Find someone you can ask questions about at any time
When you first enter a company, you are often hesitant about basic questions that may reveal a lack of expertise. But over the course of a few months, my questions began to feel natural, carefully crafted. Before, I would have accumulated a lot of questions, but now I ask them as soon as I get one, so that I don’t get overwhelmed and confused.
Look for inspiration in different areas of research
This is not a time to go it alone. Knowledge is no exception, multidisciplinary communication. Deciding what direction to pursue can be the hardest part of research for everyone, and here are some of the strategies I’ve seen employed by researchers with a long track record:
- 1. Communicate with researchers in different fields. Ask them what they are interested in, whether there are data sets they want to analyze, and what are the shortcomings of existing technologies. The most effective work in machine learning is when it collides with biology, chemistry, physics, the social sciences, or pure mathematics. For example, I’m thinking about Matthew Johnson’s NIPS 2016 paper and Justin Gilmer’s ICML 2017 paper on the analysis of mouse behavior datasets and the application of quantum chemistry.
- 2. Write a simple baseline to get a feel for the problem. For example, try writing some calibration code to control an inverted pendulum. When you write baseline code, there are a lot of situations, problems, or ideas that come up on the fly that can help you understand the problem.
- 3. Expand on the lab section of a favorite paper. Read a paper carefully, learn about the methods used and the results obtained, and try to find some areas that can be improved. First consider the simplest extension, and then consider whether the method of the paper is reasonable and whether the experimental results are imperfect.
Focus on visualization tools and skills
Running visual scripts allows us to quickly verify that the code matches the idea. More importantly, good visualization often makes mistakes in ideas and code more obvious and interpretable. For a real task, it is difficult to come up with the right way to solve the problem. If you are using an iterative optimization model (such as deep learning), mapping the loss function is a good start. In addition, visualization can partially explain the neural network parameters learned from the “black box” method of deep learning. For example, when working with a graphical model and visualizing the distribution of its one-dimensional or two-dimensional variables, a lot of information can be inferred as it changes. Visualization is a barometer of the effectiveness of the technology, and each visualization analysis provides some feedback on the method or code being used.
TensorFlow visualization tool Tensorboard
Distribution of
Q value learning graph
Be clear about the original motivation of the researcher and the paper
It’s funny to see researchers presenting at the same conference, using the same technical jargon, but with diametrically opposite motives. Motivation is divided into the following three motivations — “mathematical” motivation, “engineering” motivation and “cognitive” motivation:
- “Mathematical” Motivation: What are the essential properties and limitations of intelligent Systems?
- “Engineering” motivation: How to develop intelligent systems that solve real problems better than other approaches?
- “Cognitive” Motivation: How to simulate natural intelligence like that of humans or other animals? Some papers have more than one motive. In addition, the motivation of every researcher cannot remain constant all the time, which is related to the interest of the engineer. Good papers and researchers will state their motivation at the outset, but some papers are often not very clear, and this requires careful reading, as well as careful writing, in case the motivation is not clear and the paper is rejected or withdrawn.
Further study of
Learn to find papers
The web is awash with ai papers, most of which will be published first on arXiv, and since the platform can be published and then reviewed, you need to learn to distinguish between them. Also, follow your favorite researchers on social media. There are also conferences to watch. The three major conferences are NIPS, ICML and ICRL. Other notable general conferences include AAAI, IJCAI, and UAI. For each subdiscipline, there are more specific meetings. For example, in the field of computer vision, there are CVPR, ECCV and ICCV; In the field of natural languages, ACL, EMNLP and NAACL are available. Robotics fields include CoRL, ICAPS, ICRA, IROS and RSS. Conferences related to theoretical work are AISTAS, COLT and KDD. JAIR and JMLR are two of the most prominent journals in artificial intelligence, but there are also good papers in Nature and Science. It is also very important to find some early papers, which are regarded as “classic papers” often appear in the reference papers. Another way to find early papers is to start with the personal pages of senior professors, where famous works hang. You can also search for keywords through some search assistants, such as Google Academic and Baidu Academic.
How long does it take to read the paper?
Two pieces of advice are often given on how to read papers. The first is to read all relevant papers in the first semester or year of graduate school. The second is when you read a lot of papers, don’t read extensively, but find a breakthrough, come up with innovative ideas. I personally agree with the first suggestion but disagree with the second. You should read as many papers as you can, as long as you have enough time for the original research. For professional researchers, it is not always possible to rely on personal luck to find innovative solutions. Sometimes the approach you come up with may have been tried before, but you just don’t know it. The vast majority of researchers are patient to track the progress and development trend of research direction, methodically thinking and solving problems. Reading papers is also a good way to figure out where you are and what you need to do next. One important tip about reading as many papers as you can: It’s just as important to take the time to digest a paper as it is to read it, and take notes as you read it, rather than gobbling up quantity rather than quality.
Dialogue >> Video >> paper >> interview
Papers are certainly the easiest source of knowledge about unfamiliar research theories, but what is the most effective route? Different people may feel differently. For me, I find that conversation (with people who already understand) is by far the fastest and most effective way to understand. If you can’t find someone to talk to, look for videos about the issue, such as an interview with the author of a paper, which can provide a good perspective. Also, when speakers speak to a live audience, they tend to prioritize clarity over brevity. In most essay writing, the author switches priorities, where the number of words is king, and explaining background too much shows the author to be unfamiliar with the field. At the bottom of the list are meetings. Simple meetings tend to be formal, and the content of the conversation with the host can be very valuable.
Beware of the hype
A series of achievements in artificial intelligence have attracted the attention of the public, making more people devote themselves to this field, and thus promoting more breakthroughs in artificial intelligence. The whole cycle is benign, but one side effect is that there is a lot of hype. Journalists, hot money investors, and startups are all culprits in the hype bubble. Therefore, when we read news or papers, we should pay attention to “clickbait” so as not to be misled. During the 2017 NIPS Q&A session, a prominent professor took to the microphone (representing the hype police) and cautioned authors about using the word “imagination” in their paper titles. This is the same as when we read the news, the headline is very attractive, but the content is not related to the headline, which makes the reader disappointed. The same is true for reading papers. Beware of hype. What we need to do is to evaluate whether a paper is helpful to us based on experimental methods and results.
Research is a marathon
Always improving
In the early days of exploring research projects, I would spend hours brainstorming, hoping that some vague directness would lead to a specific direction. Sometimes the project goes nowhere, but fumbling in the dark is part of the process. When you do not know what to do next, you can write down the most vague ideas based on the existing situation, and eliminate them one by one in the process of writing (write down the reasons for eliminating them). When you don’t have any ideas, take the form of reading or talking to colleagues for inspiration.
Learn to recognize and stop losses from dead ends
Taureans generally spend more time on good ideas, and being able to tell good ideas from bad depends largely on personal experience. Nonetheless, researchers of any calibre are constantly confronted with decisions about whether the research idea is flawed, whether it should be salvaged or further supported, whether it should be abandoned altogether. Especially in the early days, researchers hit a dead end and stayed there for a long time, unwilling to give up. Although giving up means wasting your time, sometimes it’s important to cut your losses in time.
writing
Some early career advice: Writing. Write a blog or a paper, but more importantly, write down your thoughts. Because writing helps us understand and think about relevant knowledge.
Mental health and physical health are prerequisites for scientific research
In the pursuit of scientific discoveries, academic researchers often encounter problems such as staying up late and skipping meals, which are not good habits. Many PHDS are going bald, and even masters are losing their hair. Exercising and letting your mind wander is an investment in research, not a hindrance. Working four hours after sleeping for eight hours is far more productive than working eight hours after sleeping for four hours. Sometimes you will get stuck, and even if you try your best, you can’t make any progress. At this time, it is recommended to leave your job, move a little and take a long breath to let go of yourself.
Tom Silver is an author with a focus on computational science and mathematics and artificial intelligence
Lessons from My First Two Years of AI Research
The original link