Every year we communicate with students working together on short-term machine learning research projects. So far, we have talked to many students and found many problems in their projects, and given them many suggestions based on the problems. We find that many problems are common to all of us. In this article, we will focus on introducing these problems and give solutions. This article is by no means comprehensive, but highlights the pitfalls we see over and over again. For example, we won’t talk about how to choose a good project. Some of our recommendations are generally applicable to machine learning, especially deep learning or reinforcement learning research projects.
1. Main traps:
1.1 Assume your code is error-free
Machine learning, especially deep learning and reinforcement learning models, are notoriously difficult to debug. To see the myriad ways you can go wrong, check out Andrej Karpathy’s Twitter theme. All of us, even more senior researchers, make the same mistakes all the time. What makes this so hard to detect is that even flawed models can often still learn and produce meaningful output. Errors may make minor changes to your model, and most will only show up at run time. With this in mind, the worst thing you can do is assume that your code doesn’t contain any bugs. Machine learning researchers’ approach to code is also intended to divide it into productive and unproductive. If your default assumption is that there might be something wrong with your code, you’ll search for errors more carefully, go through the code line by line, and double-check the intermediate output. If possible, you’ll also visualize them. Do you think about the shape of the tensor? Are they properly initialized? Clone or separate? Monitor gradients and pay attention to nans during training. Seed the random number generator to write unit tests and make your experiments reproducible, etc. These steps can make your code more practical. For more tips on neural network debugging, see Goodfellow et al. ‘s deep Learning book.
1.2 Look only at the final assessment indicators
While your project goal may be to achieve some improvement in the metrics, it is more important that you fully understand how and why the model works. Especially in the early stages of a project, the final evaluation metrics contain very little information, but this information is useful for iterating and developing an algorithm or model. To get more information, we must ask deeper questions and get feedback. For example, if you introduce a gating mechanism or an attention mechanism, does your model actually use it? What model innovations do you propose that actually contribute to overall performance? How many training samples/periods does your model need to achieve reasonable performance, and how does this differ from the baseline you used? Are there systematic differences between test instances where your model works well or very badly? Can you predict important features from the hidden state of the model? Keep in mind that your research and project reports do not really inform the research community of some (marginal) improvement on the prior art level, but rather contribute to our understanding of the field.
1.3 Attempting random change has no clear expectation
Using current deep learning libraries, you can easily make your model more complex by adding more components, layers, and optimization tips. However, when you make a change to your code or model, you should at least have a general intuition about the change. Similarly, when you conduct an experiment, you should have clear expectations of the outcome. What do you expect the results to look like, and what do they tell you? This is even more important when you find that your model doesn’t work the way you expect it to. With these measures, you are more likely to see the cause of the error, because extending your model will not help you find the error, and may even make it harder to find the problem. Before making the model more complex, understand the bottom line where it can go wrong. Also, remember that in your report, you have to prove what you did. Because your report is the only way someone else can help you solve your problem. If you can’t formulate a research hypothesis and explain to yourself why you’re doing what you’re doing, then no one else has a chance to know what you’re doing.
1.4 Excessive complexity
We often see motivated students jump right out of difficult problems and try complex solutions. In fact, the real problems, generally difficult to analyze. Instead, ask yourself: What’s the smallest thing that should make a difference? Can a model learn to remember a small data set? What do you learn when you use just a few parameters? Does the code work on a single training instance rather than a batch training? What is the simplest form of generalization we would expect to see?
1.5 Iteration is too slow
Conducting training experiments can take a long time. In particular, deep learning and reinforcement learning can be time-consuming in accumulating statistically significant numbers of random seeds. Therefore, it is critical not to get caught up in a slow iteration cycle too early in a short project. Try to debug your models in a simple environment and implement proof-of-concept of your ideas. Try to run the whole process on your personal computer. Sometimes a simple matrix game or grid-world experiment can provide useful proof of idea. Similarly, you can use MDP’s exact value functions to test algorithmic ideas without messing around with gradient estimation, actor comment training, etc. When moving to larger experiments, simplify the process of starting the experiment and checking the results. Check these results before completing the experiment to see if the performance is stable. Investing in the infrastructure used by the training model may be time-consuming at the beginning, but will pay off at the end of the project.
2 Some suggestions:
2.1 Begin to read background knowledge and prepare related work before the project begins
We should usually analyze projects well in advance of the official start date. One of the reasons is to understand the background and the related work, and then we have to carry out the implementation and experiment, and finally we have to write a good report carefully, and we must have a big picture set before we do the project. Another reason is that we usually propose research projects, and if successful, they may be published in machine learning forums or journals. While we all know that a lot can happen during a project, most of the time we encourage you to at least start reading the literature ahead of time. Ideally, when you start working full-time, you should know what to do, how it relates to existing methods, and know how to get there.
2.2 Correct software version and backup
You really should use strict version control for your project code and project reports, and make timely backups. There’s nothing worse than losing all your hard work before a deadline. If you haven’t already, please open a GitHub account and as a student you can get free private repositories.
2.3 Random repetitions were used for evaluation
In academia, you can’t train with multiple Gpus during a project. The other awkwardness, however, especially in deep reinforcement learning, is that it is not easy to draw premature conclusions from a single experiment or a few. Ideally, you want to repeat the experiment many times and, as described above, understand the robustness of different starting conditions and hyperparameters.
2.4 Begin documentation as early as possible in the implementation of the project
If you are working on a master’s program, your work will be evaluated based on your written report, not on the outstanding work you have done, and you will need to have enough time to write clearly about what you have done. My personal advice is to start writing early and not underestimate the value of communication research. Clearly state your goals, assumptions, and contributions, and get the reader to follow your thought process and read your article carefully. You need to explain your design choices and clearly discuss your findings in the document. Ideally, you should write reports consistently throughout the project. This way, you force yourself to think about the next step, and you’re less likely to forget any important information by the time the deadline hits.
2.5 Ask for help when you need it
Your superiors are busy people, but they exist to help you. If there is a problem, contact your superiors in a timely manner, not at the next scheduled meeting. Proactively communicate and prepare the results you want to discuss ahead of time. Make the most of your supervisor! Finally, don’t panic! They’ve been through it all, and what might seem like an insoluble problem to you might seem like a natural one to your superiors.
ADVICE FOR short-term MACHINE LEARNING RESEARCH PROJECTS,