Last week, the Deep Learning Summit ICLR 2019 was held in New Orleans. Chip Huyen, a Stanford-educated computer scientist now at Nvidia, attended the conference and came up with eight takeaways.
Compiled by Chip Huyen, Heart of the Machine.
1. Inclusive
ICLR 2019 organizers stressed the importance of inclusion in AI, and the first two major presentations — Sasha Rush’s opening address and Cynthia Dwork’s invited talk — were all about fairness and equality. Here are some alarming statistics:
-
Only 8.6% of presenters (oral and poster) and 15% of attendees were women;
-
Two-thirds of LGBTQ+ researchers did not attend in a professional capacity;
-
All eight invited speakers are white.
Unfortunately, this situation does not seem to have caught the attention and reflection of AI researchers. Other workshops were full, but the AI for Social Good workshop was empty until Yoshua Bengio showed up. In the many ICLR exchanges I attended, no one mentioned diversity until I asked out loud why I had been invited to a tech event that wasn’t right for me, and a friend told me, “This may sound offensive, but you’re here because you’re a woman.”
One reason for this is that these topics are not “technical”, so spending time on them won’t help your research career. Another reason is that there are still some problems in social propaganda. A friend once told me to ignore the guy who challenged me in a group chat because “he likes to make fun of people talking about equality and diversity.” I have friends who don’t talk about diversity at all online because they don’t “want to be associated with it.”
2. Unsupervised representational learning & transfer learning
The main goal of unsupervised representation learning is to discover useful data representations from unlabeled data to complete subsequent tasks. In NLP, unsupervised representation learning is usually accomplished by language modeling. The representations learned are then applied to emotion analysis, named entity recognition, machine translation and other tasks.
There were some exciting papers last year on NLP unsupervised representational learning, Including ELMo (Peters et al.), ULMFiT (Howard et al.), OpenAI’s GPT (Radford et al.), BERT (Devlin et al.), and of course, And gpT-2, which is too dangerous to publish.
The complete GPT-2 model was demonstrated on THE ICLR with stunning results. You can enter any prompt and it will complete the subsequent text creation.
GPT – 2 application website: https://talktotransformer.com/
Adam King’s app site with GPT-2. Enter “What does Amazon want to do” (PS: the actual conversion speed is slow, the animation is the effect after deleting most of the repeated frames). Web site address: https://talktotransformer.com/
Gpt-2 can write articles, scientific papers and even make up definitions of words. But GPT-2 doesn’t seem to be fully human yet. The team is working on GPT-3, a larger and possibly better model.
Although computer vision was the first community to successfully apply transfer learning, its basic task — training classification models on ImageNet — remains supervised. Both the natural language processing community and the computer vision community are asking the same question: “How can unsupervised representation learning be applied to images?”
Although the most prestigious laboratories have begun work, only one paper appeared at ICLR: “Meta-learning Update Rules for Unsupervised Representation Learning” (Metz et al.). Instead of updating the weights, the team used algorithms to update the learning rules. Then, the representation learned from the learning rules is fine-tuned on a small number of labeled samples to solve the image classification problem. They were able to find learning rules with an accuracy greater than 70% on MNIST and Fashion MNIST. The authors do not intend to publish the code because “it is computationally relevant”. The outer loop takes about 100,000 training steps and 200 hours of training on 256 Gpus.
I have a feeling we’ll be seeing more papers like this in the near future. Tasks that can be used for unsupervised learning include: Automatic encoding, prediction of Image rotation (Gidaris et al. ‘s paper “Unsupervised Representation Learning by Predicting Image Rotations” is very popular in ICLR 2018), and prediction of the next frame in the video.
3. ML retro
The idea in machine learning, like fashion, is cyclical. Walking around the Poster session feels like walking down memory corridors. Even the most anticipated ICLR debate has ended in the “transcendental versus structural” debate that Yann LeCun and Christopher Manning discussed last year, similar to the perennial debate between the Bayes and frequency schools.
The MIT Media Lab’s Grounded Language Learning and Understanding program was shut down in 2001, but Grounded Language Learning returned this year with two papers, But it’s wearing the cloak of reinforcement learning:
-
Dom-q-net: Grounded RL on Structured Language (Jia et al.) : Given a target expressed in natural Language, this reinforcement learning algorithm navigates web pages by filling fields and clicking links.
-
BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning (Chevalier-Boisvert et al.) : A platform compatible with OpenAI Gym, with a hand-made BOT Agent, which can simulate human teachers to guide agents to learn synthetic languages.
My thoughts on these two papers are the same as those of AnonReviewer4:
“… The methods proposed here are very similar to those studied in the semantic analysis literature, although this paper only cites recent papers on deep reinforcement learning. I think the authors would benefit a lot from the semantic parsing article, and the semantic parsing community could take some inspiration from this paper… But the two communities don’t actually talk much, even though in some cases we’re working on similar issues.”
Deterministic Finite state automata (DFA) also has two papers on the ICLR 2019 stage:
-
Representing Formal Languages: A Comparison Between Finite Automata and Recurrent Neural Networks (Michalenko et al.)
-
Learning Finite State Representations of Recurrent Policy Networks (Koul et al.)
The main motivation of the two papers is whether it is possible to reduce the number of states to a finite number because of the large space of hidden states in RNN. I question whether DFA can effectively represent RNN in language, but I quite like the idea of learning RNN during the training phase and then converting it to DFA for reasoning, as shown in Koul et al. ‘s paper. As a result, the finite representation obtained in Pong requires only 3 discrete memory states and 10 observed states, which can also help explain RNN.
4. The RNN fall from grace
The change in submitted paper topics from 2018 to 2019 reflects a significant decline in RNN. This is to be expected, because RNNS, while applicable to sequential data, have a major drawback: they cannot be parallelized and thus cannot take advantage of computational power, the biggest driver of research since 2012. RNNS are not popular in the field of computer vision and reinforcement learning, and they are being replaced by attention-based architectures in the once-popular field of natural language processing.
Does this mean that RNN will die? Not so. One of ICLR’s best papers in 2019, Ordered Neurons: Integrating tree Structures into Recurrent Neural Networks, has something to do with RNN. In addition to this best paper and the two papers on automata mentioned above, ICLR 2019 received nine papers on RNN, most of which delved into the mathematical basics of RNN rather than exploring new applications of RNN.
RNN is still very much alive in industry, especially for companies that deal with sequential data, such as trading companies. Unfortunately, these companies often don’t publish their own research. Even if RNN is not so attractive to researchers now, we know it could make a comeback in the future.
5. GAN is still going strong
The relative change in GAN was negative from last year, but the number of papers actually increased, from about 70 to about 100. Ian Goodfellow was invited to give a speech on GAN, which attracted a lot of attention as usual.
ICLR 2019’s first Poster session is about GAN. There are many new GAN architectures, improvements to existing GAN architectures, and GAN analysis. GAN applications include image generation, text generation, audio synthesis and other fields, including PATE-GAN, GANSynth, ProbGAN, InstaGAN, RelGAN, MisGAN, SPIGAN, LayoutGAN, KnockoffGAN and other different architectures.
The GAN Poster session reveals just how polarized the community’s response to GAN has been. Some non-GAN researchers have commented, “I can’t wait for the GAN craze to pass,” and “My brain freezes when anyone mentions confrontation.” Maybe they’re jealous.
6. Lack of biological heuristic deep learning
Given all the angst and talk about gene sequencing and CRISPR babies, it’s surprising that the ICLR hasn’t produced many papers that combine deep learning and biology. There are six papers on this topic:
Two articles on biological heuristic architectures:
-
Biologically-Plausible Learning Algorithms Can Scale to Large Datasets (Xiao et al.)
-
A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs (Lindsey et al.)
An article on RNA Design:
-
Learning to Design RNA (Runge et al.)
Protein Manipulation
-
Human-level Protein Localization with Convolutional Neural Networks (Rumetshofer et al.)
-
Learning Protein Structure with a Differentiable Simulator (Ingraham et al.)
-
Learning protein sequence embeddings using information from structure (Bepler et al.)
There was no paper on genomics, no workshop on the topic. It’s sad, but it’s a huge opportunity for deep learning researchers who are interested in biology or biologists who are interested in deep learning.
7. Reinforcement learning is still the most popular topic in submitted papers
Research papers presented at ICLR 2019 show that the reinforcement learning community is moving away from model-free approaches to sample efficient model-based algorithms and meta-learning algorithms. This shift is likely due to the extremely high scores on the Mujoco continuous control benchmark set by TD3 (Fujimoto et al., 2018) and SAC (Haarnoja et al., 2018), And R2D2 (Kapturowski et al., ICLR 2019) set the high score on Atari discrete control figures.
Model-based algorithms (that is, learning an environment model from data and then using that model to plan or generate more data) still achieve the same asymptotic performance as model-free algorithms when trained with 1/10-1/100 of experience. This advantage suggests that model-based algorithms are suitable for real-world tasks. While the learned simulator may be defective, its error can be mitigated by more complex dynamic models, such as simulator integration (Rajeswaran et al.). Another way to apply reinforcement learning to real-world problems is to enable simulators to support arbitrarily complex randomization: strategies trained in diverse simulated environments may consider the real world to be “another randomization” and succeed (OpenAI).
Meta-learning algorithms that perform fast transfer learning on multiple tasks lead to significant improvements in sample efficiency and performance (ProMP (Rothfuss et al.), PEARL (Rakelly et al.)). These improvements bring us closer to the “ImageNet moment of reinforcement learning,” where we can use control strategies learned on other tasks rather than train strategies from scratch (something that is still not possible on complex tasks at present).
A large portion of the papers accepted by ICLR 2019, as well as all of the papers presented in Structure and Priors in RL (SPiRL) Workshop, are on how to integrate environmental knowledge into learning algorithms. While one of the main advantages of earlier deep reinforcement learning algorithms was generality (for example, DQN used the same architecture for all Atari games without knowing any particular game), the new algorithm shows that integrating prior knowledge can help solve more complex tasks. For example, in the Transporter Network (Jakab et al.), agents use prior knowledge to perform structural explorations with more information.
In summary, over the past five years, the reinforcement learning community has developed a number of effective tools to solve model-free reinforcement learning challenges. Now, it’s time to come up with more sample-efficient, transferable algorithms to leverage reinforcement learning for real-world problems.
In addition, Sergey Levine may have received the most papers this year, with 15.
8. Most accepted papers are quickly forgotten
At the conference, I asked a prominent researcher what he thought of this year’s acceptance papers. “Most papers are forgotten as soon as the conference is over,” he chuckles. In a field as fast moving as machine learning, where SOTA results can be broken within weeks or even days, it is not surprising that most accepted papers are surpassed before they are formally presented. For example, seven of the eight adversarial sample papers in THE ICLR 2018 papers had already been broken before the conference began.
One comment I often hear at conferences is that papers are accepted/rejected randomly. I won’t name names, but some papers that are well known or highly cited have been rejected. In addition, many of the accepted papers may go uncited.
As a member of this field, I often face existential crises. Whatever inspiration I came up with, it seemed that someone else was already doing it, better and faster. If a paper doesn’t help anyone, what’s the point of publishing it? Who can help me out!
conclusion
Of course, there are many trends that are not covered, such as:
-
Optimization versus regularization: The debate between Adam and SGD continues. This year’s ICLR conference presented a number of new technologies, some of them quite exciting. It seems that every lab is developing its own optimizer, and so is our team.
-
Evaluating metrics: As the generation model becomes more popular, it is inevitable that we will need to come up with new metrics to evaluate the generated output.
If you’d like to learn more, David Abel has released a 55-page ICLR 2019 note. For those who want to know what’s hot in ICLR 2019, this chart will help:
I really enjoyed ICLR, it was big enough to find a lot of like-minded friends, but small enough not to have to queue for everything, and 4 days was just right. NeurIPS, by contrast, are a little too long, and four days into the meeting I’m looking at Poster and thinking, “Look at all this knowledge, I should have it by now, but I don’t want to.”
What I gained most from ICLR was not only ideas, but also motivation. The remarkable work of many researchers my age has opened my eyes to the beauty of research and inspired me to work harder. It’s also nice to spend a week catching up on papers and seeing old friends.
Huyenchip.com/2019/05/12/…