DataCamp’s Hugo Bowne-Anderson spoke to Keras creator Francois Chollet. Francois Chollet is an artificial intelligence and deep learning researcher, the creator of Keras, the Python deep learning framework, and the author of a new book, “Python Deep Learning.”
1. Francois, you are a researcher of artificial intelligence, but the concept of artificial intelligence is too broad. Could you tell us a little bit about what you do?
This is a keen observation, and usually as an AI researcher, we can only study one area, which can lead to very big differences in what each AI researcher is actually doing. In my case, I would probably do something like build Keras, a deep learning framework. In addition to developing Keras, I also integrated Keras into Google’s machine learning framework, TensorFlow. In this regard, I have recently carried out theorem verification in machine translation, computer vision and deep learning. My main research interest is to understand the problems of abstraction and reasoning in AI, how to model from perception to abstraction to high generalization.
2. You are the author of Keras, a well-known framework for deep learning, so can you tell us what deep learning is?
Deep learning is a specific method of machine learning that is more powerful and flexible than previous methods. In most applications, what we call “deep learning” can be thought of as converting large amounts of human-annotated data into software that can automatically annotate new data in a human-like way. You can automate a lot of different tasks this way. Deep learning is particularly good at understanding “perceptual” data, such as images, video or sound.
For example, identify and categorize a large number of images, with tags associated with each image (” dog, “” cat,” etc.). Deep learning can help you automatically map data to a system of labels, just learning from examples, without any manual adjustments or custom engineering. Such a system could be applied to new data, effectively automating the task of tagging images. Similarly, you can apply deep learning to machine translation, speech recognition, text-to-speech, optical character recognition, etc.
3. Congratulations on the publication of your new book, deep Learning with Python. Why did you write it?
This book is a lesson I want to take out — to teach deep learning and Python programming to people who don’t have a machine learning background. In my opinion, deep learning should be a technology for every programmer and should not be relegated to the shelf as the exclusive domain of some great people.
4.Python is arguably the fastest growing programming language. Why did you choose Python?
I love the Python. Learning to use another language is easy no matter how long you have studied it. It feels more intuitive and elegant than any other language I’ve used before. But Python’s real killer is not the language itself, but the ecosystem and community around it. Whatever you need to do: parse a particular file format, interface with a particular system, it’s almost certainly a Python library behind it, so you don’t have to spend a lot of time writing code. This is especially true in data science and machine learning, where there are many great tools: Numpy, pandas, SciKit-learn, Plotting libraries, and so on, that make Python a very efficient language.
Another reason I like Python is that it is not a domain-specific language, but sits at the intersection of many domains, from Web development to data science to systems administration. This means that you don’t have to switch to a new language to deploy the Keras model as a Web API. Whatever you need to do, even if it’s to launch a Web application, query a REST API, parse some files, and train the most advanced deep learning models, Python can usually do it for you.
5. For those who want to enter the field of machine learning and artificial intelligence, the fear is always there. Can you offer some advice on getting started?
I personally think that in the last five years, getting into machine learning has become very easy. Of course, five years ago this was very difficult, even the minimum requirement for graduate students. Because you need to write a lot of the underlying algorithms yourself, usually using C ++ or Matlab. But now it’s different. All you need is Python, and you have access to advanced apis and easy-to-use tools like Keras. Plus, you can find a lot of really high quality learning resources online where you can practice real world problems on Kaggle. Learning has never been easier.
Alternatively, you can pick up my book, install Keras, challenge some Of Kaggle’s projects, and after a few months, you’ll find that deep learning isn’t hard at all. Be bold and take the first step.
6. What were the goals of Keras and your new book?
When I first opened source Keras, I didn’t particularly want to democratize deep learning. But over time, I’ve seen countless people master deep learning through Keras and solve all kinds of problems in ways I didn’t expect, and it’s really exciting. I realized that deep learning could be deployed in more areas in transformative ways. There are a lot of people out there who could benefit from using deep learning. So I’m more and more concerned about how to get as many people as possible to use these technologies.
My goal with this book is to get as many people into deep learning as possible, so they can start using it to solve problems they have. It’s not enough to provide tools that are easy to use, I should also provide learning materials that teach people how to use them.
7. What is the most important thing for beginners to get started? How can they do that?
Personally, I think the most important thing is to have a deep understanding of what deep learning can and cannot do. This requires a feel for key practice processes, such as how to properly evaluate the model and how to overcome overfitting. This requires a combination of a solid theoretical foundation and extensive practice in real world problems.
8. Artificial intelligence is a term that, to many people, brings to mind sentient robots. I wonder if you can uncover the mystery of AI for us?
Of course, there’s a lot of hype in this area, and most media coverage of AI and deep learning is disconnected from reality. As for what AI can do today, it’s a conundrum. I personally think there are three kinds of things that can be done:
1. Specify the rules the AI needs to follow in the tasks we perform. This is essentially what is called “symbolic AI” or, more pragmatically, “software development”. As any programmer knows, this approach is fragile and only works in Settings where everyone is controlled, which is rare in the real world.
2. Perform simple perceptual tasks where we cannot specify rules explicitly, but we can provide examples of many tasks, such as categorizing pictures, transcribing speech, etc. An important limitation of our ability to do this is that our models can only deal with inputs that are very close to what they’ve seen before, they can’t get results that are very far from your training data, and what we’re doing here is basically high-dimensional curve fitting.
3. A combination of the above two types. For example, you can imagine a robot with a deep learning module that can extract the types and positions of many objects around it and train them, plus a module that hardcodes high-level rules to deal with those objects. Another example is AlphaGo/AlphaZero, which is basically a combination of brute force search (explicit programming) and deep learning modules that can “intuitively” evaluate the value of each choice.
If you combine existing technologies well, you can achieve superhuman performance on many tasks, but only in specific areas. But for most industries, being good at specific tasks has huge potential. So you can see a lot of economic impact of AI over the next 20 years, you can think of AI as the steam engine of our time, it’s going to be a very powerful tool in human hands, it’s going to reshape the economy for decades.
9. Can you explain what is a non-artificial intelligence capability?
Of all the things we want to automate, AI today can only handle a very small subset. There are more things we can’t do than what we can do. In general, we are particularly bad at:
1. Anything that requires “deep understanding.” For example, AI can’t understand the meaning of natural language and instead treats language with statistical dependencies or hard-coded processing rules. “Meaning,” as it exists in the human mind, comes from concrete human experience that our AI models cannot capture. So no AI system today can “understand” its own tasks in a way that makes sense to humans, and models simply plot statistics of their training data.
2. Anything that works with data that the AI hasn’t seen before. Ai can only apply rules that you have explicitly coded, or recognize things that are very, very close to what it has been trained to do. The more uncertainty or variation you introduce into a mission, the less capable the AI will be.
3. Anything that involves reasoning and abstraction. We can hardcode obvious rules of inference into machines, and at present AI cannot abstract models on its own. This is arguably the major bottleneck in AI development today. If you solve this problem, you can solve the first two in no time.
10. What do you think are the main challenges facing the deep learning community right now?
Hype, develop moral awareness, maintain scientific rigor.
1. Hype: This is the biggest problem plaguing our field. Some are exaggerating recent progress in absurd ways, far beyond our current capabilities, but that is not the case. If we set sky-high expectations and then fail to meet them, it will have a very negative impact on science.
2. Ethics: The people who develop AI systems today don’t come from very diverse backgrounds, and they often don’t realize that there are ethical violations in the systems they build. We need to talk more about this issue and raise awareness of the potentially unethical applications of AI.
3. Scientific rigor: A large number of deep learning documents are published every day, and most of them don’t really yield any meaningful new knowledge because they don’t follow the scientific method. They “evaluate” models in vague ways, or test overtrained models (specifically generative models and reinforcement learning, two of the fastest growing topics in deep learning research) on their training data, artificially using weak baselines to adjust hyperparameters, leading to overadaptation to specific tasks. Deep learning is a scientific disaster zone, and peer review often doesn’t address these issues in any meaningful way, perhaps in part because most peer reviewers have worked in the field for a year or two at most. If we want to make faster progress, then we need to inject more stringent expectations into this area when it comes to reproducibility of studies, baselines, model evaluation and statistical significance.
11. Can you talk about the future of deep learning?
I actually wrote a blog post about this. In short, I expect AI to increasingly combine “intuitive” pattern-recognition modules with formal reasoning modules. I also hope that AI will evolve into something more like automated software development, borrowing many of the patterns and practices in software engineering today.
12. Not only are you a developer, but on your Keras blog, in your book and on Twitter, you are always thoughtful and vocal about the ethics of deep learning, artificial intelligence technologies. Do you think developers have an obligation to have ethical considerations around the work they do?
Of course. I think technology in general has been lacking this in recent years, for example, most smartphone games, whether they enrich our lives or persecute them. Technology is never neutral because it is powerful and it can affect our lives. So if you’re building a social media platform that maximizes engagement, it’s an ethical decision that makes a lot of sense. Either we care about making morality the explicit design goal of the technology and power structures we build, or we abandon our values. If you’re in tech, you can’t choose to be “on the sidelines.” It’s an illusion.
This article is recommended by Beijing Post @ Love coco – Love life teacher, translated by Ali Yunqi Community organization.
An Interview with Francois Cholle
By Hugo Bowne-Anderson
The tiger said eight things.
The article is a brief translation. For more details, please refer to the original text