Dr. Hans Uszkoreit, Science and Technology Director of The German Artificial Intelligence Research Center, believes that language technology is a core part of artificial intelligence, but current deep learning methods are not enough to solve the core problems in the NLP field.
Reply to “language” on the wechat official account of AI science and technology base (rgznai100) and download the TOP10 papers cited by Hans Uszkoreit.
The reporter | hu
The 3rd China Artificial Intelligence Conference 2017 (CCAI 2017), sponsored by Chinese Society for Artificial Intelligence, Alibaba Group & Ant Financial, and undertaken by CSDN and Institute of Automation, Chinese Academy of Sciences, will be held in Hangzhou International Convention Center from July 22 to 23.
Before the conference, we interviewed the Keynote guest, Dr. Hans Uszkoreit, scientific and technological Director of The German Artificial Intelligence Research Center.
Dr. Uszkoreit is a central figure in the Sino-German AI cooperation, responsible for all the joint projects of the German AI research Center in China. He was just appointed director and chief scientist of the newly established Artificial Intelligence Technology Center (AITC) in Beijing in March. In the interview, Dr. Uszkoreit talked about the application of AI in Industry 4.0 and business intelligence, as well as the differences between China, the United States and Europe in the field of AI.
As for his old profession, Dr Uszkoreit says:
Language technology is a core part of artificial intelligence, but current deep learning methods are not enough to solve the core problems in THE NLP field.
He mentioned the potential of The Chinese language for semantic understanding.
Referring to the recent NLP controversy, Dr Uszkoreit sees Yoav Goldberg as a hero for advocating strict rules on the right way to conduct research.
Below is the full text of the interview:
Chinese AI research needs to cover all fields
CSDN: Recently, the media often compares the AI industry and research achievements of China and the United States. What do you think of the differences in AI between China and the United States and Between China and Europe? In your opinion, which party will lead this round of AI revolution?
HansUszkoreit: AI research in Europe and the US has a long and broad base, but China is catching up with amazing strength and enthusiasm. In some areas of research, China may be developing faster than the US. However, AI research in China is concentrated in a few hot areas, but hot spots and trends come and go. The next generation of AI architecture will use systems that are broadly good at AI with a large number of cognitive tasks and capabilities. I hope Chinese researchers will have enough motivation to cover all areas of AI research quickly.
The breakthroughs I personally look forward to in AI are the intersection of multiple sensory inputs and AI’s acquisition of common sense knowledge and intuition.
Europe tends to be well prepared to invest in research and development in areas such as semantic technology, neural networks and machine translation, but the scientific maturity and commercial gains are more likely to occur in the United States. The exception is manufacturing AI, where Europe, and Germany in particular, excel. Now That China is competing with the United States in downstream applied research and upstream funding for AI, it is quite intriguing. The former is key to TODAY’s AI applications, while the latter has the potential to reverse our past AI innovation processes.
CSDN: In recent years, there are many Chinese AI scientists and AI researchers like Li Feifei, who have made outstanding contributions to the progress of deep learning. I understand that many of your research partners are also Chinese. Could you tell me why you chose them? What are the advantages of Chinese in AI research?
HansUszkoreit: I have long enjoyed working with PhD students or postdoctoral fellows in China. They are smart, proactive, energetic and pragmatic. Generally speaking, Chinese researchers have a very solid foundation in high school and university. My own experience is that mixed teams of Chinese and Western researchers work surprisingly well together. I will continue to promote such cross-cultural cooperation in Beijing. I am looking forward to interacting with my former close colleagues and students, most of whom are now working at CAS, Chinese universities and companies.
The investment environment and early market are the guarantee for the success of AI start-ups
CSDN: Not long ago, you just served as the director and chief scientist of Beijing Artificial Intelligence Technology Center (AITC). Can you tell us a little bit about your new job and your new research institute?
HansUszkoreit: AITC was established in Beijing’s Yizhuang Economic and Technological Development Zone in March this year. Its mission is to transform AI technology from research to industrial applications. In Germany, we haven’t had a particularly successful commercial AI case. Many times, I’ve been involved with companies that entered the market too early and survived years of struggle. But more often than not, we have watched our American competitors succeed because of a lack of capital. Not only do they have a better investment climate, they also have a bigger early stage market.
In China, I have noticed both of these factors: a friendly investment climate and a cutting-edge B2B market with huge demand. At DFKI (German Artificial Intelligence Research Center), my old employer, we conducted research with more than 20 industrial shareholders and founded more than 80 spin-off companies. This experience in AI technology transformation was not easy to come by.
Based on this experience, AITC’s ability to implement such technology transformation and research mechanisms, as well as best practices for successful commercialization of AI, gives us the ability to help others in the field as well.
CSDN: Industry 4.0 and business intelligence will become the mainstream AI application scenarios, but what are the differences between these two fields? Are there any concrete examples of AI’s success here?
HansUszkoreit: Industry 4.0 is a broad term for the fourth industrial Revolution, which is triggered by the complete digital connectivity between all sectors, equipment and people in the industry. This complete digital connection is made possible by the Internet of Things, which also includes connections between machines, products, vehicles and buildings.
Business intelligence applies to all companies, not just manufacturing. It is based on data within the organization about all decision-making processes, from strategic decisions to day-to-day operations. Most of this data comes from inside companies, but many of the important signals come from outside consumers, investors, policy makers, suppliers and contractors, and the lives of employees. Analysis of all this data can help you make better decisions and even optimize and fine-tune the decision-making process.
For manufacturing, business intelligence is part of Industry 4.0. What we see today is just the first step in business intelligence and industry 4.0. Data here is usually acquired and integrated actively. For data interpretation, especially for unstructured data, AI will play an important role and learn from the data. Control, optimization, and predictive management in logistics and supply chains are examples of AI applications in this area.
Language technology is at the heart of AI
CSDN: You are a top expert in language technology. What is the role of language technology in AI? What are the prospects? Is there a breakthrough moment for natural language processing, as deep learning did for image recognition and speech recognition?
HansUszkoreit: Language is the key to knowledge, and knowledge is the ultimate goal of AI. It is through language that knowledge of human society is passed down from generation to generation. Humans cannot acquire a wide range of reusable knowledge simply by observing others. Ai must be able to “read” and “listen” at the same time to acquire the knowledge needed for the next generation of intelligent systems. The key to this level of machine learning is NLP. NLP is also the key technology for successful communication between human and AI. Therefore, language technology is a core part of AI and will be largely integrated with knowledge technology.
CSDN: What do you think of current consumer language technologies? Especially the current wave of smart voice assistants, like Amazon’s Echo and Apple’s HomePod?
HansUszkoreit: These smart assistants are becoming part of our daily lives. I use it every day myself. They are far from perfect, but they can be improved quickly because their avant-garde users are giving them lots of free data every day.
Chinese has some potential in semantic understanding
CSDN: What are the differences in language processing techniques for different languages? For example, Chinese and English.
HansUszkoreit: Different languages do vary a lot. Although spoken, Both Chinese and English can be learned by children in the same time. But in detail, Chinese has no morphology and a fairly simple syntax. The two, as written languages, could never have been learned in the same time. In fact, the Complexity of the Chinese language is unparalleled. This is trickier for NLP: Chinese words do not even have a starting mark. Apart from the inherent complexity of the language, there is another reason why Chinese is more difficult to process on a computer: NLP has been dominated by England-centric research institutes.
But I wouldn’t be particularly surprised if NLP’s future research methods and algorithms work better in Chinese and other East Asian languages than they do in English. One prerequisite for this is to find ways to improve semantic understanding, since syntax is far less important in Chinese than in western languages.
CSDN: The NLP debate between Yann LeCun and Yoav Goldberg last month was notable. What do you think of this debate, especially the relationship between deep learning and NLP? Which side do you support? Why is that?
HansUszkoreit: I think this debate has been misread, it’s not a fight between deep learning advocates and skeptics in the NLP space, it didn’t start that way. Yoav Goldberg is not anti-deep learning, nor is he anti-deep learning in NLP. On the contrary, Yoav has greatly promoted the application of deep learning in NLP.
Yoav Goldberg was just complaining about the clickbait paper in the field of natural language generation (NLG), which boasted of some very small results. Yoav is right: the clickbait paper makes no sense for the progress of the NLG field, and it fails to address any of its acknowledged problems.
The reason Yann LeCun and Fernande Pereira think they should side with the authors of the paper is that there are indeed many NLP researchers who are extremely skeptical of the role of deep learning in language analysis and generation. LeCun and Pereira see this skepticism as an outdated research paradigm’s feeble attempt to defy deep learning. It’s the old game of the scientific revolution, conservatives versus progressives. But that is not Goldberg’s argument.
My personal view is
Current deep learning methods are not enough to solve the core problems in the NLP field. But they have improved and implemented many applications of NLP technology. The deficiency of deep learning here does not lie in the various artificial neural networks currently used and their respective learning algorithms, but in the fact that we do not have the correct type and sufficient number of language class annotation data.
Human language and the human brain have co-evolved in such a way that language can be used to express information and knowledge, while also allowing children to learn it in a very short period of time. The learnability of language is closely related to the learnability of basic knowledge concepts. You can’t learn a concept without a language, and you can’t learn a language without combining it with a concept.
If we could figure out a way to teach ai both language and concepts, the problem would be solved. The first step here is machine learning of reusable knowledge based on artificial neural networks.
In the face of such technological change, Goldberg is simply advocating strict rules for the right kind of research. But we all know that in such circumstances, the usual standards of behaviour do not always apply. The people who have the courage to speak at the right moment during social change are often the heroes of our history.
Three pieces of advice for young practitioners
CSDN: What is your most valuable experience in your AI career? What advice do you have for the new generation of AI practitioners?
HansUszkoreit: I have three tips.
Expand your horizons: Travel abroad, or at least work for a multinational company. I spent nearly a decade in the United States and led several international projects. I’ve been co-director of an international doctoral program, and I’ve run an international graduate program. My experiences in international programs, summer schools and conferences have greatly enriched my professional and personal life.
Love data, and work with data you love: it could be business statistics, images, audio, video, or text, all of which have their own unique, rich, and meaningful internal structure. Try to understand the structure here, try to interpret the data on your own. Always do high-quality error analysis, or even read the error data yourself. Try to relate the properties of the algorithm to the properties of the data.
Try to reach out to research outside your field: at a minimum, consider from time to time how your subfield relates to neighboring fields, how your data relates to other types of data, and how your methods relate to other methods. Don’t pass up opportunities to talk to experts in other fields just because you don’t understand. Urge them to explain their problems and solutions in the simplest way possible, and try to explain your own research work in the same way. Learn more about the mechanisms of human cognition, even if the mechanisms of machine intelligence are entirely different.
CCAI speech highlights
CSDN: Your topic at CCAI was “Business Intelligence applications combining machine learning and knowledge interpretation”. However, compared to the previous generation of rule-based ARTIFICIAL intelligence, machine learning and deep learning have made great progress in recent years, so why do we still need this kind of rule-based knowledge engineering?
HansUszkoreit: Currently, deep learning is mainly used to acquire some form of “intelligent” behavior. Given input, the system can learn and react to human methods. These systems do not yet have external reusable knowledge, but can acquire some internal knowledge. It’s just that such knowledge often can’t be reused for other tasks. I’m not a fan of ai using knowledge engineering like it was trying to do 30 years ago, but I do believe that AI will eventually find a way to use the vast amount of external knowledge that humans already have (wikipedia, structured DBpedia, etc.) and that it will soon automatically acquire more of it.
Rather than discuss the competition between deep learning and deep knowledge, I think about the potential for the two technologies to combine effectively: if only machines could learn from humans, it would be possible for them to learn from millions of people.
CSDN: What are your expectations for this CCAI conference? Which lecture would you most like to hear?
HansUszkoreit: There are many AI research teams and centers in China that I don’t know about yet, and I’m looking forward to their research results and application innovations. I’m curious about what Chinese companies can contribute to AI.