New wisdom yuan original

Authors: Zhang Gan, Jin Lei, Daming

【 New Jiyuan Guide 】Why didn’t the Turing Award go to Jurgen Schmidhuber, the father of LSTM? Schmidhuber, as a maverick in the AI industry, has been engaged in a war of words with the big three of deep learning, and confronted the developer of GAN on the spot, which has offended many people.

A forgotten god.

Yoshua Bengio, Geoffrey Hinton, and Yann LeCun were named the winners of the 2018 Turing Award.

The news that HLB (Hinton, LeCun, Bengio) has won the Turing Award is a well-deserved one, and the computing world is sending its best wishes.

Jurgen Schmidhuber, the father of LSTM, was not awarded the Turing Award. He’s also a big fan of deep learning.

Why Yoshua Bengio, Geoffrey Hinton, Yann LeCun and not Jurgen Schmidhuber won the Turing Prize?

Yoshua Bengio, Geoffrey Hinton and Yann LeCun won the Turing Prize (computing’s Equivalent of the Nobel Prize), making deep neural networks an important part of computing. Very cool! But Jurgen Schmidhuber?

Even Professor Zhou Zhihua, dean of the School of Artificial Intelligence at Nanjing University, said on his microblog that the LSTM was a textbook contribution.

Jurgen Schmidhuber is co-director of the Dalle Molle Institute for Artificial Intelligence in Switzerland. LSTM, which he developed in 1997 and is now widely used in applications such as Google Translate, Apple’s Siri and Amazon’s Alex, is one of the most commercialized technologies in deep learning.

Jürgen Schmidhuber

In addition to LSTM, Jurgen Schmidhuber is proud to have introduced a PM Minimization model in 1992.

Why is “proud” in quotation marks?

Jurgen Schmidhuber and Ian Goodfellow, the promoter of GAN, had a fierce battle online and offline, which is still fresh in the mind of the industry.

As for the HLB trio, Jurgen Schmidhuber fought several rounds of criticism, arguing that the HLB trio were playing in their own game while ignoring the contributions of earlier pioneers in deep learning. LeCun then wrote a post on the issue.

Did Jurgen Schmidhuber lose the award because he offended people?

The story starts five years ago.

In 2014, Ian Goodfellow’s first GAN paper was submitted to NIPS Conference. Among the three reviews, two were approved and one was rejected.

The rejection judge was Jurgen Schmidhuber.

Why did Jurgen Schmidhuber give Goodfellow, a young junior, such a contrary opinion?

It turned out that Jurgen Schmidhuber could not claim GAN as the first adversarial network; his own PM model, proposed in 1992, was.

In his review opinion for Goodfellow, Jurgen Schmidhuber directly challenged Goodfellow: GAN and PM look similar in many ways. Both methods use “adversarial” MLPS to estimate certain probabilities and learn coding distributions. The difference is that the new system learns to generate non-trivial distributions from statistically independent random inputs, whereas the old PM learns to generate statistically independent random outputs in response to non-trivial distributions (by extracting mutually independent factor feature coding distributions).

So GAN essentially changes the direction of PM — is that the main difference? Should GAN be called “reverse PM”?

Finally, Goodfellow had no choice but to add the comparison of the differences between GAN and PM in the final version of the paper, which resulted in the birth of the first GAN paper.

However, Jurgen Schmidhuber persisted, arguing with Goodfellow privately by email.

The most intense thing happened in 2016.

At that time, GAN was already gaining popularity in the academic world. Goodfellow had a Tutorial at the NIPS Conference in 2016. When he was talking about the comparison between GAN and other models, he was interrupted by a question from the audience.

That’s Jurgen Schmidhuber.

Jurgen Schmidhuber asked Goodfellow

What do you want to do with your GAN (GAN) and my work (PM) if it is similar to what I want it to do? (13) What is the problem with PM? (13) What is the problem with PM?

Goodfellow: We have communicated many times about the problem you mentioned in emails before, and I have publicly responded to you long ago. I don’t want to waste the audience’s patience on this occasion. (applause.)

play

Ian Goodfellow responds to Jurgen Schmidhuber at NIPS 2016

There was an awkward moment when a man in his fifties tried to run over a man in his early thirties and was killed by the man.

Later, Goodfellow revealed on Quora Shanghai that he contacted the sponsor of NIPS to ask if Jurgen Schmidhuber had a way to lodge a complaint with him, and NIPS would judge whether Goodfellow’s paper was unfair on behalf of the committee. But organizers say there is no such process.

In addition, Jurgen Schmidhuber offered to work with Goodfellow to write a paper describing the similarities and differences between PMS and gans, but only if the two were actually on the same page. That now seems unlikely.

Jurgen Schmidhuber had already had a spat with Bengio, Goodfellow’s mentor, and the HLB trio before he debutted with Goodfellow.

In May 2015, Bengio, Hinton and LeCun jointly published a review on Nature entitled “Deep Learning”. This paper starts from the traditional machine learning technology, summarizes the main architecture and methods of modern machine learning, describes the back propagation algorithm for training multi-layer network architecture, as well as the birth of convolutional neural network, distributed representation and language processing, and recursive neural network and its applications.

The paper is a classic in deep learning and has been cited nearly 14,000 times, according to Google Academic Statistics.

Judging from the writing style and content of this article, a considerable part of the content is the epoch-making achievement for which the three authors are famous. It has the meaning of “reviewing history and looking into the future” in deep learning, and the heroic meaning of “drawing a final conclusion from all the books” can not be blocked.

But less than a month after the Nature paper was published, he criticized it on his blog.

Schmidhuber said he was frustrated by the fact that the paper repeatedly cites the authors’ own work and makes no mention of earlier contributions to deep learning by other pioneers:

  • The authors, who consider themselves AI pioneers, do not mention Alexey GrigorevichIvakhnenko, the father of deep learning, who published the first paper on a general deep learning algorithm for deep networks back in 1965. A paper in 1971 proposed an 8-layer deep neural network.

  • The article mentions back propagation (BP), but it cites its own papers and does not mention the work of the inventors and early pioneers of back propagation at all. In fact, the earliest back propagation models were developed in the 1960s and 1970s.

  • As for feedforward neural networks, Review said that researchers in CIFAR lab made efforts in 2006 to lead to the revival of FNN, which is another boast and misleading. In fact, researchers have been using Ivakhnenko for decades.

  • In this paper, the author’s own research was quoted when referring to the UNsupervised pre-training FNN, but the unsupervised pre-training RNN proposed by Schmidhuber in 1992-1993 was not called RNN at that time, but the principle and idea were the same.

  • When it comes to the profound impact of unsupervised learning on the revival of deep learning, only the authors’ own results are cited.

  • Similarly, in the section of convolutional neural network, the article mentioned “pooling”, but did not mention the pioneers who proposed the maximum pooling technology and so on.

In short, Schmidhuber argues that the “three giants of deep learning” who won the Turing Prize this year have become thieves who flatter each other and suppress the old masters by virtue of their social status. The operation of this article is really just very.

Schmidhuber also said that Hinton and LeCun became famous because they were supported by Google and Facebook.

LeCun later wrote in an email response: “Jurgen is obsessed with recognition and always says he doesn’t get as much as he deserves. Almost habitually, he would stand up at the end of every speech and claim credit for what had just been proposed, which, on the whole, did not make sense.”

 

Schmidhuber and The Big Three are Schmidhuber.

In 1997, Jurgen Schmidhuber and Sepp Hochreiter published a paper on a recursive neural network known as the Long and Short-term Memory Network (LSTM).

In 2015, LSTM was used in a new implementation of speech recognition in Google’s smartphone software. Google also uses LSTM for its smart assistant Allo and Google Translate. Apple later used LSTM in the “Quicktype” feature of the iPhone and Siri. Amazon’s Alexa also uses LSTM. In 2017, Facebook used its LSTM network to perform approximately 4.5 billion automated translations per day, and LSTM is one of the most widely commercialized AI technologies.

In addition to LSTM, in 2011 Jurgen Schmidhuber worked with his post-doctoral students to achieve significant acceleration of CNN(Convolutional Neural Networks) on Gpus, which is now at the heart of the computer vision field.

When this year’s Turing Award was announced, Jurgen Schmidhuber, who developed LSTM, also received the award.

Well deserved congratulations to all three. However, Jurgen Schmidhuber’s exclusion was surprising and wrong.

Professor Pei Jian, vice President of JINGdong Group, professor of School of Computing Science, Department of Statistics and Actuarial Science, Simon Fraser University, Canada, Canadian First-class Research Professor, ACM Fellow, IEEE Fellow, ACM SIGKDD Chair, told Xinzhiyuan: After the Turing prize was announced, the question was often asked: why didn’t the other laureates, who had made outstanding contributions to this direction and this field, even more than one of them? And then there’s all kinds of speculation.

“It is my personal understanding that each Turing Award is given to an individual or a team, rather than to multiple pioneers in a relatively separate field. “The ACM A.M. Turing Award is an annual prize given by The Association for Computing Machinery (ACM) to an individual Selected for contributions’ of lasting and major technical importance to the computer field ‘.
In the history of the Turing Prize, many scientists and teams who made outstanding contributions in the same field have won the prize successively, such as computational complexity theory and database theory.

So, you do not worry, do not see the state of mind to see the Turing award.

OpenCV creator and AI scientist Gary Bradski said of Schmidhuber: “He did a lot of groundbreaking work, but he wasn’t the one who made it popular. It’s like the Vikings were the first to discover America, but Columbus is the one who will be remembered forever.”

Jurgen Schmidhuber and HLB have their own achievements, but the gods always have something in common, that is persistence.

Hinton was convinced in college that neural networks were the future, and has been for three decades.

Jurgen Schmidhuber, who was born in 1963, was convinced that universal ARTIFICIAL intelligence would be realized when he was 15 years old, just as China was opening up and reform.

Jurgen Schmidhuber as an infant, with his father on the left

“As a teenager, I realized that the most important thing people can do is to build something that learns to be smarter than humans.”

As a young man, Jurgen Schmidhuber told his brother that man could reconstruct the brain atom by atom, and that he could replace our slow neurons with copper wires as connections, with a bold imagination. The younger brother initially objected to his brother’s idea that this artificial brain could mimic human emotion and free will. But eventually, “I realized he was right. “

Schmidhuber studied computer science and mathematics after high school in 1981 and served in the West German army for 15 months. He showed a maverick personality during his military service. He didn’t like being bossed around, especially when asked to do something he didn’t think was useful.

Schmidhuber’s online resume meticulously charts his academic journey, including things like “Caltech rejected him for a postdoctoral fellowship,” and his personal web page still features Schmidhuber.

Schmidhuber has been pursuing general AI for more than 40 years and dreamed of a utopia of intelligent machine labor, so in 1988 he donated millions of dollars to start the Dalle Molle INSTITUTE for ARTIFICIAL Intelligence in Switzerland. Its partnership with the local university, along with a steady stream of government funding, has helped turn the town into a paradise center for ARTIFICIAL intelligence.

The author of Silicon Valley’s Iron Man wrote a Feature by Jurgen Schmidhuber in May 2018, titled “The Man THE AI Community Wants to Forget.”

Schmidhuber remains largely unknown outside of most academia, the article notes. Mainly because he was disliked by his peers in the academic community, many of whom judged him as selfish, cunning and painful.

Schmidhuber frequently challenged researchers at academic journals and conferences, interrupting speeches to demand that peers would admit they borrowed or even stole his idea. The verb “Schmidhubered” was coined to describe anyone who was attacked by others.

Schmidhuber was also sidelined partly because of his institute’s isolated location in the Alps, far from big tech companies.

Schmidhuber also launched a Swiss start-up, Nnaisense, in 2013, with a mission to bring universal AI to life and influence DeepMind.

A principal member of Nnaisense

Schmidhuber’s students include DeepMind co-founder Shane Legg and Daan Wierstra, one of its first employees. Some of his other Ph.D. students also joined DeepMind.

Jurgen Schmidhuber has said that he decided from the age of 15 that he would create robots that were smarter than humans and retire, and he has kept the same idea to this day.

Self-aware or “conscious machines” would soon appear, he thought. This view exacerbated his peers’ disdain for him. The debate needs to be asked: is AI an engineering discipline, or a “god making movement” to create new super-intelligent beings?

Schmidhuber took a firm stand on the idea of creating gods. He believed that the basic concepts of these technologies already existed and that human consciousness was not magical. He believed that the consciousness of machines would emerge from more powerful computers and algorithms that were very close to those he had already designed.

Behind this belief lies his unwavering belief that we live in a Matrix-style computer simulation.

“When I was growing up, I kept asking myself, what’s the biggest impact I could have?” “It became clear that I was going to build something smarter than myself, and this thing was going to build something smarter, and so on, and eventually it was going to take over and change the universe and make it intelligent,” Dr. Schmidhuber recalled.

Today, he is no longer confused about whether such a machine will ever appear, saying it will happen soon, as long as there is a big leap in computing power.

References:

https://www.bloomberg.com/news/features/2018-05-15/google-amazon-and-facebook-owe-j-rgen-schmidhuber-a-fortune

https://www.quora.com/Was-J%C3%BCrgen-Schmidhuber-right-when-he-claimed-credit-for-GANs-at-NIPS-2016

https://www.inverse.com/article/25521-juergen-schmidhuber-ai-consciousness

Highlights of 2019 New Intelligence AI Technology Summit

On March 27, 2019, Xinzhiyuan gathered its AI power again and held the 2019 New Zhiyuan AI Technology Summit in Beijing Taifu Hotel. With the theme of “Intelligent Cloud • Core World”, the summit focuses on the development of intelligent cloud and AI chips, reshaping the future AI world pattern.

At the same time, New Intelligence will release a number of AI white papers at the summit, focusing on the innovation and vitality of the industrial chain, commenting on the influence of AI unicorns, and helping China surpass in the world-class AI competition.

Highlights:

Iqiyi (all day) :

https://live.iqiyi.com/s/19rsj6q75j.html

Headline technology (am) :

m.365yg.com/i6672243313506044680/

Headline Technology (PM) :

m.365yg.com/i6672570058826550030/