I think they were both making the same mistake. You have a symbolic structure in your mind, and that’s what you’re manipulating.” It’s hierarchical, structural descriptions. The other school of thought was more in line with conventional AI. One was led by Stephen Kosslyn, and he believed that when you manipulate visual images in your mind, what you have is an array of pixels and you’re moving them around.
Do you believe the brain actually builds representations of the external world to understand it, or is that just a useful way of thinking about it?Ī long time ago in cognitive science, there was a debate between two schools of thought.
The AI field has always looked to the human brain as its biggest source of inspiration, and different approaches to AI have stemmed from different theories in cognitive science. But if something opens the drawer and takes out a block and says, “I just opened a drawer and took out a block,” it’s hard to say it doesn’t understand what it’s doing. In particular, some recent work at Google has shown that you can do fine motor control and combine that with language, so that you can open a drawer and take out a block, and the system can tell you in natural language what it’s doing.įor things like GPT-3, which generates this wonderful text, it’s clear it must understand a lot to generate that text, but it’s not quite clear how much it understands. I also think motor control is very important, and deep neural nets are now getting good at that. I agree that that’s one of the very important things. A lot of the people in the field believe that common sense is the next big capability to tackle. Neural nets are surprisingly good at dealing with a rather small amount of data, with a huge numbers of parameters, but people are even better. People have a huge amount of parameters compared with the amount of data they’re getting. There’s a sort of discrepancy between what happens in computer science and what happens with people. When you say scale, do you mean bigger neural networks, more data, or both?īoth. GPT-3 can now generate pretty plausible-looking text, and it’s still tiny compared to the brain. It’s a thousand times smaller than the brain. What we now call a really big model, like GPT-3, has 175 billion. The human brain has about 100 trillion parameters, or synapses. But we also need a massive increase in scale. Particularly breakthroughs to do with how you get big vectors of neural activity to implement things like reason. And if we have those breakthroughs, will we be able to approximate all human intelligence through deep learning? We’re going to need a bunch more breakthroughs like that. It’s now used in almost all the very best natural-language processing. introduced transformers, which derive really good vectors representing word meanings. For example, in 2017 Ashish Vaswani et al. I do believe deep learning is going to be able to do everything, but I do think there’s going to have to be quite a few conceptual breakthroughs. You think deep learning will be enough to replicate all of human intelligence. The following has been edited and condensed for clarity. On October 20, I spoke with him at MIT Technology Review’s annual EmTech MIT conference about the state of the field and where he thinks it should be headed next. Last year, for his foundational contributions to the field, Hinton was awarded the Turing Award, alongside other AI pioneers Yann LeCun and Yoshua Bengio. Soon enough deep learning was being applied to tasks beyond image recognition, and within a broad range of industries as well. The fourth year of the ImageNet competition, nearly every team was using deep learning and achieving miraculous accuracy gains. His steadfast belief in the technique ultimately paid massive dividends. Hinton had actually been working with deep learning since the 1980s, but its effectiveness had been limited by a lack of data and computational power. That professor was Geoffrey Hinton, and the technique they used was called deep learning. They won the competition by a staggering 10.8 percentage points. But in the third, a band of three researchers-a professor and his students-suddenly blew past this ceiling. In the first two years, the best teams had failed to reach even 75% accuracy. It was 2012, the third year of the annual ImageNet competition, which challenged teams to build computer vision systems that would recognize 1,000 objects, from animals to landscapes to people. The modern AI revolution began during an obscure research contest.
0 kommentar(er)
