{"id":1192,"date":"2023-07-05T12:06:00","date_gmt":"2023-07-05T10:06:00","guid":{"rendered":"https:\/\/www.pixel-service.com\/?p=1192"},"modified":"2023-07-05T13:57:27","modified_gmt":"2023-07-05T11:57:27","slug":"geoffrey-hinton-tells-us-why-hes-now-scared-of-the-tech-he-helped-build","status":"publish","type":"post","link":"https:\/\/www.pixel-service.com\/?p=1192&lang=en","title":{"rendered":"Geoffrey Hinton tells us why he\u2019s now scared of the tech he helped build"},"content":{"rendered":"

LINDA NYLIND \/ EYEVINE VIA REDUX<\/p>\n

I met Geoffrey Hinton at his house on a pretty street in north London just four days before the bombshell announcement that he is quitting Google. Hinton is a\u00a0pioneer of deep learning<\/a>\u00a0who helped develop some of the most important techniques at the heart of modern artificial intelligence, but after a decade at Google, he is\u00a0stepping down<\/a>\u00a0to focus on new concerns he now has about AI.<\/p>\n

Stunned by the capabilities of new\u00a0large language models like GPT-4<\/a>, Hinton wants to raise public awareness of the serious risks that he now believes may accompany the technology he ushered in.<\/p>\n

At the start of our conversation, I took a seat at the kitchen table, and Hinton started pacing. Plagued for years by chronic back pain, Hinton almost never sits down. For the next hour I watched him walk from one end of the room to the other, my head swiveling as he spoke. And he had plenty to say.<\/p>\n

The 75-year-old computer scientist, who was a joint recipient with\u00a0Yann LeCun<\/a>\u00a0and Yoshua Bengio of the 2018 Turing Award for his work on deep learning, says he is ready to shift gears. \u201cI’m getting too old to do technical work that requires remembering lots of details,\u201d he told me. \u201cI\u2019m still okay, but I\u2019m not nearly as good as I was, and that\u2019s annoying.\u201d<\/p>\n

But that\u2019s not the only reason he\u2019s leaving Google. Hinton wants to spend his time on what he describes as \u201cmore philosophical work.\u201d And that will focus on the small but\u2014to him\u2014very real danger that AI will turn out to be a disaster.<\/p>\n

Related Story<\/strong><\/p>\n<\/div>

<\/span><\/div>

Deep learning pioneer Geoffrey Hinton has quit Google<\/a><\/u><\/p>\n

Hinton will be speaking at EmTech Digital on Wednesday.<\/p>\n

Leaving Google will let him speak his mind, without the self-censorship a Google executive must engage in. \u201cI want to talk about AI safety issues without having to worry about how it interacts with Google\u2019s business,\u201d he says. \u201cAs long as I\u2019m paid by Google, I can\u2019t do that.\u201d<\/p>\n

That doesn\u2019t mean Hinton is unhappy with Google by any means. \u201cIt may surprise you,\u201d he says. \u201cThere\u2019s a lot of good things about Google that I want to say, and they\u2019re much more credible if I\u2019m not at Google anymore.\u201d<\/p>\n

Hinton says that the new generation of large language models\u2014especially GPT-4, which OpenAI released in March\u2014has made him realize that machines are on track to be a lot smarter than he thought they\u2019d be. And he\u2019s scared about how that might play out.<\/p>\n

\u201cThese things are totally different from us,\u201d he says. \u201cSometimes I think it\u2019s as if aliens had landed and people haven\u2019t realized because they speak very good English.\u201d<\/p>\n

Foundations<\/p>\n

Hinton is best known for his work on a technique called backpropagation, which he proposed (with a pair of colleagues) in the 1980s. In a nutshell, this is the algorithm that allows machines to learn. It underpins almost all neural networks today, from computer vision systems to large language models.<\/p>\n

It took until the 2010s for the power of neural networks trained via backpropagation to truly make an impact. Working with a couple of graduate students, Hinton showed that his technique was better than any others at getting a computer to identify objects in images. They also trained a neural network to predict the next letters in a sentence, a precursor to today\u2019s large language models.<\/p>\n

One of these graduate students was Ilya Sutskever, who went on to cofound OpenAI and lead the\u00a0development of ChatGPT<\/a>. \u201cWe got the first inklings that this stuff could be amazing,\u201d says Hinton. \u201cBut it\u2019s taken a long time to sink in that it needs to be done at a huge scale to be good.\u201d Back in the 1980s, neural networks were a joke. The dominant idea at the time, known as symbolic AI, was that intelligence involved processing symbols, such as words or numbers.<\/p>\n

But Hinton wasn\u2019t convinced. He worked on neural networks, software abstractions of brains in which neurons and the connections between them are represented by code. By changing how those neurons are connected\u2014changing the numbers used to represent them\u2014the neural network can be rewired on the fly. In other words, it can be made to learn.<\/p>\n

\u201cMy father was a biologist, so I was thinking in biological terms,\u201d says Hinton. \u201cAnd symbolic reasoning is clearly not at the core of biological intelligence.<\/p>\n

\u201cCrows can solve puzzles, and they don\u2019t have language. They\u2019re not doing it by storing strings of symbols and manipulating them. They\u2019re doing it by changing the strengths of connections between neurons in their brain. And so it has to be possible to learn complicated things by changing the strengths of connections in an artificial neural network.\u201d<\/p>\n

A new intelligence<\/p>\n

For 40 years, Hinton has seen artificial neural networks as a poor attempt to mimic biological ones. Now he thinks that\u2019s changed: in trying to mimic what biological brains do, he thinks, we\u2019ve come up with something better. \u201cIt\u2019s scary when you see that,\u201d he says. \u201cIt\u2019s a sudden flip.\u201d<\/p>\n

Hinton\u2019s fears will strike many as the stuff of science fiction. But here\u2019s his case.<\/p>\n

As their name suggests, large language models are made from massive neural networks with vast numbers of connections. But they are tiny compared with the brain. \u201cOur brains have 100 trillion connections,\u201d says Hinton. \u201cLarge language models have up to half a trillion, a trillion at most. Yet GPT-4 knows hundreds of times more than any one person does. So maybe it\u2019s actually got a much better learning algorithm than us.\u201d<\/p>\n

Compared with brains, neural networks are widely believed to be bad at learning: it takes vast amounts of data and energy to train them. Brains, on the other hand, pick up new ideas and skills quickly, using a fraction as much energy as neural networks do.<\/p>\n

\u201cPeople seemed to have some kind of magic,\u201d says Hinton. \u201cWell, the bottom falls out of that argument as soon as you take one of these large language models and train it to do something new. It can learn new tasks extremely quickly.\u201d<\/p>\n

Hinton is talking about \u201cfew-shot learning,\u201d in which pretrained neural networks, such as large language models, can be trained to do something new given just a few examples. For example, he notes that some of these language models can string a series of logical statements together into an argument even though they were never trained to do so directly.<\/p>\n

Compare a pretrained large language model with a human in the speed of learning a task like that and the human\u2019s edge vanishes, he says.<\/p>\n

Related Story<\/strong><\/p>\n<\/div>

<\/span><\/div>

Geoffrey Hinton has a hunch about what\u2019s next for AI<\/a><\/u><\/p>\n

A decade ago, the artificial-intelligence pioneer transformed the field with a major breakthrough. Now he\u2019s working on a new imaginary system named GLOM.<\/p>\n

What about the fact that large language models make so much stuff up? Known as \u201challucinations\u201d by AI researchers (though Hinton prefers the term \u201cconfabulations,\u201d because it\u2019s the correct term in psychology), these errors are often seen as a fatal flaw in the technology. The tendency to generate them makes chatbots untrustworthy and, many argue, shows that these models have no true understanding of what they say.<\/p>\n

Hinton has an answer for that too: bullshitting is a feature, not a bug. \u201cPeople always confabulate,\u201d he says. Half-truths and misremembered details are hallmarks of human conversation: \u201cConfabulation is a signature of human memory. These models are doing something just like people.\u201d<\/p>\n

The difference is that humans usually confabulate more or less correctly, says Hinton. To Hinton, making stuff up isn\u2019t the problem. Computers just need a bit more practice.<\/p>\n

We also expect computers to be either right or wrong\u2014not something in between. \u201cWe don\u2019t expect them to blather the way people do,\u201d says Hinton. \u201cWhen a computer does that, we think it made a mistake. But when a person does that, that\u2019s just the way people work. The problem is most people have a hopelessly wrong view of how people work.\u201d<\/p>\n

Of course, brains still do many things better than computers: drive a car, learn to walk, imagine the future. And brains do it on a cup of coffee and a slice of toast. \u201cWhen biological intelligence was evolving, it didn\u2019t have access to a nuclear power station,\u201d he says.<\/p>\n

But Hinton\u2019s point is that if we are willing to pay the higher costs of computing, there are crucial ways in which neural networks might beat biology at learning. (And it’s worth pausing\u00a0to consider what those costs entail<\/a>\u00a0in terms of energy and carbon.)<\/p>\n

Learning is just the first string of Hinton\u2019s argument. The second is communicating. \u201cIf you or I learn something and want to transfer that knowledge to someone else, we can\u2019t just send them a copy,\u201d he says. \u201cBut I can have 10,000 neural networks, each having their own experiences, and any of them can share what they learn instantly. That\u2019s a huge difference. It\u2019s as if there were 10,000 of us, and as soon as one person learns something, all of us know it.\u201d<\/p>\n

What does all this add up to? Hinton now thinks there are two types of intelligence in the world: animal brains and neural networks. \u201cIt\u2019s a completely different form of intelligence,\u201d he says. \u201cA new and better form of intelligence.\u201d<\/p>\n

That\u2019s a huge claim. But AI is a polarized field: it would be easy to find people who would laugh in his face\u2014and others who would nod in agreement.<\/p>\n

People are also divided on whether the consequences of this new form of intelligence, if it exists, would be beneficial or apocalyptic. \u201cWhether you think superintelligence is going to be good or bad depends very much on whether you\u2019re an optimist or a pessimist,\u201d he says. \u201cIf you ask people to estimate the risks of bad things happening, like what\u2019s the chance of someone in your family getting really sick or being hit by a car, an optimist might say 5% and a pessimist might say it\u2019s guaranteed to happen. But the mildly depressed person will say the odds are maybe around 40%, and they\u2019re usually right.\u201d<\/p>\n

Which is Hinton? \u201cI\u2019m mildly depressed,\u201d he says. \u201cWhich is why I\u2019m scared.\u201d<\/p>\n

How it could all go wrong<\/p>\n

Hinton fears that these tools are capable of figuring out ways to manipulate or kill humans who aren\u2019t prepared for the new technology.<\/p>\n

\u201cI have suddenly switched my views on whether these things are going to be more intelligent than us. I think they\u2019re very close to it now and they will be much more intelligent than us in the future,\u201d he says. \u201cHow do we survive that?\u201d<\/p>\n

He is especially worried that people could harness the tools he himself helped breathe life into to tilt the scales of some of the most consequential human experiences, especially elections and wars.<\/p>\n

\u201cLook, here\u2019s one way it could all go wrong,\u201d he says. \u201cWe know that a lot of the people who want to use these tools are bad actors like Putin or DeSantis. They want to use them for winning wars or manipulating electorates.\u201d<\/p>\n

Hinton believes that the next step for smart machines is the ability to create their own subgoals, interim steps required to carry out a task. What happens, he asks, when that ability is applied to something inherently immoral?<\/p>\n

\u201cDon\u2019t think for a moment that Putin wouldn\u2019t make hyper-intelligent robots with the goal of killing Ukrainians,\u201d he says. \u201cHe wouldn\u2019t hesitate. And if you want them to be good at it, you don\u2019t want to micromanage them\u2014you want them to figure out how to do it.\u201d<\/p>\n

There are already a handful of experimental projects, such as BabyAGI and AutoGPT, that hook chatbots up with other programs such as web browsers or word processors so that they can string together simple tasks. Tiny steps, for sure\u2014but they signal the direction that some people want to take this tech. And even if a bad actor doesn\u2019t seize the machines, there are other concerns about subgoals, Hinton says.<\/p>\n

\u201cWell, here\u2019s a subgoal that almost always helps in biology: get more energy. So the first thing that could happen is these robots are going to say, \u2018Let\u2019s get more power. Let\u2019s reroute all the electricity to my chips.\u2019 Another great subgoal would be to make more copies of yourself. Does that sound good?\u201d<\/p>\n

Maybe not. But Yann LeCun, Meta\u2019s chief AI scientist, agrees with the premise but does not share Hinton\u2019s fears. \u201cThere is no question that machines will become smarter than humans\u2014in all domains in which humans are smart\u2014in the future,\u201d says LeCun. \u201cIt\u2019s a question of when and how, not a question of if.\u201d<\/p>\n

But he takes a totally different view on where things go from there. \u201cI believe that intelligent machines will usher in a new renaissance for humanity, a new era of enlightenment,\u201d says LeCun. \u201cI completely disagree with the idea that machines will dominate humans simply because they are smarter, let alone destroy humans.\u201d<\/p>\n

\u201cEven within the human species, the smartest among us are not the ones who are the most dominating,\u201d says LeCun. \u201cAnd the most dominating are definitely not the smartest. We have numerous examples of that in politics and business.\u201d<\/p>\n

Yoshua Bengio, who is a professor at the University of Montreal and scientific director of the Montreal Institute for Learning Algorithms, feels more agnostic. \u201cI hear people who denigrate these fears, but I don\u2019t see any solid argument that would convince me that there are no risks of the magnitude that Geoff thinks about,\u201d he says. But fear is only useful if it kicks us into action, he says: \u201cExcessive fear can be paralyzing, so we should try to keep the debates at a rational level.\u201d<\/p>\n

Just look up<\/p>\n

One of Hinton\u2019s priorities is to try to work with leaders in the technology industry to see if they can come together and agree on what the risks are and what to do about them. He thinks the international ban on chemical weapons might be one model of how to go about curbing the development and use of dangerous AI. \u201cIt wasn\u2019t foolproof, but on the whole people don\u2019t use chemical weapons,\u201d he says.<\/p>\n

Related Story<\/strong><\/p>\n<\/div>

<\/span><\/div>

AI pioneer Geoff Hinton: \u201cDeep learning is going to be able to do everything\u201d<\/a><\/u><\/p>\n

Thirty years ago, Hinton\u2019s belief in neural networks was contrarian. Now it\u2019s hard to find anyone who disagrees, he says.<\/p>\n

Bengio agrees with Hinton that these issues need to be addressed at a societal level as soon as possible. But he says the development of AI is accelerating faster than societies can keep up. The capabilities of this tech leap forward every few months; legislation, regulation, and international treaties take years.<\/p>\n

This makes Bengio wonder whether the way our societies are currently organized\u2014at both national and global levels\u2014is up to the challenge. \u201cI believe that we should be open to the possibility of fairly different models for the social organization of our planet,\u201d he says.<\/p>\n

Does Hinton really think he can get enough people in power to share his concerns? He doesn\u2019t know. A few weeks ago, he watched the movie\u00a0Don\u2019t Look Up<\/em>, in which an asteroid zips toward Earth, nobody can agree what to do about it, and everyone dies\u2014an allegory for how the world is failing to address climate change.<\/p>\n

\u201cI think it\u2019s like that with AI,\u201d he says, and with other big intractable problems as well.\u00a0\u201cThe US can\u2019t even agree to keep assault rifles out of the hands of teenage boys,\u201d he says.<\/p>\n

Hinton\u2019s argument is sobering. I share his bleak assessment of people\u2019s collective inability to act when faced with serious threats. It is also true that AI risks causing real harm\u2014upending the job market, entrenching inequality, worsening sexism and racism, and more. We need to focus on those problems. But I still can\u2019t make the jump from large language models to robot overlords. Perhaps I\u2019m an optimist.<\/p>\n

When Hinton saw me out, the spring day had turned gray and wet. \u201cEnjoy yourself, because you may not have long left,\u201d he said. He chuckled and shut the door.<\/p>\n<\/div>

Geoffrey Hinton tells us why he\u2019s now scared of the tech he helped build<\/strong><\/div>\n
Will Douglas Heaven<\/div>\n
May 2, 2023<\/div>\n<\/div><\/div>