Random Image Display on Page Reload

A Letter Prompted Talk of AI Doomsday. Many Who Signed Weren’t Actually AI Doomers

Aug 17, 2023 12:00 PM

A Letter Prompted Talk of AI Doomsday. Many Who Signed Weren't Actually AI Doomers

In March a viral letter called for a pause on AI development, warning that algorithms could outsmart humanity—but many experts who signed on did not believe the technology poses an existential risk.

Push to exit green button mounted on white wall

Photograph: ANNVIPS/Getty Images

This March, nearly 35,000 AI researchers, technologists, entrepreneurs, and concerned citizens signed an open letter from the nonprofit Future of Life Institute that called for a “pause” on AI development, due to the risks to humanity revealed in the capabilities of programs such as ChatGPT.

“Contemporary AI systems are now becoming human-competitive at general tasks, and we must ask ourselves … Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us?”

I could still be proven wrong, but almost six months later and with AI development faster than ever, civilization hasn’t crumbled. Heck, Bing Chat, Microsoft’s “revolutionary,” ChatGPT-infused search oracle, hasn’t even displaced Google as the leader in search. So what should we make of the letter and similar sci-fi warnings backed by worthy names about the risks posed by AI?

Two enterprising students at MIT, Isabella Struckman and Sofie Kupiec, reached out to the first hundred signatories of the letter calling for a pause on AI development to learn more about their motivations and concerns. The duo’s write-up of their findings reveals a broad array of perspectives among those who put their name to the document. Despite the letter’s public reception, relatively few were actually worried about AI posing a looming threat to humanity itself.

Many of the people Struckman and Kupiec spoke to did not believe a six-month pause would happen or would have much effect. Most of those who signed did not envision the “apocalyptic scenario” that one anonymous respondent acknowledged some parts of the letter evoked.

A significant number of those who signed were, it seems, primarily concerned with the pace of competition between Google, OpenAI, Microsoft, and others, as hype around the potential of AI tools like ChatGPT reached giddy heights. Google was the original developer of several algorithms key to the chatbot’s creation, but it moved relatively slowly until ChatGPT-mania took hold. To these people, the prospect of companies rushing to release experimental algorithms without exploring the risks was a cause for concern—not because they might wipe out humanity but because they might spread disinformation, produce harmful or biased advice, or increase the influence and wealth of already very powerful tech companies.

Some signatories also worried about the more distant possibility of AI displacing workers at hitherto unseen speed. And a number also felt that the statement would help draw the public’s attention to significant and surprising leaps in the performance of AI models, perhaps pushing regulators into taking some sort of action to address the near-term risks posed by advances in AI.

Most Popular

Back in May, I spoke to a few of those who signed the letter, and it was clear that they did not all agree entirely with everything it said. They signed out of a feeling that the momentum building behind the letter would draw attention to the various risks that worried them, and was therefore worth backing.

But perhaps it was a mistake to try to cover so many issues potentially raised by existing and recently developed AI in a letter that would inevitably be defined by its most outlandish and scary claim. Some AI researchers have spent the past few yearswarning presciently about the more immediate societal problems that large language models could cause, including exacerbating ingrained biases. Their concerns were barely audible amid the furor the letter prompted around doomsday scenarios about AI. The prominence of that apocalyptic strand of thinking was reinforced by a follow-up statement in May, also signed by many high-profile AI researchers, that compared the extinction threat of AI to that of nuclear weapons and pandemics.

Nirit Weiss-Blatt, author of The Techlash and Tech Crisis Communication, who reviewed the MIT paper before its publication, says the letter and statement ended up serving the interests of the tech firms building cutting-edge AI, because the focus on far-off worst-case scenarios makes regulators believe the technology is both incredibly valuable and hard to handle. Many of the professors who signed the letter were not thinking about AI as an existential risk as they did so, Weiss-Blatt says. “But they lent their name to the extreme AI doomers. That’s the real misinformation here.”

In the end, the letter asking for a pause on AI development may have done the opposite of what many of those who signed wanted. By making discussion of doomsday scenarios more prominent, the letter made it harder for concerns about less-than-superintelligent machines to win notice or inspire action.

Updated 8-17-2023, 1.50 pm EDT: Weiss-Blatt thinks most professors who signed weren't thinking about existential risk, not all.

Get More From WIRED

Will Knight is a senior writer for WIRED, covering artificial intelligence. He writes the Fast Forward newsletter that explores how advances in AI and other emerging technology are set to change our lives—sign up here. He was previously a senior editor at MIT Technology Review, where he wrote about fundamental… Read more
Senior Writer

More from WIRED

A New Attack Impacts Major AI Chatbots—and No One Knows How to Stop It

Researchers found a simple way to make ChatGPT, Bard, and other chatbots misbehave, proving that AI is hard to tame.

Will Knight

Meta’s News Block Causes Chaos as Canada Burns

News has been removed from Facebook and Instagram after the Canadian government told Meta to pay publishers, leaving consumers unable to access vital information during wildfires.

Tracey Lindeman

In a World of Fakes, Trump’s Real Mug Shot Matters

The first booking photo of a US president stands out among a sea of photoshops and AI-generated images online.

Amanda Hoover

Trump’s Prosecution Is America’s Last Hope

Social norms—not laws—are the underlying fabric of democracy. The Georgia indictment against Donald Trump is the last tool remaining to repair that which he’s torn apart.

Dell Cameron

AI Can’t Read Books. It’s Reviewing Them Anyway

ChatGPT-powered avatars recently gushed over a new book—unconvincingly. But if AI develops actual opinions, authors might be in for a shock.

Steven Levy

The Myth of ‘Open Source’ AI

A new analysis shows that “open source” AI tools like Llama 2 are still controlled by big tech companies in a number of ways.

Will Knight

Meta Just Released a Coding Version of Llama 2

Code Llama may spur a new wave of experimentation around AI and programming—but it will also help Meta.

Will Knight

Nvidia Chip Shortages Leave AI Startups Scrambling for Computing Power

Trimming profits, delaying launches, begging friends. Companies are going to extreme lengths to make do with shortages of GPUs, the chips at the heart of generative AI programs.

Paresh Dave

Credit belongs to : www.wired.com

Check Also

Welcome to the Age of Technofeudalism

By Morgan Meaker Business Apr 9, 2024 2:00 AM Welcome to the Age of Technofeudalism …