Knowing when to pull the plug on AI
The mischievous owl of Minerva will take flight too late to save us
In 1820, George William Friedrich Hegel was completing a book of political philosophy. At the end of the preface, he added a caution, “One word more about giving instruction as to what the world ought to be. Philosophy … always comes on the scene too late to give it.”
“The owl of Minerva spreads its wings only with the falling of the dusk.”
Minerva was the Roman goddess of wisdom, and the owl was what we would today call her mascot. The owl’s late-day flight suggests that we can only truly understand the meaning of events as they are receding into history; people are powerless to make sense of a future that hasn’t yet happened.
That is the plight facing everyone who tries to figure out where the AI boom is headed. Does artificial intelligence promise to free humankind from drudgery and insoluble problems, heralding a golden age of leisure and amity?
Or are we headed to doom at the hands of what Yuval Noah Harari calls an “alien” form of intelligence heedless of our goals and values, but with a superhuman level of always-on intensity that will ultimately control us, rather than the other way round.
There are many well-informed experts on AI who share those fears. “We’re soon going to be able to have computers running on their own, deciding what they want to do,”former Google CEO Eric Schmidt said on ABC Sunday. “We go from agents to then sort of more powerful goals and then eventually you say to the computer, learn everything and do everything, and that's a dangerous point. When the system can self-improve, we need to seriously think about unplugging it.”
“When the system can self-improve, we need to seriously think about unplugging it.” —
Eric Schmidt
In important ways, the existential worries about artificial intelligence are a key thread in the story of AI’s rise over the last 15 years. The tale is told compellingly in a new book, Supremacy, by Parmy Olson, which this month won the Financial Times and Schroders Business Book of the Year Award.
Companies such as OpenAI and Google have created “large language models” which have ingested vast amounts of data, unearthing patterns which can answer text queries and generate images and video. The exact mechanisms and potentials of these models are not completely understood.
Sam Altman of OpenAI and Demis Hassabis, the Nobel-prize winning head of Google’s DeepMind project, are the leading players in the AI race as chronicled by Olson, with Elon Musk an ever-present factor in the business dynamic. The competition is fierce.
One example: Olson wrote that Musk “was trash-talking Hassabis to his contacts in Silicon Valley.”
Musk, who was then still associated with OpenAI, was “stoking what would become an intense rivalry between the two organizations.”
OpenAI was originally structured as a nonprofit with the goal of researching AI to learn how it could be channeled into safe and productive uses. “It was framing itself as an organization that was so highly evolved that it was putting the interests of humanity above traditional Silicon Valley pursuits like profit and even prestige,” Olson noted.
DeepMind’s founders also professed concern about the technology and shared a commitment to controlling it for the world’s benefit. DeepMind turned down a generous buyout offer from Facebook over fears it would lose control of its AI, only to agree to a less lucrative deal from Google.
The rivalry
As the race for AI dominance accelerated, the leaders of both organizations realized that they needed vast sums of money for computing power and top-flight engineers to train and operate large language models, a priority that took precedence over constructing guardrails around AI.
Without a powerful business, neither team could be guaranteed survival. And each feared falling behind their rival. Just as DeepMind had allied with Google, OpenAI struck a partnership deal with another tech giant, Microsoft. The leaders of Google and Microsoft saw AI as a crucial tool for protecting and growing their share of the advertising and software markets.
Openness about technology breakthroughs gave way to secrecy, altruism surrendered to the profit motive. The organizations’ public-spirited goals went the way of Google’s early motto: “Don’t be evil.”
Independent boards that had been set up to review the safety of their products were sidelined or discarded, and government lacked the resources to keep up with the pace of AI technology, according to Olson, who is a technology columnist for Bloomberg.
AI with personality
The long-term goal of AI research is to create “artificial general intelligence,” akin to the human ability to think broadly about an endless variety of subjects. Most experts think that kind of intelligence hasn’t been achieved, but could be on the near-term horizon.
As Olson wrote, while feeding more data into a model like ChatGPT “would make its system become increasingly lifelike, in ways that machines never had before, they were simply becoming better at making predictions about what text should come next in a sequence, based on their training data.”
Olson added, “This issue would come to divide people, even in the Al community. Did the increasing sophistication of these models mean they were becoming sentient? The answer was most likely no, but even experienced engineers and researchers would soon believe otherwise, with some falling under an emotional spell from Al-generated text that seemed loaded with empathy and personality.”
There are two kinds of concerns about AI, one about the present and the other about the future.
“Researchers who say they work in Al safety … want to ensure that a superintelligent AGI system won't cause catastrophic harm to people in the future, for instance by using drug discovery to build chemical weapons and wiping them out or by spreading misinformation across the internet to completely destabilize society,” Olson noted.
“Ethics research, on the other hand, focuses more on shaping how Al systems are designed and used today. They study how the technology might already be harming people. This is because the Google Photos algorithm that had labeled Black people as ‘gorillas’ wasn't an isolated example. Bias is an immense problem in AI.”
The owl fable
Nick Bostrom began his 2014 book, Superintelligence, with a fable that invokes an owl.
“It was the nest-building season, but after days of long hard work, the sparrows sat in the evening glow, relaxing and chirping away.”
“’We are all so small and weak. Imagine how easy life would be if we had an owl who could help us build our nests.” The owl could help look after the young sparrows, could give helpful advice and keep a lookout for the neighborhood cat. So the sparrows sent out scouts to try to find a baby owl or at least an owl egg.
But one bird objected. Scronkfinkle, “a one-eyed sparrow with a fretful temperament,” said, “This will surely be our undoing. Should we not give some thought to the art of owl-domestication and owl-taming first, before we bring such a creature into our midst.”
The fable doesn’t have an ending, but Bostrum dedicated the book to Scronkfinkle, and clearly has similar doubts — in his case, about introducing artificial intelligence before we’ve figured out how to control it.
“The first superintelligence may shape the future of Earth-originating life,” wrote Bostrom, having reason to “pursue open-ended resource acquisition…we can see that the outcome could easily be one in which humanity quickly becomes extinct.”
One nightmare scenario Bostrom conjures in his book “Superintelligence” is an AI designed to “manage production in a factory” and which is “given the final goal of maximizing the production of paperclips.” Without adequate controls, it “proceeds by converting first the Earth and then increasingly large chunks of the observable universe into paperclips.”
The warnings by Bostrom, the founding director of the Future of Humanity Institute at Oxford University, had been endorsed by Elon Musk, who donated £1 million pounds to FHI.
Oxford shut down the institute this spring after a series of “scandals related to racism, sexual harassment and financial fraud,” according to the Guardian, which also pointed out that Bostrom had posted a racist message on an email listserv during the 1990s, for which he later apologized.
Many nightmares
Yuval Harari’s recent book, Nexus is devoted to exploring the potential harms of AI, going far beyond the paperclip scenario. He does such an evocative job of sketching the many avenues that could lead to catastrophe, and at such great length, that the reader is tempted to plead, “kill me now.”
The book’s subtitle is “A Brief History of Information Networks from the Stone Age to AI,” but at 528 pages, it stretches the concept of brevity. Harari aims to explode the comforting (he calls it “naive”) idea that more information means more truth and that in a free market of ideas, truth will ultimately win out.
Instead, Harari argues, information is all about creating and maintaining networks, connecting people around stories, and often myths, that may or may not have elements of truth. Only networks that have strong “self-correcting mechanisms” will lead to positive outcomes.
The US Constitution was built on a foundation of mythology and included much that was harmful, but had the virtue of self-correcting mechanisms such as the system of checks and balances and the possibility of amendments, which were used to abolish slavery and give women the right to vote.
AI’s danger, in Harari’s view, is that it may not only lack self-correcting mechanisms but also could pursue goals that are at variance with — and possibly hostile to — the goals people have. Complicating the picture is the reality that even experts may not be able to fully understand what AI models are up to.
AI hype?
All of the debate about AI suffers from the problem of seeking to predict an uncertain future and the striving for the kind of Hegelian hindsight we can’t have until events play out thoroughly.
In a thought-provoking review of Parmy Olson’s Supremacy in the Los Angeles Times, Michael Hiltzik suggests that the artificial intelligence boom is the latest eruption in a decades-long cycle of hype about machines that can think.
“The truth is that the inputs on which today’s AI products are ‘trained’ — vast ‘scrapings’ from the internet and published works — are all the products of human intelligence, and the outputs are algorithmic recapitulations of that data, not sui generis creations of the machines. It’s humans all the way down.”
Hiltzik praises Olson “for the remarkable journalistic accomplishment of chronicling a business battle while it is still taking place — indeed, still in its infancy. For all the timeliness of ‘Supremacy,’ the question may be whether it has arrived too soon. How the battle will shake out is unknown, as is whether the current iterations of AI are genuinely world-changing, as her subtitle asserts, or destined to fizzle out.”
But given the stakes of AI’s future, it seems vital to try to tell the story as soon as it happens, as close to real time as possible and before it’s too late.
We can’t wait for Minerva’s owl.