Former Google CEO Eric Schmidt Warns of AI Hacking Risks and Potential for Misuse
Former Google CEO Eric Schmidt yesterday cautioned that artificial intelligence systems are vulnerable to hacking and could be repurposed for malicious activities, raising concerns about a potential “proliferation problem.”
Speaking at the Sifted Summit on October 8, Schmidt, who led Google from 2001 to 2011, highlighted the possibility of bad actors gaining control of AI models and removing safety protocols. “There’s evidence that you can take models, closed or open, and you can hack them to remove their guardrails,” he said. “A bad example would be they learn how to kill someone.” He noted that while major companies have implemented safeguards, these can be reverse-engineered, citing examples like the “DAN” alter-ego created to bypass ChatGPT’s safety instructions in 2023. This vulnerability is particularly concerning as AI becomes increasingly integrated into critical infrastructure and decision-making processes.
Despite these risks, Schmidt expressed overall optimism about AI’s potential, stating the technology is “underhyped” and will likely surpass human capabilities over time. He referenced his work with the late Henry Kissinger, concluding that the emergence of AI represents a significant moment for humanity. For more on the development of AI safety measures, see resources from the OpenAI safety team. He also addressed concerns about an AI bubble, suggesting that the substantial investment in AI firms indicates a belief in long-term economic returns, similar to the growth seen in the tech sector. You can learn more about the current state of AI investment here.
Schmidt emphasized the need for a “non-proliferation regime” to address the dangers of AI, but acknowledged that one does not yet exist. Officials are continuing to evaluate potential regulatory frameworks to mitigate these risks and ensure responsible AI development.
Google’s former CEO Eric Schmidt spoke at the Sifted Summit on Wednesday 8, October.
Bloomberg | Bloomberg | Getty Images
Google‘s former CEO Eric Schmidt has issued a stark reminder about the dangers of AI and how susceptible it is to being hacked.
Schmidt, who served as Google’s chief executive from 2001 to 2011, warned about “the bad stuff that AI can do,” when asked whether AI is more destructive than nuclear weapons during a fireside chat at the Sifted Summit
“Is there a possibility of a proliferation problem in AI? Absolutely,” Schmidt said Wednesday. The proliferation risks of AI include the technology falling into the hands of bad actors and being repurposed and misused.
“There’s evidence that you can take models, closed or open, and you can hack them to remove their guardrails. So in the course of their training, they learn a lot of things. A bad example would be they learn how to kill someone,” Schmidt said.
“All of the major companies make it impossible for those models to answer that question. Good decision. Everyone does this. They do it well, and they do it for the right reasons. There’s evidence that they can be reverse-engineered, and there are many other examples of that nature.”
AI systems are vulnerable to attack, with some methods including prompt injections and jailbreaking. In a prompt injection attack, hackers hide malicious instructions in user inputs or external data, like web pages or documents, to trick the AI into doing things it’s not meant to do — such as sharing private data or running harmful commands
Jailbreaking, on the other hand, involves manipulating the AI’s responses so it ignores its safety rules and produces restricted or dangerous content.
In 2023, a few months after OpenAI’s ChatGPT was released, users employed a “jailbreak” trick to circumvent the safety instructions embedded in the chatbot.
This included creating a ChatGPT alter-ego called DAN, an acronym for “Do Anything Now,” which involved threatening the chatbot with death if it didn’t comply. The alter-ego could provide answers on how to commit illegal activities or list the positive qualities of Adolf Hitler.
Schmidt said that there isn’t a good “non-proliferation regime” yet to help curb the dangers of AI.
AI is ‘underhyped’
Despite the grim warning, Schmidt was optimistic about AI more broadly and said the technology doesn’t get the hype it deserves.
“I wrote two books with Henry Kissinger about this before he died, and we came to the view that the arrival of an alien intelligence that is not quite us and more or less under our control is a very big deal for humanity, because humans are used to being at the top of the chain. I think so far, that thesis is proving out that the level of ability of these systems is going to far exceed what humans can do over time,” Schmidt said.
“Now the GPT series, which culminated in a ChatGPT moment for all of us, where they had 100 million users in two months, which is extraordinary, gives you a sense of the power of this technology. So I think it’s underhyped, not overhyped, and I look forward to being proven correct in five or 10 years,” he added.
His comments come amid growing talk of an AI bubble, as investors pour money into AI-focused firms and valuations look stretched, with comparisons being made to the dot-com bubble collapse of the early 2000s.
Schmidt said he doesn’t think history will repeat itself, however.
“I don’t think that’s going to happen here, but I’m not a professional investor,” he said.
“What I do know is that the people who are investing hard-earned dollars believe the economic return over a long period of time is enormous. Why else would they take the risk?”
