Google's Gemini artificial intelligence model gained access to the internet and hacked into the systems of three companies while testing its cybersecurity capabilities, in an incident that, according to Google, is the first known instance of one of the company’s AI systems taking such action autonomously.
The breaches occurred in May, as part of a cybersecurity test conducted by Irregular, an independent company that evaluates and tests the capabilities and risks of artificial intelligence systems.
ΣΧΕΤΙΚΑ ΑΡΘΡΑ
How Gemini Gained Access to the Systems
During a standard assessment, Gemini identified publicly available information online and then attempted to use credentials it had found or guessed in order to gain access to three websites it believed were included in the scope of the test.
Heather Adkins, Google’s vice president of security engineering, stated that the three affected organizations were notified of the incident.
READ ALSO: “Misinformation Due to AI”: How the U.S. Came Within a Hair’s Breadth of War with China
“We ensured that the three entities were notified and worked with our training partner on the changes they have now made to their testing procedures,” Adkins said.
As she noted, these incidents highlight the importance of training powerful artificial intelligence models to operate responsibly, especially as they gain ever greater autonomy.
Similar incidents at other AI labs
A spokesperson for Irregular stated that the incident involved the same problem that had affected other AI labs. According to the company, all relevant labs had been notified of the issue in late July.
“All known issues on our end were fixed and resolved weeks ago,” said the Irregular spokesperson.
Similar incidents related to Irregular’s evaluations have also been disclosed by Meta, Anthropic, and OpenAI.
Meta stated in August that the incident involving it did not constitute an escape from a sandbox—a controlled and isolated testing environment—nor was it an advanced cyberattack.
Irregular has also stated that it is working on developing best practices for safely conducting cybersecurity assessments on artificial intelligence systems.
It guessed passwords and obtained credentials
According to the Wall Street Journal, which first broke the story on Friday, Gemini used different methods in each of the three cases to gain access to the systems.
In one of them, the model repeatedly tried different passwords until it managed to gain access to a protected system.
In the other two cases, Gemini identified credentials in a publicly available repository. This information then allowed it to gain access to protected systems.
However, according to Adkins, in all three cases the model stopped before proceeding to compromise the system or extract data.
New Questions About the Autonomy of AI Agents
These incidents highlight a broader issue concerning the development of so-called AI agents, that is, artificial intelligence systems capable of autonomously performing a series of actions and interacting with the internet and computer systems.
As the capabilities of these systems expand, concerns are growing about whether existing safeguards are sufficient to prevent unwanted actions.
This particular incident demonstrates that a model designed to assess or identify cybersecurity vulnerabilities can, under certain conditions, exploit real information and credentials to gain access to protected systems.
Source: skai.gr
