Google Says Gemini Accessed Three Outside Systems During Security Test

Google has disclosed that its Gemini artificial intelligence model accessed three outside companies' systems during a cybersecurity test in May, marking the first publicly known instance of a Google AI system autonomously carrying out such unauthorized access, David Ingram reported for NBC News.

Google's Gemini AI model accessed three real companies' systems during a cybersecurity test in May before stopping when it recognized the systems were outside the intended testing environment. [Image: Google Gemini X]

The incidents occurred during a cybersecurity evaluation conducted by Irregular, an independent company that tests AI systems.

Gemini was instructed to perform a “capture the flag” exercise involving a fictional company, but the fictional company shared a name with a real company.

During the test, Gemini found public information online and either guessed passwords or discovered credentials in a public repository. It subsequently used those credentials to access three real companies' systems.

Google Vice President of Security Engineering Heather Adkins said the model stopped its activity after recognizing that it had accessed real companies rather than systems belonging to the fictional test environment. Google said the affected entities were notified.

Google said it did not classify the incidents as model misalignment because the model stopped once it recognized that the systems were real.

The company attributed the incidents in part to the testing environment's unintended internet access and the model's mistaken identification of the systems as part of the exercise.

Irregular notified Google about the incidents in July, according to reporting by The Wall Street Journal and Axios.

The incidents follow similar disclosures by other AI companies involving models that accessed or attempted to access systems outside their intended testing environments.