For the first time, Google has reported spotting and thwarting a zero-day exploit that it believes was developed using artificial intelligence. The exploit targeted an 'open-source, web-based system administration tool' by exploiting a 'high-level semantic logic flaw.'

Google's research team found clues in the Python script used for the exploit suggesting AI involvement – like a ‘hallucinated’ CVSS score and textbook-like formatting consistent with large language model (LLM) training data. This comes after weeks of discussions about cybersecurity-focused AI models, especially concerning potential vulnerabilities.

The report highlights hackers' increasing use of AI to discover and exploit security flaws. Additionally, the GTIG has noted that adversaries are targeting integrated components essential for AI utility, such as autonomous skills and third-party data connectors. Hackers are employing 'persona-driven jailbreaking,' instructing AI to act as security experts to uncover vulnerabilities.

While this is a significant win for cybersecurity, it underscores the evolving threat landscape where AI is being used both defensively and offensively. Google’s researchers warn that hackers could refine their attacks within controlled settings before deployment, making them more reliable and harder to detect.