Published on
Artificial intelligence has made it easier to write emails, generate spreadsheets and plan holidays, as the widespread popularity of the various AI models can attest.
It has also, according to a recent Google report, made it considerably easier to figure out previously unmapped or impossible-to-predict gaps in the software of our systems.
Google’s Threat Intelligence Group said it had for the first time caught hackers using AI to discover and exploit a so-called zero-day vulnerability, or a security flaw the software’s developer does not yet know exists and for which no fix is available.
The target was a popular web-based system administration tool and the flaw allowed attackers to bypass two-factor authentication, that second layer of security most people believe keeps their accounts safe.
Google said it spotted the attack before it could be deployed at scale and quietly alerted the software vendor.
“The criminal threat actor planned to use it in a mass exploitation event but our proactive counter discovery may have prevented its use,” the report noted.
“Threat actors associated with the People’s Republic of China (PRC) and the Democratic People’s Republic of Korea (DPRK) have also demonstrated significant interest in capitalizing on AI for vulnerability discovery.”
A crack developers failed to see
The zero-day vulnerability is not a conventional flaw. Traditional security scanners look for crashes and memory errors, the software equivalent of a spellchecker looking for the digital equivalent of a typo – but this vulnerability was buried in the logic of the code, a subtle hardcoded assumption by the developer that no automated scanner would have caught.
It is the kind of mistake where everything looks correct on the surface but the underlying reasoning is broken. Think of a bank vault with a working lock that nonetheless opens for someone who knows the exception exists because the designer, without realising it, built one in.
That is exactly the kind of contradiction AI is good at finding. “Frontier LLMs excel at identifying these types of high-level flaws and hardcoded static anomalies,” the report continued.
Though frontier LLMs struggle to navigate complex enterprise authorisation logic, “they have an increasing ability to perform contextual reasoning… and [catch] the contradictions of its hardcoded exceptions”, it concluded.
This capability can allow models to surface dormant logic errors that appear functionally correct to traditional scanners but are broken from a security perspective.
Not just one trick
While the zero-day vulnerability was the main finding, the full report makes for uncomfortable reading.
Chinese and North Korean state-sponsored hackers are using AI to hunt for vulnerabilities at an industrial scale, sending automated prompts to probe for weaknesses in everything from home routers to corporate networks.
Google observed one North Korean group “sending thousands of repetitive prompts that recursively analyze different CVEs and validate PoC exploits”, building what the report calls “a more robust arsenal of exploit capabilities that would be impractical to manage without AI assistance”.
Russian-linked groups, meanwhile, are using AI to develop malware that rewrites itself on the fly to evade detection, a capability that previously required significant human expertise.
AI is also transforming phishing. Rather than mass-blasting generic emails, attackers are now using AI to map corporate hierarchies and identify specific targets with access to sensitive data and generate “higher-fidelity phishing lures tailored to individuals with administrative privileges”, in the report’s words, that go well beyond “the commodity tactics of traditional bulk phishing”.
The broader shift, Google warns, is from AI as a research tool to AI as sort of active combatant in the security sphere.
“The LLM is no longer merely a passive advisor but an active participant in the offensive chain, capable of orchestrating complex toolsets and making tactical decisions at machine speed.”
Google’s own AI tools flagged the zero-day before it could cause damage, which is the silver lining here. The company is deploying AI agents itself to find and patch vulnerabilities faster than human teams could manage.