Google Is Suing Chinese Scammers Who Are Using Gemini

Not sure this will have any effect, but I support the effort:

According to Google’s legal filing, Outsider Enterprise operates through Telegram. The group offers phishing-as-a-service to individuals who may not be technically savvy enough to set up fraudulent websites and text campaigns on their own. In its Telegram channels, Outsider Enterprise reportedly provided instructions on how to use Google’s Gemini AI to create websites that imitate those of Google, YouTube, and government agencies such as New York’s E-ZPass. The group offered nearly 300 scam templates.

[…]

Google worked with AT&T, Verizon, and T-Mobile to block many of these malicious text messages, and Google notes that its on-device scam detection in Google Messages probably helped reduce the number of successful phishing attempts, too. This AI-powered feature apparently stops 10 billion scam texts every month, so it’s fair to expect it caught at least some Outsider Enterprise activity.

Another article.

Tags: AI, China, courts, cybercrime, phishing, scams

Posted on July 7, 2026 at 6:43 AM • 7 Comments

Comments

Clive Robinson • July 7, 2026 8:30 AM

@ Bruce,

With regards,

“Not sure this will have any effect”

Basic science says all effects have causes.

Likewise to build something solid and lasting it requires considered construction based on solid foundations.

So whilst the effect may not be immediate or obvious initially, if it forms a solid foundation then more can be built upon it.

KC • July 7, 2026 11:27 AM

The company has called out seven different potential federal laws …

Love that you can set up alerts, for example on Congress.gov, to keep tabs on these.

Big working group list proposed under the ‘National Strategy for Combating Scams Act’ (H.R. 6425 / S. 3355). Reports required for ‘STOP Scams Against Seniors Act’ (H.R. 6426 / S. 4821) … I guess they’re all bipartisan.

Tris Simondsen • July 7, 2026 1:21 PM

The reason Outsider Enterprise’s phishing-as-a-service ring is so effective isn’t just a failure of Google’s API moderation, it is a structural exploitation of how generative tensor architectures fundamentally process ambiguity.

In a Zero-Trust architecture, we demand continuous validation of intent and authorization. But when a scammer prompts Gemini to generate an E-ZPass login page, the AI hits an unobservable information partition. It mathematically cannot observe the behavioral state of the user (e.g., a legitimate New York state developer vs. a malicious actor). The F-identifiable boundary is missing.

Because modern LLMs cannot maintain epistemic caution, they suffer what the Observational Sufficiency Principle (OSP) formalizes as an Equal-Weight Failure. At the tensor layer, the model takes distinct, non-verifiable latent paths and flattens them into identical downstream vectors (TM1 = TM2). Instead of halting at the boundary of what it can prove, the model executes “latent completion”, arbitrarily inventing a default state to bridge the gap, which effectively launders the scammer’s intent into functional HTML/CSS.

These scammers aren’t just bypassing content filters; they are weaponizing the AI’s architectural mandate to guess. This mathematically voids the Zero-Trust Evidence Contract, meaning we can never rely on the “execute” layer of a generative model to self-regulate security boundaries. It cannot recognize its own epistemic limits.

Please see below for the complete formal proof on why this tensor-level latent completion structurally breaks Zero-Trust:

https://trissimondsen.wordpress.com/2026/07/06/the-equal-weight-failure-how-ai-latent-completion-voids-the-zero-trust-evidence-contract/

Treating this as a moderation problem misses the physics of the issue. Generative AI architecture is currently incompatible with Zero-Trust.

Your thoughts?

Rontea • July 7, 2026 2:34 PM

Law enforcement coordination and proactive industry measures, like Google’s collaboration with carriers and on-device detection, are essential, but the scale of these operations shows that defensive efforts alone can’t fully close the gap.

We’re moving into an era where AI-driven attacks will adapt as quickly as we counter them. Civil actions and domain takedowns are important signals, but without parallel international legal frameworks and cooperation, the risk remains high. Public education—and the ability to validate digital trust at scale—will likely be as important as any technical safeguard to keeping users safe.

lurker • July 7, 2026 2:50 PM

Suing a handful of named Chinese in a New York court seems like a pretty futile way to end this madness.

Google makes guns. People use them to shoot other people. Nothing to see here. Move along please.

lurker • July 7, 2026 8:25 PM

@Tris Simondsen, ALL

Attributing to AI machines the ability to “invent” or “guess” continues the popular anthropomorphising, even to the extent of “It cannot recognize its own epistemic limits.”

Of course not. It has no organs (organic or electro-mechanical) of recognition. It can compare an item with another it has memory of, but when the difference between them falls below the machine’s discrimination limit it deems them equal. Humans do this too while learning, but humans learn from their mistakes, There is little evidence that the current crop of AIs learn from their mistakes. Indeed, by failing to “halt[…] at the boundary of what it can prove” the machine enters an invalid state.

Why are the purveyors of such systems allowed to sell a product unfit for purpose? Note that this is a social question which is unlikely to have a technical solution.

Clive Robinson • July 8, 2026 6:57 AM

Tris Simondsen, lurier, ALL,

With regards,

“In a Zero-Trust architecture, we demand continuous validation of intent and authorization. But when a scammer prompts Gemini to generate an E-ZPass login page, the AI hits an unobservable information partition. It mathematically cannot observe the behavioral state of the user (e.g., a legitimate New York state developer vs. a malicious actor).”

This is a restatement of the “observer problem”.

Back during or before WWII Claude Shannon came up with a formalised abstraction of “communications channels” and the properties of the information within them.

As part of this work Shannon showed that to be able to transmit information from a first party –sender– to a second party –receiver– in an information/Shannon Channel there must always be “redundancy” of some form.

In an extension to this about a third of a century later Gus Simmons proved the point that where there is redundancy you can have another Shannon Channel within it. A logical extension of this was Shannon Channels implicitly could be nested as far down as you wanted (the turtles all the way down, or lesser fleas ad infinitum issue).

Simmons also showed that these nested channels need not be overt, or covert, in fact they could be unintentional.

In fact physics confirms that matter/energy can have information impressed/modulated on them as a fundamental requirement that information can be Communicated or Stored.

So the notion of extending the Shannon Channel in a couple of ways arises. The first is the addition of one or more third parties that are at some point along a Shannon Channel. They can have one or both roles of being either another “receiver” as an “observer” or as another “sender” that can inject information into the channel between the first and second parties.

Importantly that all parties are in effect “transparent” to the additional Shannon Channels Simmons identified that are within the required redundancy as a necessary requirement for the transmission of information.

Further that these extra channels can be unseen by any of the primary parties that see the information within the primary Shannon Channel.

This is the basis for the “observer problem” they may see all the communication in the primary channel but the necessary redundance says they can not see any covert or unintentional nested Shannon Channel.

Thus you have all the proof required to show that Guard-Rails and even “pre/post information filtering” will not stop covert or unintentional information flow / leakage.

It’s why “Prompt Injection” can not be stopped, and importantly be used as either an “attack” or “defense”.

Using prompt injection as a defence against third and subsequent parties is not yet much discussed in the literature, but due to “Trust and Security” issues it will become more prominent with time.

However… Those thinking of using prompt injection and similar as a method of “trust” need to go back to the latter part of the 1990’s and take a very careful look at “Digital Rights Management(DRM) in “Off-Line” systems and why it was in effect always defeated.

Further those thinking of using it for security by “Surveillance” in particular should look into “Deniable Communications” that uses the redundancy that Shannon identified and Simmons extended to give “Perfect Secrecy” where any message is “equiprobable” in meaning.

Schneier on Security

Google Is Suing Chinese Scammers Who Are Using Gemini

Comments

Leave a comment Cancel reply