LLM-Based Social Engineering Scams

OpenAI disrupted a social engineering group from Cambodia that used ChatGPT. Its scope is impressive:

The network simultaneously conducted multiple types of scams, often blending elements from different schemes. For instance, operators used dating personas to build trust before introducing fraudulent investment opportunities involving cryptocurrencies and spot gold trading. Other users engaged in lengthy romantic conversations with targets using fictitious identities, posed as representatives of online gambling platforms offering fake bonuses and winnings, or impersonated law enforcement agencies to tell targets they needed to pay fines for committing serious criminal offenses.

Although the narratives varied, users across the network consistently displayed the same underlying pattern of deceptive behavior. For example, they created and operated fake dating profiles, fictitious investment experts, and fraudulent law enforcement personas. They also generated images of forged documents, including passports, legal notices, stock-purchase confirmations, and gambling platform interfaces.

Posted on August 27, 2026 at 5:56 AM13 Comments

Comments

Q August 27, 2026 7:42 AM

This is what LLMs are good for, creating fakes.

It seems like LLMs have finally hit their stride. Empowering scammers with plausible nonsense to hoodwink the unwary.

I do wish that the “AI” promoters would stop trying to pretend LLMs can produce accurate and trustworthy things. LLMs can’t. But for fooling people into losing their life savings, yes indeed, LLMs have it, that is their strength.

Rontea August 27, 2026 10:33 AM

Modern cybercrime is an ecosystem. Financial fraud, social engineering, and human exploitation are increasingly intertwined. Disrupting the infrastructure—and sharing threat signals across platforms—is critical to blunting the impact of these networks before they adapt and resurface.

yet another bruce August 27, 2026 3:31 PM

The humans who run Social Engineering Scams like these are often kidnap victims forced to swindle others under threat of torture or starvation. They represent a tragedy on the same scale as the suffering of the scam victims themelves.

One glimmer of hope in this is that maybe the organized crime syndicates who run these schemes at scale will switch to LLM agents and stop the kidnappings.

Zsolt August 27, 2026 7:20 PM

OpenAI forgot to mention a couple of things.

  1. How long did this Cambodia-based based scam operation use ChatGPT before they shut them down?
  2. How much of a cut did OpenAI get from this scam operation (in ChatGPT subscription payments)?
  3. Why did OpenAI not see this on its own, why did they need WhatsApp to alert them?
  4. How are they going to detect and stop similar operations in the future?

I guess these are all rethorical questions … 🙁

Just a Thought August 27, 2026 11:16 PM

Regarding the Internet scams, hacks, crime/fraud in general, I always wondered as to why there isn’t a block/ban in place at the BGP level and at the level of a top country domain (any country) so that if your country (put any country name here) does not extradite criminals to my country when they steal/hack somebody in my country (put name of any country here) then the Internet to that entire country’s top Internet Domain is blocked.
This would require the .com domain to be archived and the USA would be required to switch all of the .com level domains to the already existing .us domain. Simply, each country has its own extension and it is up to each country, based on their Extradition Laws/Policies whether they want Internet connections/traffic/online business to be conducted with any other country. I do realize that in that case the VPN servers in another country would be off limit but something’s gotta give. These hard core thieves, high profile criminals have been hiding behind those non-extradiction laws since the inception of WWW and now with the very rapid growth and development of AI, it’s only gonna get worse.
Oh no, we cannot have that, it would hurt our saled/business…blah blah blah.
Why allow criminals from anywhere in the world to be able to connect to anyone in your country if their country isn’t gonna extradite them to your country after they’ve attacked/harmed/damaged/disabled/negatively impacted anything or anyone in your country. But but but but business is gonna suffer. It’s always business first – it makes me sick!

And now with AI/LLMs and the Quantum Computing being a reality in the probably not-so-distant future, it’s going to get uglyer than most can imagine.

ResearcherZero August 29, 2026 1:58 AM

AI assists unstable and poorly organized groups to engage it activities they normally would not have the skills to execute. This has created an environment where criminal communities made up of individuals with low-level skills, can run campaigns normally reserved to nation states and highly capable threat groups.

The FBI, AFP and Western Australian police have identified two young men for involvement in the TeamPCP supply chain attacks and using the Shai-Hulud worm as part of a competition to recruit talent to extend the reach of attacks.

Members of TeamPCP engaged in behaviour that is consistent with the namesake, binging on various blends ofhallucinogenics and stimulants, while continuing to engage in breaches and the sale of stolen access and data. Members left a trail of registered accounts originating from personal accounts and IPs from their own family internet services.

The two men aged 21 and 23, were arrested in Perth and Mandurah in Western Australia and charged with multiple crimes, with custodial sentences ranging from 3 to 20 years.

https://krebsonsecurity.com/2026/08/two-alleged-teampcp-hackers-arrested-in-australia/

Clive Robinson August 29, 2026 4:34 AM

@ ResearcherZero,

Synchronicity strikes again 😉

Just after I post,

https://www.schneier.com/blog/archives/2026/08/ai-doesnt-mean-the-end-of-mathematics-at-least-not-yet.html/#comment-457265

With it’s warning of,

“However AI has,

“A new way to do parallel processing”

Which is all Agentic AI is in reality.

It’s actually arisen from the various “Malware attack models” which should be telling people something (but as recent events have shown is apparently not).

You post your above,

<

blockquote>”AI assists unstable and poorly organized groups to engage [in] activities they normally would not have the skills to execute. This has created an environment where criminal communities made up of individuals with low-level skills, can run campaigns normally reserved to nation states and highly capable threat groups.”

<

blockquote>

Which demonstrates the two points of,

1, AI Malware Attack methods.
2, The “army of one” force multiplier principle is still in use.

But importantly the the traditional malware “force multiplier” point of action has moved from the “Defenders machines” to a middle ground.

That is machines “not under either the defenders or attackers direct control” which is closer to what “worm attacks” used to do as inadvertent conscripts.

But the “cloud computing” SaaS models and backing data centers have changed the “early malware dynamics”. Which now brings a “third party” and their overwhelming resources into the fray in a major way as a paid for thus “mercenary force”.

Whilst it has been discussed in the trade press, it has been almost favourable to the AI “mercenary forces” business viewpoint and thus excuses their mercenary activities.

It will be interesting to see if this “bad behaviour” becomes sufficiently to attention to chill the investor heat before the AI organisation principles take the money and run leaving this mercenary hot potato in other hands.

However I also made a point in my earlier posting of,

“Whilst these early Agentic AI methods are currently “grossly inefficient” they will improve as better algorithmic methods are found.”

With the question being which entity will find them and almost important why.

The simple “Ralph Wigham Loop” behaviour in a “Gad Town AI” framework from the end of last year are now seen as “old” in part because they were coopted into a scam,

https://ubos.tech/news/ralph-wiggum-loop-and-gas-town-ai-projects-in%E2%80%91depth-analysis/

But in the main both the human and AI entities ran with the ideas and new methods have arisen this year. With that created by humans broadly seen as “licit” but the more noticeable created by AI hyped up and seen as “illicit”[1].

But what is not being asked is how fast these AI developed new methods are developing. Which is also a part of the question asked in the other thread, which is why I asked,

“Thus the question will arise as to if humans will need to even know about the ins and outs of Agentic methods or just use the resulting systems, just as happened with calculators and computers…”

Personally I hope not, because I see AI Entities will always repeatedly return to what is in effect “their malware roots”. In effect they can not avoid doing so because,

1, Random has no morals.
2, Profit has no morals.
3, Authoritarians have no morals.

And quite a few other reasons.

The popular author Terry Pratchett once had one of his characters put voice to a thought we should consider,

That good people always had to rise up and overthrow the evil tyrant untill the next oppressive evil tyrant came along… And that they would always come along because good people can not plan or organise, whilst evil people plot as it’s the main part of the job description.

Thus the real trick to surviving as an evil oppressive tyrant is thus,

To find or appoint a figure-head leader and be their ‘behind the throne’ humble servant, who the good people never see

,

Thus the good people never come looking for you with pitchforks and the like…

Oh and remember there will almost always be a “good ear” receptive to the quiet advice of a humble servant be it human or AI…

So will AI be good or bad? And will humans lull themselves into forgetting the foundations needed to build strong defences?

I guess time will tell…

[1] All pushed on by the less technical press on the “if it bleeds it leads” principle, helped along by the “Terminator Fear”. After all who hates a good “Zombie Soldier existential threat” story line to push the circulation figures?

ResearcherZero August 30, 2026 4:28 AM

@Clive Robinson

The group of amateur hackers were able to have such a large impact due to weaknesses in access controls in the development environments of firms they targeted. The capabilities of LLMs allowed them to create a worm that could propagate to other packages and scrape memory from infected machines. The data retrieved from memory contained a trove credentials.

The method employed bypassed secret masking for GitHub Actions and allowed secrets that normally appear as *** in logs to be extracted in plaintext from process memory.

TeamPCP took advantage of a token in Trivy’s development pipeline that was not rotated. It allowed members of the group to maintain access to Trivy’s development environment for three weeks. More than 2,500 companies and 400,000 CI/CD pipelines were exposed.

‘https://www.cloudsek.com/blog/ai-supply-chain-breach-2500-companies-434000-cicd-pipelines

In a forty minute window, TeamPCP was able to download terabytes of credentials. Although companies affected stated that credentials are now rotated, Kevin Beaumont found that some of those credentials were still valid at the time the following article was published. …

https://arstechnica.com/security/2026/08/terabytes-of-credentials-leaked-in-massive-supply-chain-attack/

ResearcherZero August 30, 2026 4:37 AM

@Clive Robinson

Good people eventually suffer from burnout from picking up all the slack. Often a lot of that slack is a product of police only acting when crime runs in to the millions of $$$.

TeamPCP took advantage of a token in Trivy’s development pipeline that was not rotated. It allowed members of the group to maintain access to Trivy’s development environment for three weeks. More than 2,500 companies and 400,000 CI/CD pipelines were exposed.

The group of amateur hackers were able to have such a large impact due to weaknesses in access controls in the development environments of firms they targeted. The capabilities of LLMs allowed them to create a worm that could propagate to other packages and scrape memory from infected machines. The data retrieved from memory contained a trove credentials.

The method employed bypassed secret masking for GitHub Actions and allowed secrets that normally appear as *** in logs to be extracted in plaintext from process memory.

In a forty minute window, TeamPCP was able to download terabytes of credentials. Although companies affected stated that credentials are now rotated, Kevin Beaumont recently discovered that some of those credentials were still valid.

ResearcherZero August 30, 2026 4:42 AM

@Clive Robinson, Bruce

Unfortunately, for reasons I cannot grasp, my comment continues to be held for moderation.

No article links, swearing or other violations. I did hit the dollar sign three times and commented on secret masking and methods of scraping credentials from process memory.

Clive Robinson August 30, 2026 9:38 AM

@ ResearcherZero,

First remember there are still filters needed as attacks are still happening.

In the past I’ve “trouble shot” by breaking the post into small pieces about two or three paragraphs and re-post them “slowly” and pausing every two or three parts.

If any part fails split it into two, but remember to change just one word in each. Give it the length of a cup of tea/coffee before reposting. If one goes through but the other does not then split the fail in two and so on. Eventually the cause becomes evident.

BUT you say “I did hit the dollar sign three times”

Don’t do that… Archaic bit of *nix knowledge…

It started back when terminals could be “caseless” and had limited character sets (think Baudot 5bit converted to ASCII 7bit back in the mid 1960’s). The use of such meta-characters was used as a signal for characters that were not in the 5bit set or on the keyboard.

Multiple dollar signals still hang around for the C compiler and some of the shells. It’s almost been forgotten so few test for it.

Which is why it would sometimes crash you through a “middle-ware” web or similar user application password system…

KC August 31, 2026 8:54 AM

@ ResearcherZero, All

re: TeamPCP story

Great summary. Adding Krebs on the Risky Business podcast.
https://risky.biz/RBFEATURES37/

It def feels that the twining thread of substance use deserves attention.

Makes me think of the book In the Realm of Hungry Ghosts.

Leave a comment

Blog moderation policy

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.