Delivering Malware Through Abandoned Amazon S3 Buckets

Here’s a supply-chain attack just waiting to happen. A group of researchers searched for, and then registered, abandoned Amazon S3 buckets for about $400. These buckets contained software libraries that are still used. Presumably the projects don’t realize that they have been abandoned, and still ping them for patches, updates, and etc.

The TL;DR is that this time, we ended up discovering ~150 Amazon S3 buckets that had previously been used across commercial and open source software products, governments, and infrastructure deployment/update pipelines—and then abandoned.

Naturally, we registered them, just to see what would happen—”how many people are really trying to request software updates from S3 buckets that appear to have been abandoned months or even years ago?”, we naively thought to ourselves.

Turns out they got eight million requests over two months.

Had this been an actual attack, they would have modified the code in those buckets to contain malware and watch as it was incorporated in different software builds around the internet. This is basically the SolarWinds attack, but much more extensive.

But there’s a second dimension to this attack. Because these update buckets are abandoned, the developers who are using them also no longer have the power to patch them automatically to protect them. The mechanism they would use to do so is now in the hands of adversaries. Moreover, often—but not always—losing the bucket that they’d use for it also removes the original vendor’s ability to identify the vulnerable software in the first place. That hampers their ability to communicate with vulnerable installations.

Software supply-chain security is an absolute mess. And it’s not going to be easy, or cheap, to fix. Which means that it won’t be. Which is an even worse mess.

Posted on February 12, 2025 at 7:09 AM13 Comments

Comments

Steve February 12, 2025 7:42 AM

It seems that Amazon should also be able to identify the same abandoned buckets and place an automatic lock on any abandoned bucket. The next one who wish to resume the project would have to identify themselves, and when that occurs an automatic notice goes out to all who subscribe with an appropriate warning.

Clive Robinson February 12, 2025 10:02 AM

@ Bruce,

With regards,

“Presumably the projects don’t realize that they have been abandoned, and still ping them for patches, updates, and etc.”

You might want to reword that as it’s ambiguous to read.

However what you are talking about is a rather more general issue, and one that hits encryption via the likes of “Public Key Certificates” etc, and lots of hardware and similar devices the worst of which appear to end up in consumer and SoHo type commercial products via “Internet of things”(IoT) “network Edge”, and many others.

In fact Amazon are conspicuously guilty of “enshitification” that includes designing and placing on the market “competing products”. With the sole intent to kill a business product for Amazon’s profit reasons. Then just dumping the “competing product” without any support. Call it “abandonware” or “product orphaning” the result is the same.

The simple fact is there is no way to guarantee that something that is revoked, abandoned, orphand or in other ways “left entirely vulnerable” can be recognised as such by the product user.

In fact any process to do so so far thought up can or does cause more harm than good.

The PubKey “Certificate Authorities” have conspicuously failed in their attempts to resolve this issue. So far the best they have come up with is “shortening the life” of the certificate to in effect a year or lot less.

Which might be good for their ignorant investor/shareholder business model, but it’s actually harmful to just about everyone.

Especially as it will encourage “further ignorance” by “automation”.

Put simply the job of updating a certificate will become automated, and it will work in the background forca length of time that it in effect becomes not just “invisable to use” but “forgoton to use”. Then one day way way upstream sonebody makes a change… And at best it cascades out as a fault that brings sufficient of the Internet down that people have to take action.

At worst it opens up massive security or worse vulnerabilities that “nobody at the sharp end” understands and don’t see anything catastrophic.

An example of which was a crypto system that when the KeyMat expired, rather than “block the comms” just changed the comms protocol down to the only one that still worked of “plaintext” no encryption. Because originally the design called for “high fault tolerance” not “assured security”, thus a very large petro-chem site remote management system became “open house” (luckily it was sufficiently off-shore that nobody got close enough to make it a “plaything that went pop”)…

This sort of crap happens all the time and it just does not get noticed, or even if noticed nobody understands it’s a serious issue so untill unavoidable pain happens next to nothing or nothing happens.

I can see this “nonsense” being “setup to happen” with “Smart Grids” and hopefully I won’t be around when it goes pop or people get killed on mass.

Sok Puppette February 12, 2025 10:14 AM

They. Let. Random. People. Register. Old. Expired. Names.

That’s idiotic. The risks are obvious to anybody smarter than your average bowl of pudding.

Which, of course, doesn’t include the esteemed stewards of the DNS. But still.

TimH February 12, 2025 10:35 AM

This just shows that the ability for a user to explicitly and unilaterally prevent unrequested software updates is just as important as security fixes being available for badly written and tested code.

Carl Breen February 12, 2025 11:31 AM

My 5 cent fix incoming:

The code contains the expiry date + a gracious offset (we all might be on holidays).
Beyond that date and grace period you must pass a CLI switch

–do-unsafe-update

And thus the web was saved.

Tom Kenney February 12, 2025 12:55 PM

Reminds me of late Vernor Vinge’s Fire Upon the Deep, locating lost ‘archives’ and discovering some nasty stuff in the box.

lurker February 12, 2025 4:09 PM

There’s no commercial incentive for Amazon to do anything about “abandoned” buckets. They have reportedly “sinkholed” (whatever that means) the ~150 found in this exercise, but there is still no imperative for them to lock, sanitize, or demolish any future examples. How do they know a bucket is “abandoned”? If the customer is a big.nameless.corp there’ll be an automatic payment system, so flat monthly, or traffic based, as far aa Amz is concerned, it’s still alive, even though the 404 msg says
“The specified bucket does not exist”.

Maybe the Cloud really has broken the internet …

Clive Robinson February 12, 2025 7:52 PM

@ Bruce,

Speaking of “leaking through abandoned” stuff… Just released today.

It appears Google was guilty of leaking peoples private information due to poor use of “Security by Obscurity”.

https://brutecat.com/articles/leaking-youtube-emails

Basically Google gives people a “Google Account ID number”(gaia ID) which others can in part find when blocking them on YouTube.

Whilst the person blocking gets the gaia ID in an obsfucated form, with a little playing around it can be got to the point where used through an old Google service to get the persons registering email address…

From the article,

“I told my friend nathan about the YouTube Gaia ID leak and we started looking into old forgotten Google products since they probably contained some bug or logic flaw to resolve a Gaia ID to an email. Pixel Recorder was one of them.”

Hmm.

The thing is there is an old saying that applies to Google and abandoned projects,

“There are plenty of more fish in the sea!”

Que “Moby” jokes 😉

Skiddie February 13, 2025 11:47 AM

GCP project names and IDs are unique and not reusable – even by the account that created/deleted them. This i found out quite by accident when trying to avoid updating configuration files by creating a new project with an old name.

The article says the AWS issue affects all cloud providers so this may need either clarification or correction.

Clive Robinson February 13, 2025 8:46 PM

@ Bruce, ALL,

Supply chains two ends to attack.

Most people think of a supply chain as being “one way” thus having,

“Risk from the source down.”

But often neglect to think about,

“Risk from the destination up.”

And for most they can not instantly see how that can happen.

To see how, we need to remember that the supply chain is just a form of technology, and like all technology it is agnostic to “Use”.

The “Use” however is ultimately decided by an “Agent” acting as a “Directing Mind”, where the “Use” is to the agents “benefit” not “Others” benefit. Thus can be seen as a “Profit” or “Gain” or similar in your analysis model uses.

Further the “Use” as “good or bad” is decided by usually uninvolved “Others” at some future point which brings in the “Observer Problem”, where they “see and judge” and “act” in some way.

So if the “Destination” is seen as “Bad” by an “Observing” other it reflects on the “Sources” “Reputation”, which in turn effects how “others” decide or have decided for them how they interact with the “Source”.

There are people who consider such risks for a source as part of “Reputation Management”.

Of late Politicians have been jumping up and down about the supply chain risks, and recently especially the reverse risk of “Reputation”. Unfortunately as with most politicians they mistakenly think,

“Technology can solve everything ‘automagically'”

Thus it should,

“Solve all the worlds problems and make everything good”…

Which any sane person who can actually think after a little reflection will realise is an “impossible ask” (those who doubt this should go study the work back in the 1930’s by Kurt Godël on mathematics and undecidability, and the most well known proof of the “Halting problem” that gave rise to modern “Turing Machine” computers[2]).

But worse Politicians think that “failure” to fulfill their fantasies should be “punished” and punished harshly (See UK RIPA etc and the issues around E2EE and what has happened with Apple just recently[1]).

Well as we all should know Politicians love “dog whistle” sound bytes that induce “knee jerk reactions” in others that are not thinking[3]. One such is all the “Think of the children” nonsense they use to “deflect” and “blame others” for their failings.

Well the latest on this is seen in,

https://www.bbc.co.uk/future/article/20250207-how-google-amazon-and-microsoft-funnelled-ad-money-to-a-site-hosting-child-abuse-images

[1] Covered on this blog just a few days ago,

https://www.schneier.com/blog/archives/2025/02/uk-is-ordering-apple-to-break-its-own-encryption.html

[2] The ever popular Veritasium has a reasonable introduction on undecidability,

https://m.youtube.com/watch?v=HeQX2HjkcNo

Though he inadvertantly gives the impression it’s “a maths thing” when in fact it goes much further down to both logic and set theory which both under pin “reasoning” thus just about everything most humans do in science, engineering, architecture and what makes our modern physical world what it is.

[3] The term “dog whistle” used in politics and sound bytes has an interesting history,

https://en.m.wikipedia.org/wiki/Dog_whistle_(politics)

lurker February 14, 2025 3:51 PM

@Clive Robinson
re imgBB

‘We have no idea where our ads are going’

ties in with Amazon’s ‘We have no idea when our buckets are abandoned’

iow “The computer dunnit, honest guv!”

which sounds like humans abandoning responsibility for their actions.
We live in interesting times …

Clive Robinson February 14, 2025 10:23 PM

@ Lurker,

With regards,

“We live in interesting times…”

Don’t we just…

But,

“The computer dunnit, honest guv!”

Is I suspect going to replace,

“The computer says No!”

As current AI LLM and ML systems get used as “arms length” agents or “cut outs” for those with nasty agendas they want to push but simultaneously pretend that,

“Their hands are clean”

and not in reality dripping in gore way up and beyond their elbows.

Leave a comment

Blog moderation policy

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.