AI-Generated Steganography
New research suggests that AIs can produce perfectly secure steganographic images:
Abstract: Steganography is the practice of encoding secret information into innocuous content in such a manner that an adversarial third party would not realize that there is hidden meaning. While this problem has classically been studied in security literature, recent advances in generative models have led to a shared interest among security and machine learning researchers in developing scalable steganography techniques. In this work, we show that a steganography procedure is perfectly secure under Cachin (1998)’s information theoretic-model of steganography if and only if it is induced by a coupling. Furthermore, we show that, among perfectly secure procedures, a procedure is maximally efficient if and only if it is induced by a minimum entropy coupling. These insights yield what are, to the best of our knowledge, the first steganography algorithms to achieve perfect security guarantees with non-trivial efficiency; additionally, these algorithms are highly scalable. To provide empirical validation, we compare a minimum entropy coupling-based approach to three modern baselines—arithmetic coding, Meteor, and adaptive dynamic grouping—using GPT-2, WaveRNN, and Image Transformer as communication channels. We find that the minimum entropy coupling-based approach achieves superior encoding efficiency, despite its stronger security constraints. In aggregate, these results suggest that it may be natural to view information-theoretic steganography through the lens of minimum entropy coupling.
News article.
EDITED TO ADD (6/13): Comments.
Clive Robinson • June 12, 2023 10:44 AM
@ ALL,
Re : The article does not explain…
If you read the article it’s clear the journalist does not actually understand what they are writting about, sufficiently to explain it clearly.
They repeatedly say “minimum entropy coupling” as though it’s an incantation that will magically convay meaning to a reader…
Also you will find to things to ponder,
1, In order to come up with a new message indistinguishable from the original, innocuous one, you have to create a perfect simulation of the cover text distribution… …For human-generated text, this is not feasible… For that reason, perfectly secure steganography has long seemed out of reach.
2, But machine-generated text, of course, is not created by humans. The recent rise of generative models that focus on language, or others that produce images or sounds, suggests that perfectly secure steganography might be possible in the real world.
Both are “general case statements” which are “mostly but not always true”…
There are ways human generated text can carry a steganographic channel without it being detected by statistics (I’ve shown this in the past on this blog a number of times when arguing about the impossability of governments stoping crypto but still alowing communications, so why “back doors” will not work).
Likewise there are ways LLM generated text will show by statistics that there is a non negligable probability it has a steganographic channel within it.
The trick behind this “minimum entropy coupling” is to get the “statistical curves” to be as indentical as possible, thus no test becomes posible…
In essence the way to do that simplistically is with a random source with flat distribution. Or more formally a fixed phrase with a stocastic element… You might call a “Stocastic Parrot” if your mind want’s to go that way (and some do with LLM’s)..
A very simplistic way to describe this so you can get an idea is to have a stock phrase such as,
“We should meetup for a XXX”
Where the XXX is a word randomly selected from a list of words that have equal probability. Such as,
{drink, beer, tea, coffee, sandwich, etc…}
As long as the selection is random no one phrase generated has any more or less meaning than any other, if any, thus “all are equiprobable” which is the basis of Shannon “Perfect secrecy”.
How to make the selection “random” or “stochastic” to an observer, but not an intended recipient is actually a bit harder (actually almost impossibly hard the more general the language used).
Which is why when I previously described it I fell back on an open unicity distance, of Shannon’s “Perfect Secrecy”.
Now some of you are probably scratching your heads about LLM’s and why they might be good for this.
Well as I’ve said before LLM’s are realy just “noise shaping filters” that have massively parallel filter paths any one of which might be stocasticaly selected according to a fixed probability. The statistics of the filter, provided the weights are not changed, will remain the same regardless of the noise put in. If the noise starts off with a flat distribution then the ouput from the LLM filter will always have the same statistical charecteristics, no matter how often you run it.
The use of “Shannon perfect secrecy” –AKA OTP– means the observer can not pick up any statistical inference…
So hopefully that fills in a few gaps the article author jumped over.
Oh “minimum entropy coupling” is quite a new spin on an old idea. Thus you might have trouble looking it up, and if you do find a paper, you might find it a little tough getting your head around the language…
But I can assure you the idea if written up sensibly for a beginer is fairly easy to understand (you just need sufficient column inches).