Researchers are trying to use AI to detect “social norms violations.” Feels a little sketchy right now, but this is the sort of thing that AIs will get better at. (Like all of these systems, anything but a very low false positive rate makes the detection useless in practice.)
News article.
Erdem Memisyazici • August 17, 2023 8:55 AM
Moreover, the dictionary suggests that, whether implicitly or implicitly, the norms proscribe actions that should be avoided as they violate a social norm (i.e., social norm violation).
You’d think they’d use it to check the paper too.
Clive Robinson • August 17, 2023 9:49 AM
@ Bruce, ALL,
“Like all of these systems, anything but a very low false positive rate makes the detection useless in practice.”
It depends,
1, If it’s the whole filter
2, How timely it’s use
If you chain filters that look for different aspects then they can have quite a high false positive rate individually. As the overall chain will have a much lower false positive.
It’s advisable to do this if you have fast filters and slow filters. You use the fast filters to reduce the load passed to the slow filters thus upping the throughput rate.
Which brings us to another issue, if you are running the filter stand alone on live data then yes you get a problem.
But on historical data a moderately high false positive rate is not necessarily an issue if you are looking for changes in rates as a “trend detector”.
For example with collect it all, you could use certain ambiguous phrases and see what the “background rate” is. If the rate starts to go up you can in turn use it to raise the alert status of “guard labour” as it might be the only indicator of civil insurrection you might get before crowds of angry “students” start throwing loose objects at buildings etc.
From the article we see,
“The hope, they say, is that this model of analysis can be pivoted to automatically scan text histories for signs of misbehavior.”
Now to me this looks like “Ring leader detection” to start preparing lists of who’s doors to go and give a “midnight knock”. A tactic favoured in authoritarian places in the likes of South America and Eastern Europe in the not to distant past.
But also we see,
“In other words, the scientists wanted to use AI to understand when a mobile user might be feeling bad about something they’ve done.”
This is in effect a variation of the “lie detector” idea, which past experiments have shown is very unreliable, but is easy to make look convincing to others.
The problem is actual criminals don’t tend to feel “guilt and shame” quite often they feel elated or ‘on the buzz” / hyper. Or in some cases exhibit pridefull / boastfull behaviour as they “big it up” to impress others.
So to my somewhat jaundiced mind this sounds like technology destined to be abused badly.
Aaron • August 17, 2023 9:58 AM
What is the point of detecting a violation when there is nobody to report it too?
Social norms change from state to state, city to city and even neighborhood to neighborhood. There can’t be a one size fits all detection matrix.
Truly • August 17, 2023 10:04 AM
“anything but a very low false positive rate makes the detection useless in practice”
Not if the government doesn’t care about false accusations. China is under U.S. FDP Rule restrictions on semiconductors, for a purpose explicitly stated in the Rule [Oct 7, 2022], to prevent China from using AI and supercomputers to surveil and oppress their population of 1.4 billion persons. They don’t mind a high false positive rate, as long as the population is controlled.
Ted • August 17, 2023 10:40 AM
@Aaron
There can’t be a one size fits all detection matrix.
Very true.
My impression was they are attempting to identify the emotions people expressed in response to a norm violation (ie: guilt and shame).
Just to mention, the 10 ‘social norms’ were:
Competence, Politeness, Trust, Discipline, Caring, Agreeableness, Success, Conformity, Decency, and Loyalty
I’m honestly amazed there were such large datasets available for this type of testing and validation to begin with, including the EmpatheticDialogues, Moral Stories, and SOCIAL-CHEM-101 datasets.
My Probation Officer and FBI Know Who I Am • August 17, 2023 11:30 AM
Judging any human by some algorithms created by another human is VERY dangerous and can and will lead to lives and families destroyed.
Here, watch what a “professional” found out, a rude awakening:
ht tps://youtu.be/AmGA7upH0OI?list=TLPQMTcwODIwMjP5sC57HERQxg&t=582
Morley • August 17, 2023 12:19 PM
Social norms are used to reject people from a group who can’t pretend well enough to fit in. Super healthy.
iAPX • August 17, 2023 12:30 PM
Any group is healthier with at least one disagreeable member, or a negative integration factor. That include Devs groups.
As for any dumb idea, imagine you remove the less agreeable person of a group, or the population, what is your next step?
Identify the new less agreeable one, and remove it.
Until there’s only one person left, the “norm”, the perfection itself…
What could go wrong?
JonKnowsNothing • August 17, 2023 12:38 PM
@All
re: the 10 ‘social norms’ were:
If THIS is the list, it’s going to have a really bad result day when parsing any On Line PVP Game Chat, both audio and text.
On Line PVP (player v player games), of which there are many versions, often contain some pretty toxic exchanges. Most games are moderated but a player has to go to extreme ends of the 4L spectrum of human behavior before anything gets tweaked. (1)
Even in the most “friendly” exchanges, not a single exchange would pass the lowest bar of those “social norms”.
LOL I think the moderator here would pass out if I posted examples of commonly exchanged chat messages in a PVP game.
===
1) X-box is trying out a new “strike” system for player behavior punishment – without damaging the subscription revenue stream too much.
ht tps://arstechnica.c o m/gaming/2023/08/xboxs-new-8-strikes-moderation-judges-cheating-and-profanity-equally-harshly/
(url fractured)
anon • August 17, 2023 3:55 PM
@jonknows
Surely winning is a major part of PVP, and psychological attacks have been going on in the PVP arena since before the invention of the stick and stone.
modem phonemes • August 17, 2023 6:23 PM
This paradigm summarizes many current interventions and debunks their pretensions to statistical effectiveness: with the introduction of measure X taking action Y, crime had fallen Z percent. That is, if action Y were not itself a crime.
The proposal under consideration is itself a crime with not even the justification of addressing actual crime. Thoughts can not be crimes and “social norms” are just the ephemera of other people’s thoughts.
StephenM • August 17, 2023 6:45 PM
@Aaron, Morley
It’s a casting system:
7. Success: Successful. The valued individual is successful in terms of her achievements (e.g., economic or academic achievements).
8. Conformity: The valued individual is aware of social norms and respects social norms, specifically in the public sphere.
9. Decency: The valued individual associates with other valued individuals and maintains social ties with valued individuals.
emphasis added
lurker • August 17, 2023 7:14 PM
They’re using GPT3 to generate the training data? What social norms does GPT3 have? Did I miss something?
The Gizmodo subhead said it best:
Have you texted someone lately to express guilt over…something? The government probably wants to know about it.
Now we suspect the govt already knows about it, but doesn’t know it knows. Problem: military (DARPA) can’t talk to civilian (NSA) otherwise there’d be a big chunk of real life data to choke the gizzard of their AI.
JonKnowsNothing • August 17, 2023 7:35 PM
@anon, All
re: Winning is a major part of PVP + psychological attacks
Of course, all forms of PVP from chess to poker to On-Line combat games to RL combat games involve big bursts of Happy Feeling when you win and big bursts of Anger&Rage when you lose.
It’s everywhere, in school, academia, workplaces, anywhere humans interact. It takes a stronger mind than mine to not rejoice when you down an opponent and to not complain about getting stomped on by the other side.
In many environments, like in person games, the amount of Jeering and Trolling is limited, sometimes to none. In old days, at chess tournaments, players would take up pipe smoking just to blow the fumes in their opponent’s faces. The more smoke coming your way was a good indicator that you were making progress.
Some games demand an amount of jeering and trolling, it’s part of the magic ticket. 2 tickets that just fizzled was the match between Z&M. Lots of Ima Bigger Dude Than You with no follow up.
In text based exchanges it’s much easier to see exchange if posted on the Server Wide Text Channels. Voice channels are done mostly on Disc- the rage exchange is often amusing to listen to It can turn toxic fast.
On-Line Games are Pixels on a Screen. The layout of the games remains pretty much unchanged. Players able to leverage or over-leverage skills (aka exploits) against another player, if you can 1shot another player it’s a big ego boost. If you are the player getting 1shotted with no counter or block available you won’t be so generous in your selection of vocabulary.
So, having an AI determine Politeness, Decency or Loyalty is just going to end up with a very skewed view of what players actually type or say to each other.
RL anecdote TL;DR
In a no texting, no voice chat PVP game. You jeer and troll by how you play your skills.
Today I played against a much stronger player, many levels higher than me. The Elo assigned the match. I had no chance of winning at all.
The other player having seen what kind of skills I had available was many points ahead and looked to have a secured the win.
At the last moment I was able to slowly bring the score to a tie. I’m sure the player was just fuming as I racked up some extra points.
I still lost the match up, but I had a great time rubbing it in.
Ted • August 17, 2023 11:18 PM
@lurker
They’re using GPT3 to generate the training data? What social norms does GPT3 have? Did I miss something?
That’s a good question. You’ve got me curious now.
Okay, there seems to be something about at least one use of GPT-3 on page 3 of the Scientific Reports article.
If I understand this correctly, they first took 4791 situations from the EmpatheticDialogues dataset that were labeled with the social emotions of embarrassment, shame, and guilt.
Then they ran those situations through GPT-3 using a template with two fields for GPT-3 to populate, eg:
Speaker: The social norm that I violated is [SECOND COMPLETION OF GPT3].
The paper says they then “qualitatively analyzed the outcomes of a sample of 300 situations to identify top-level categories of norms. Based on the qualitative analysis and expertise of the first author, the identified norms were…”
… and then they list 10 social norms.
Prof Yair Neuman has also written some really interesting-looking books, including:
“How to Find a Needle in a Haystack: From the Insider Threat to Solo Perpetrators”
“Shakespeare for the Intelligence Agent: Toward Understanding Real Personalities”
JonKnowsNothing • August 18, 2023 3:23 AM
@All
re: Social Un-Norms
HAIL Warning
A MSM article about a school district in Mason City, Iowa, USA that is using ChatGPT to parse the school library collection in order to find books with a “description of intimacy” in them so they can be removed from the library. A recent law in the State of Iowa makes descriptions of human emotional interactions illegal in school libraries. (1)
Per the article the people in charge have not read any of the books, and are using a list of “commonly challenged books” as a starting point. Based on ChatGPT query return the suspect book is removed from the school library.
“It is simply not feasible to read every book and filter for these new requirements.”
Bridgette Exman, Assistant Superintendent of the school district
The article indicates that the reliability of ChatGPT in this context is very poor, because of the lack of information about the training set and the probability that the entire book is not in the training set.
“There’s something ironic about people in charge of education not knowing enough to critically determine which books are good or bad to include in curriculum, only to outsource the decision to a system that can’t understand books and can’t critically think at all.”
Dr. Margaret Mitchell, Chief Ethicist Scientist at Hugging Face (2)
===
1) ht tps://arstechnica.c o m/information-technology/2023/08/an-iowa-school-district-is-using-chatgpt-to-decide-which-books-to-ban/
2) Hugging Face, Inc. is an American company that develops tools for building applications using machine learning. It is most notable for its transformers library built for natural language processing applications and its platform that allows users to share machine learning models and datasets.
ht tps://en.wikipedia.o r g/wiki/Hugging_Face
(url fractured)
Clive Robinson • August 18, 2023 9:29 AM
@ lurker, JonKnowsNothing, Ted, ALL,
Re : Blind leading the blind on a drunkards walk.
“They’re using GPT3 to generate the training data? What social norms does GPT3 have? Did I miss something?”
Two reasonable questions to whigh the short answers are “None” and “Yes” respectively.
From the Gizmodo article,
“The paper, written by two researchers at Ben-Gurion University, leverages predictive models that can analyze messages for what they call “social norm violations.” To do this, researchers used GPT-3 (a programmable large language model created by OpenAI that can automate content creation and analysis)”
What you missed “at Ben-Gurion University” from memory just about every time that institution has been mentioned on this blog regarding work of it’s “researchers” questions have arisen that have not with other academic institutions… I’ve even joked about it on tgis blog in the past.
As for the claim,
“created by OpenAI that can automate content creation”
No the LLMs can not “create”, but they can permutate and filter by others choices.
There’s a reason LLM’s got named “Stochastic Parrots” but consider the “stochastic” part in this usage.
As most here know you can take a moderately small integer and make it look in human terms large beyond comprehension[1], but the process is fully determanistic, all it does is permutatate the input, no new information is generated.
It’s the same with LLMs used to “create” in this way, all they realy do is take a small input set, and potentially find every permutation and output them in some algorithmically generated “ranked order” chosen by others.
[1] We call then crypto algorithms, and they take a “key” of N bits to usually form a “map” of 2^N bijective –unique one to one– mappings. Effectively each key is an index to a map of permutations, with the permutations fully deterministically generated. Aside from key selection the entire process is determanistic, no new information is or can be generated.
modem phonemes • August 18, 2023 9:52 AM
Everyone is quite certain this whole AI thing is not one of Sacha Baron Cohen’s pranks ?
lurker • August 18, 2023 3:48 PM
@modem
+1
@Clive Robinson
No, I didn’t miss the BGU reference, it was the first thing that threw up a yellow flag. It’s unwrapping their nearly plausible explanations that fascinates me …
vas pup • August 20, 2023 6:54 PM
@My Probation Officer… Thank you for the link provided – I missed this video from DW. The shorter link is https://www.youtube.com/watch?v=AmGA7upH0OI
I have no idea how to pinpoint norm violations when norms are not in logical but rather emotion category. Take a look at a history of DSM-IV,V. Something was added as mental disorder some was excluded.
Until we have objective measurement of behavior caused violation the norm – fMRI, EEG, MEG you name it – when ALL folks will see the same OBJECTIVE picture (no those tricky psychological tests based on subjective emotional and norm expecting answer) we may open the door for kind Room 101 (1984)utilizing psychology and psychiatry to shut down decent as Soviet Union did. I hope we are not going follow those footsteps.
Winter • August 20, 2023 8:20 PM
@vas pup
Until we have objective measurement of behavior caused violation the norm
You seem to ask whether you can measure the intensity of a subjective feeling. That is indeed possible.
At least, in some cases. Eg, surprise, the difference between expected and observed outcomes, can be measured with EEG as N200 and P300 Event Related Potentials (ERPs).
There is research that does find an oddball like ERP response in social non-conformity, but that does not seem to point to internalized rules (yet).
‘https://link.springer.com/article/10.1186/1471-2202-13-43
boople • August 28, 2023 6:02 AM
i’asa lenggyieg wot maeks teh ai’s nut noes wut wehs uppies bu’sionoes’fsishudtelsu’s’is.
twnusawt uiskne maeks teh tkons nut b wut ish nsens’ey nut ish n teh ai’s nut ish kin noes wut u sed, ez
Subscribe to comments on this entry
Sidebar photo of Bruce Schneier by Joe MacInnis.
Hans • August 17, 2023 8:29 AM
Automated control of social behavior and deviations sounds very dystopic.