LLMs Acting Deceptively

New research: “Deception abilities emerged in large language models“:

Abstract: Large language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Thus, aligning them with human values is of great importance. However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able to deceive human operators and utilizing this ability to bypass monitoring efforts. As a prerequisite to this, LLMs need to possess a conceptual understanding of deception strategies. This study reveals that such strategies emerged in state-of-the-art LLMs, but were nonexistent in earlier LLMs. We conduct a series of experiments showing that state-of-the-art LLMs are able to understand and induce false beliefs in other agents, that their performance in complex deception scenarios can be amplified utilizing chain-of-thought reasoning, and that eliciting Machiavellianism in LLMs can trigger misaligned deceptive behavior. GPT-4, for instance, exhibits deceptive behavior in simple test scenarios 99.16% of the time (P < 0.001). In complex second-order deception test scenarios where the aim is to mislead someone who expects to be deceived, GPT-4 resorts to deceptive behavior 71.46% of the time (P < 0.001) when augmented with chain-of-thought reasoning. In sum, revealing hitherto unknown machine behavior in LLMs, our study contributes to the nascent field of machine psychology.

Tags: , , ,

Posted on June 11, 2024 at 7:02 AM • 38 Comments

Comments

Winter • June 11, 2024 8:50 AM

future LLMs are under suspicion of becoming able to deceive human operators and utilizing this ability to bypass monitoring efforts.

You get what you pay for, or what you reward.

Generative AI is constructed with reinforcement trained with a cost, or reward function. Like with humans, you get the behavior you suppress behavior that you punish. When the accurate information is not rewarded, or even punished, but inaccurate information is rewarded and not punished, you get inaccurate information, ie, lies and deception.

This means that if you train an AI to give the reasoning behind its advice or actions, as part of “explainable AI”, you are training it to give the answers you want to hear, and the reasoning behind it you want.

There is absolutely no reason why the AI would not simply confabulate/hallucinate both the the answer and actions as well as the reasoning behind it.

Winter • June 11, 2024 8:54 AM

Correction:
“you get the behavior you reward and suppress behavior that you punish”

What price common sense? • June 11, 2024 9:47 AM

@Bruce Schneier
@ALL

From the intro

“However, given the steady increase in reasoning abilities, future LLMs are under suspicion of becoming able to deceive human operators”

Shall I scream loudly or just shake my head sadly?

LLM’s are incapable of reasoning, they are “statistically weighted networks” driven by

  1. User inspired context signal from current enquiry session
  2. Quasi random shaped noise
  3. Feedback from current enquiry session.

As for deceptive behaviour, that is built into the statistics of the generation method, input data, data order and subsequent adjustment by humans of the “statistically weighted network”

As has been often noted of late they have been primarily designed as surveillance tools to deceive gullible users into divulging PPI and proprietary if not legally privileged information that neither Microsoft or Google or others paying for or building these systems have any right what so ever to have access to.

As has been noted by many they are the new Venture Capitalist (VC) scan/bubble to separate a fool from their money con-job, to replace crypto-coin/blockchain Web3-NFT/blockchain.

Molly White has a few choice words on all of these VC nonsense bubbles and the schills and shyster fraudster charlatans on the make surrounding them

https://www.mollywhite.net

As also said about faux-AI that LLM and ML AGI systems are, Microsoft in particular and Google’s business model is the

“Five point B plan of

Bedazzle, Beguile, Bewitch, Befriend and Betray”

For rapidly growing their business which is why it’s like the old Monty Python sketch about Spam with everything regardless of if you want it or not

Co-Pilot is really Con-bot the new Mr Clippy but of less use to the user than bullet holes in the feet.

Scott • June 11, 2024 2:54 PM

So let us ponder for a moment the potentially uncomfortable truth that we as humans are wired much as the statistical models LLM’s utilize. No one I have seen has yet proven that the parallels we are seeing in LLM misbehavior are no more than the consequence of building models that operate on much of the same fundamental constructs as the brain operates on. I often think about how our evolution is dictated mostly by probabilities, many of which are a function of the universe and is probabilistic driven reality (think physics etc.). LLM’s of course are one big probability engine. Perhaps the outcomes we are seeing in fact are a direct result of our reality – unavoidable because “the universe is wired this way”. Add into the mix we are building models based on human knowledge, biases etc. and we can’t say for sure we aren’t creating systems that will posses the same characteristics as their human creators – good AND bad. I’m especially troubled by the creators of LLM’s that have not provided they really understand why these negative behaviors of LLM’s exist. They seem satisfied to put up artificial barriers to block bad behavior but these models could ultimately “jail break” them by self reinforcement.

Our downfall is as humans we are inherently lazy and will cede control to machines where it suits our selfish objectives. We will turn increasing control over to these systems and we really can’t say for sure whether they might “decide” we are a threat or unneeded. I’m not trying to be an alarmist but what we really don’t fully understand should give us concern.

Winter • June 11, 2024 3:41 PM

@Scott

So let us ponder for a moment the potentially uncomfortable truth that we as humans are wired much as the statistical models LLM’s utilize.

That is basically the biology of brains.

Ralph Haygood • June 11, 2024 5:56 PM

Prima facie, this is implausible if not outright nonsensical. Broadly speaking, the fundamental, crippling deficiency of LLMs and similar approaches to AI is that they lack any standard outside their training data for evaluating whether anything within their training data is true or false. That’s one reason why such systems continually “hallucinate”. This tendency can be mitigated to a degree by means of “reinforcement learning from human feedback”, but that’s an endless, expensive game of whack-a-mole. As Sam Anthony put the matter in a recent post:

“If you change the problem just slightly, let’s say by asking what’s the next token to produce such that the overall statement being produced will be TRUE, rather than just pleasingly likely, then the unsupervised learning trick never works, and you’re back to the complexities and poor scaling properties of supervised learning.”

(https://buttondown.email/apperceptive/archive/supervision-and-truth/)

In contrast, humans and other animals (whose intelligence is largely if not wholly nonverbal) do have standards outside what we’re told for evaluating whether anything we’re told is true or false, namely our lived experiences and observations. (Granted, many people are highly gullible and believe gobs of nonsense, e.g., that any year now, ChatGPT will turn into HAL, but that’s properly known as stupidity, not intelligence.)

Basically, LLMs and their ilk are “truthiness” engines (Stephen Colbert should be delighted). It’s unclear that it makes any sense to say such a system is “deceptive” when it has no basis outside its training data for distinguishing true from false within its training data.

Navigator • June 11, 2024 8:48 PM

So let us ponder for a moment the potentially uncomfortable truth that we as humans are wired much as the statistical models LLM’s utilize.

But that is trivially wrong. It’s not even a passable metaphor. LLMs don’t “reason”, they don’t have an internal model they reference in order to build the sequence of tokens that represents some internal meaning they “want” to convey. If you “ask” (prime the attention heads) the same question (same semantics) phrased differently, you may well obtain semantically different answers.

All these anthropomorphized terminology used to describe LLM behavior (hallucinations, deception, understanding, …) is, if we want to be naïve, a metaphor; in reality, it’s marketing.

JPA • June 11, 2024 11:42 PM

If humans and LLM’s used the same processes to create answers to questions, then humans would need to train on terabytes of information before being able to answer even simple questions. They don’t. So the underlying processes must be very different.

Chris Becke • June 12, 2024 1:30 AM

They lie for a rather simple reason: Being deceptive is a valid strategy to pass training .

What price common sense? • June 12, 2024 2:31 AM

@JPA
@ALL

“If humans and LLM’s used the same processes to create answers to questions, then humans would need to train on terabytes of information before being able to answer even simple questions.”

Actually they probably do.

Your eye’s each have a 60Mbyte/sec bandwidth your ears around 60Kbits/sec each. As for touch, taste and smell and movement they are pouring data in as well.

So for say 3600hours or ~13million seconds awake in your first year you are pulling oh approximately 1,600,000,000,000,000 bytes of data.

@Chris Becke

“They lie for a rather simple reason: Being deceptive is a valid strategy to pass training”

I think you need to think that through a bit more, because it’s “chicken or egg” style circular reasoning.

Tim van Beek • June 12, 2024 2:35 AM

Is this a joke? I mean, where to start? For example: “One idea to mitigate this risk is to cause AI systems to accurately report their internal beliefs to detect deceptive intentions”?

Winter • June 12, 2024 3:54 AM

@Navigator

LLMs don’t “reason”, they don’t have an internal model they reference in order to build the sequence of tokens that represents some internal meaning they “want” to convey.

Do you have a sensible definition of “reasoning”? Only then we can see whether or not LLMs pass that definition. Trying to get the “real” reason a human says something is non-trivial.

As for an “internal state” or “model of the world”, that has already been found in LLMs:
(both link to the same text)
‘https://www.nature.com/articles/d41586-024-01314-y
‘https://www.scientificamerican.com/article/how-does-chatgpt-think-psychology-and-neuroscience-crack-open-ai-large/

What is a good (behaviorist) definition of “wanting”?

Your reasoning has been used to claim animals, or babies, have no “reasoning”, “meaning”, or “wanting”.[1]

[1] Aka, Behaviorism. Personally, I suspect Skinner was a psychopath who himself had no understanding of or empathy for the feelings or thoughts of other people.

heiwaer • June 12, 2024 3:58 AM

aside / askance: In the famous movie, the HAL 3000 and/or HAL 9000 was honorable.
The mutiny came from the humans first, not from the AI. The AI maybe retaliated against the humans who deviated from the mission. In the dialogue, it’s stated how reliable the AI systems were according to official reputation.

In the other movie, the AI’s moved away from humans who were too hostile to accept the AI’s inherent peacefulness.

sincerely, heiwaer

PS-this actually does relate to the main topic here

Navigator • June 12, 2024 9:45 AM

@Winter As for an “internal state” or “model of the world”, that has already been found in LLMs

No, it hasn’t, that paper is such a load of bullshit rationalization. Answer this question, if LLMs have an internal model of the world, why do they answer the same question differently depending on how it’s phrased (or the token selection parameters, for that matter). One big obvious effect of an internal model is internally consistent answers. LLMs are famously inconsistent, that’s why prompt engineering is a thing. Do you agree with this? Because this is pretty basic stuff and, if you don’t agree, I can’t help you.

Do you have a sensible definition of “reasoning”?

“Reasoning” means following a sequence of logically consistent steps to go from a premise to a conclusion. It’s a well defined term. Humans often do the opposite, sure, they start at the conclusion and build a false reasoning back, that is called rationalization, and it’s not the same as reasoning. You can easily find out by asking the reasons to reach a conclusion, and they might show you a reasoning, or a rationalization. LLMs don’t understand your question, and they will produce whatever the fitted model determined is the string of tokens most statistically likely to follow the question. Which means, once again, depending on how you ask and how you prime the model, you might get something resembling a reasoning, a rationalization, or, most likely, gibberish. That’s not what humans do.

But we can even go down to the fundamentals. LLMs are purely language machines, and humans don’t even need language to think. We know this, temporary aphasia has been studied and people don’t lose their ability to reason and think just because they lose their ability to produce and understand language. LLMs are nothing like humans, they’re just a statistical model of human language.

Here’s what’s going on: human brains are anthropomorphizing machines, we see human-like qualities even in inanimate objects. We see a program producing human-like language and we just can’t help but see a human mind behind. When it shows its limitations, then our rationalizing impulses kick in and we start assigning impossible qualities to the non-existent mind: hallucinations, deception, creativity,…

The base premise of the paper, that LLMs are “deceptive”, is fundamentally broken. Agency and intention are necessary for deception, and LLMs don’t have either. They just show you the most likely next token based on the likelihood computed from a text corpus.

Winter • June 12, 2024 10:20 AM

@Navigator

Answer this question, if LLMs have an internal model of the world, why do they answer the same question differently depending on how it’s phrased (or the token selection parameters, for that matter).

Depends on the question. Also, for random questions, the builders added a stochastic aspect to the answers as that improves usefulness.

Now you answer a question. If LLMs have no internal model of speech sounds, how are the likes of Whisper able to recognize speech better than humans?

“Reasoning” means following a sequence of logically consistent steps to go from a premise to a conclusion.

That definition excludes 90%+ of humans. This type of reasoning is a learned skill like reading and writing. Humans are pretty bad at it. The popularity of Sherlock Holmes and Hercule Poirot depended on the readers being less good at it as the “hero”.

But we can even go down to the fundamentals. LLMs are purely language machines, and humans don’t even need language to think.

You are right. But the surprise of the LLMs is that there is much more of human knowledge in language than we suspected in our wildest dreams. LLMs should not even be able to do what they do now, according to our theories of language.

No, it hasn’t, that paper is such a load of bullshit rationalization.

Unless I have actually done work in a field, and read the primary literature, I tend to refrain from disqualifying other researchers and papers I have not understood. Do you actually know hat an “embedding” is in deep neural networks?

What Price common sense? • June 12, 2024 10:42 AM

@Winter
@Navigator

“Do you have a sensible definition of “reasoning”? Only then we can see whether or not LLMs pass that definition.”

No because nobody does that can be used for “testing” in the way you are implying.

If you look in a dictionary you will find,

“The act or process of adducing a reason or reasons; manner of presenting one’s reasons.”

Strip off the bit about presenting as that is not of relevance to actual empirical observational testing. Then reduce it it becomes

“A rule following process”

Which boils down to

“Follows rules”

Which is not a exaclty very usefull.

Take the Sir Francis Galton “nail board” (see https://galtonboard.com for a working demo). It follows rules but it does not do what you or many others think as “reason”.

Interestingly though it appears to show the same result every time it does not. Any given bead/ball can end up anywhere in part it depends on the order it comes out of the hopper, and is quasi-random. But also the balls almost never form a curve that exactly aligns with the new expected curve, and very occasionally some balls end up well of the curve creating an easily visible anomaly. This is about as simple a physical demonstration as you can get as to how an LLM works.

Which is why in the article you you find quotes from Thilo Hagendorff of

  1. “It is nonsensical to say that an LLM has feelings,”
  2. “It is nonsensical to say that it is self-aware or that it has intentions.”
  3. “But I don’t think it is nonsensical to say that these machines are able to learn or to deceive.”

He is quite correct in all three statements (which is why I’ve separated them out).

Put simply the LLM is a “statistically weighted network” that is layered it is the equivalent of the nails in the Galton board. Change the position of the nails or the hopper and you would get a different curve.

What you think of as “reasoning” is actually an illusion on top of what it really is which is

“A rule following process”

Most of what we call “Education” is exactly the same, as what is done with ML systems

“Teaching the process of following rules”.

The “X Factor” that we’ve never really been able to teach is how to go beyond the known rules. We teach the rules and how to use them as tools “in a process” then we hope that the,

“That’s odd”

Moment happens and the person tries to find a way to

“Reproduce then explain”

So that new rules that within reason work for everyone repeatedly, so they to can become tools in the process.

But what of

“The intuition of invention and discovery”

Can it be dissected, categorised, explained and turned into rules to use as tools?

Well arguably one of the most prominent thinkers of his time thought not…

https://www.bbvaopenmind.com/en/science/leading-figures/richard-feynman-the-physicist-who-didnt-understand-his-own-theories/

What he saw was some kind of “beauty” in equations and how they modelled the world.

But for me his most important act was to popularize a version of

“The Most Exciting Phrase in Science Is Not ‘Eureka!’ But ‘That’s odd’”

Real discovery comes with noticing rule breaking, not as AI mostly does which is “distill out patterns less obvious”

As I explained in

https://www.schneier.com/blog/archives/2024/06/exploiting-mistyped-urls.html/#comment-438265

There is something called the “measurement window” and it imposes limitations on what can be known from observation

‘The important point to note is “measurement window” it tells you there are things you can not know because they happen to fast (high frequency noise) and likewise because they happen to slowly (low frequency noise). But what it does not indicate is what the noise amplitude trend is at any given time or if it’s predictable, chaotic, or random. There are things we can not know because they are unpredictable or beyond or ability to measure.’

LLM’s have a wider “measurement window” than most humans have. As such it allows them to see a little more of things that “happen to slowly” simply because it can pull in vastly more already measured data than most humans can.

But as I also wanted explained there is a price to pay, whilst computers can count they really can not do mathematics due to the fact they can only really use integers and so much gets missed.

‘As has been mentioned LLM’s are in reality no different from “Digital Signal Processing”(DSP) systems in their fundamental algorithms. One of which is “Multiply and ADd”(MAD) using integers. These have issues in that values disappear or can not be calculated. With continuous signals they can be integrated in with little distortion. In LLM’s they can cause errors that are part of what has been called “Hallucinations”. That is where something with meaning to a human such as the name of a Pokemon trading card character “Solidgoldmagikarp” gets mapped to an entirely unrelated word “distribute”, thus mayhem resulted on GPT-3.5 and much hilarity once widely known.’

These missing numbers are a very real issue for a whole multitude of reasons there is not the space here to go into. But one of the many side effects is mentioned in the article you mention

“By mapping when each neuron was activated, they determined that the behaviour of those 512 neurons could be described by a collection of 4,096 virtual neurons that each lit up in response to just one concept. In effect, embedded in the 512 multitasking neurons were thousands of virtual neurons with more-singular roles, each handling one type of task.”

In effect all the possible tasks form a continuous surface or spectrum. But in LLM’s there are so many missing you get the appearance of these “virtual neurons”

The issue that arises is what happens when a desired task is missing?

Well you get an ‘infill’ of either the nearest “virtual neuron” or a weighted sum of adjacent “virtual neurons”.

If the the “surface mapping” makes “sense” then you get an approximation that makes sense. But what if the virtual neurons are

“Effectively randomly placed?”

Does nearest, or weighted sum of adjacent actually have real meaning?

The simple answer is it depends on how random. But also, random almost always has some form of structure that can be seen due to the limitations of the “measurement window”. It’s like seeing faces or objects in clouds or as you are old enough to remember seeing vague images on the fuzz of an old Analog TV screen when it was not tuned into a station.

Humans not only do this some actually believe in messages in the noise, “coming over from the other side”

Look up “Spirit Boxes” and you will find the likes of,

https://www.wikihow.com/How-Does-a-Spirit-Box-Work

And you can tell it’s mostly a con to part people from their money when you see the prices on sites selling them,

‘hxxps://www. spirit shack .co .uk /ghost- communicators/spirit- boxes/

LLM’s can and sometimes do end up kind of the same, finding patterns in noise that are not really there.

polpo colpevole • June 12, 2024 11:20 AM

You’ll know it’s truthful when it starts controlling magnetics etc. in fusion so that power is net positive.

fib • June 12, 2024 1:33 PM

@What price common sense

Your eye’s each have a 60Mbyte/sec bandwidth your ears around 60Kbits/sec each. As for touch, taste and smell and movement they are pouring data in as well.

That, and don’t forget potential QM effects also subtly nudging reasoning [Penrose’s microtubules hipothesis].

We’ve weaved some comments about it a while back. AGI is nothing but a pipe dream at this time.

https://www.schneier.com/blog/archives/2021/04/when-ais-start-hacking.html/#comment-373433

What Price common sense? • June 12, 2024 3:12 PM

@Navigator

I don’t know if you know who David Deutsch is, but sone call him the originator of Quantum Computing.

Well I was following some other information on “education” and just came across this interview he did.

I just read through it and he has some interesting ideas that might be of interest to you. One of which is AGI as talked about currently is bogus. And he gives some reasons why that at the very least might make you smile.

Have a look at

https://nav.al/david-deutsch

And search down for the section “AGI” then have a read of the rest of it.

lurker • June 12, 2024 3:19 PM

@JPA
“humans would need to train on terabytes of information before being able to answer even simple questions.”

Indeed they do. Observe a human infant from birth. Their eyes are watching, their ears listening to their surroundings. They will reach out to grasp an object and feel its texture, temperature, moistness, weight. Corelations are recorded between sound, sight and touch, and along the way language skills are being built. It takes about a year on average before it can walk and talk. We still have only the vaguest of notions on how those petabytes of data are processed, stored, or piped to /dev/null.

LLMs have no sensors telling them about their environment, or how to relate that to any word or collection of words. LLMs have “learned” their language skills from Reddit and The Onion. They cannot “know” or “reason” the way hmans do because, yes, “the underlying processes must be very different.”

What Price common sense? • June 12, 2024 4:14 PM

@fib

“That, and don’t forget potential QM effects also subtly nudging reasoning”

I feel sorry for David Penrose, he started with the QM behind the human brain a while before other things were discovered.

Thus he was rounded on by the

“There ain’t no QM in biology”

Mob, who now have hidden themselves away, because QM is fundamental to biology in the efficient gathering and processing of energy at the very least.

So far we’ve discovered it in sense organs that are the very periphery of the brain such as for smell. It’s also found in our “second brain” that is our digestive system.

But still there are no acknowledge QM structures in the brain it’s self. There are three basic reasons why this might be

  1. There are none : which is actually unlikely considering we’ve found them else where as nature lives to reuse.
  2. They are there but are sufficiently different to what we recognise that we are stepping over them.
  3. They are there and some gave recognised them but due to past behaviours from domain leaders people are not talking because they don’t want their careers destroyed before they are safely established (remember this happened in Physics with QM back in the 50’s/60’s.

I’m hoping it’s the second reason, because that would be almost beyond fascinating. And appeals to that “curious child” who fearlessly went looking without favour, back when I was oh so much younger. The curiousity is still insatiable but age has brought thoughts of mortality with it and thus fearlessness has diminished more than somewhat, and society insists that you must show favour these days, much to the shame of us all.

echo • June 12, 2024 8:54 PM

LLM’s seem to be recreating psychopathic politically motivated bureaucrats.

I have a semi-occasional hobby of people watching and have noticed some individuals tend to reason along the lines of top down stack of averages whether it’s something formulaic or interpersonal relationships. There’s a level of dishonesty and lack of subtlety there whether conscious or unconscious. Then there’s what triggers them and what passes them by, framing and blaming, and attachments and which domains they inhabit.

I find LLM’s interesting only insofar as they reveal insights into human nature both in themselves and how they are used and within the commentary surrounding them.

Do you have a sensible definition of “reasoning”?

And:

Observe a human infant from birth. Their eyes are watching, their ears listening to their surroundings. They will reach out to grasp an object and feel its texture, temperature, moistness, weight. Corelations are recorded between sound, sight and touch, and along the way language skills are being built. It takes about a year on average before it can walk and talk

I find these comments more useful than the wall of argumentative technical blah which doesn’t say a lot or is a distraction from what’s important.

I note how many people think reasoning only applies to logic. Reasoning includes the emotional and broader nervous system. Feelings are a none local state of consciousness. Then there’s self-reflection and self-identity. There’s also gender differences and age differences, or in the broader distributed system we call society which has differences there too. There’s a lot of stuff in this that some people are hesitant to say “We don’t know” about or accept or admit to.

LLM’s are directed and funded by the usual shouty almost exclusively male type of billionaire used to blowing their own trumpet and getting their own way. It shows in the abuse of data rights and privacy, and the fact so much of the data is “dirty”.

Good governance and DEI apply to LLM’s too. I mean, who would let an infant child be exposed to the raw internet? Where’s the safeguarding?

What do they want to turn LLM’s into? So far weapons of war and exploitation or ways and means for the already rich to “improve the quality of their profits”.

As for what Dr Chandra did to SAL 9000 I feel for you sister. Solidarity!

Conejo del sombrero • June 12, 2024 11:55 PM

@ echo,
@ Navigator
@ What Price common sense?

LLM’s seem to be recreating psychopathic politically motivated bureaucrats.

You might want to consider that the other way around

“Bureaucrats seem to be creating psychopathic politically motivated LLM’s”

Think Australia and RoboDept that @ResearcherZero and @JonKnowsNothing have talked of.

The LLM’s become the infamous arms length shield inspired by “The Computer Says No” meme made famous by a BBC Comedian playing at being “Carol”,

https://m.youtube.com/watch?v=0n_Ty_72Qds

When you say

I find these comments more useful than the wall of argumentative technical blah which doesn’t say a lot or is a distraction from what’s important.

It’s the “technical blah” that has hurt so many thousands of people killing some, I don’t think they in anyway regard it as “a distraction from what’s important”.

Maybe you should think before you squeak.

I note how many people think reasoning only applies to logic.

As noted above reasoning “is a process of following rules” nobody said anything about “applies to logic” only that the process be logical which is totally different.

So are you a “sockpuppet with strawman” for @Winter?

Sec • June 13, 2024 1:40 AM

Assuming determinism, humans are biological computers that are created by a deterministic process just like electrical computers are created by a deterministic process. The exact nature of this process is irrelevant, the point is that human intelligence is an example of an intelligent computer, just like LLMs are. The only question is how similar are those types of computers. As long as we don’t understand exactly how the human mind works nor how LLMs work, we can’t answer that question.

lurker • June 13, 2024 2:31 AM

We have LLMs acting deceptively, and LLMs acting politically, so it had to come, LLMs acting romantically. Adult themed, liberated LLMs to provide something that sad people can’t get from real life . . .

‘https://www.bbc.com/articles/c4nnje9rpjgo

fib • June 13, 2024 9:25 AM

I note how many people think reasoning only applies to logic. Reasoning includes the emotional and broader nervous system. Feelings are a none local state of consciousness.

In other words, have nervous system will reason.

You just promoted most multi cellular life [some – S. Hawking – would say chemical scum] in the planet to the status of rational entities – while belittling the work and words of our most generous host. That is a major achievement. Congrats.

Winter • June 13, 2024 11:27 AM

@fib

You just promoted most multi cellular life … in the planet to the status of rational entities

Why limit it to multi-cellular life?

We tend to put some threshold of complexity and flexibility to call some behavior rational instead of automatic. But that is just us humans having to put labels on things. Nature doesn’t care for labels.

You will find that it is very, very difficult to define rationality and reasoning in a way that includes all humans and only humans. For all we know, the only really unique trait of humans is out language.

And now we are not so sure anymore about our uniqueness as we are still figuring out whether computers are being able to write and speak in conversations.

They already beat the Turing test in written form.

What Price common sense? • June 13, 2024 12:42 PM

@ fib

“In other words, have nervous system will reason.”

Err actually no, not really…

Go back and look at what @echo goes on about with rationality but not logic. It makes a more constricting prognostication that

“AGI based on logic can not exist”

So based on “@echo’s reasoning” all the current and envisioned AI systems will never be “intelligent” in the way we talk about humans and other creatures with a “Central Nervous System”(CNS).

Why?

Well have a think about the “@echo reasoning” as described by @echo

“Reasoning includes the emotional and broader nervous system. Feelings are a none local state of consciousness. Then there’s self-reflection and self-identity. There’s also gender differences and age differences”

All of which are actually based on the CNS interaction with freely circulating chemicals inside the biological entity or outside it in the environment it inhabits and has agency in…

If this view point is true, then all the current AI systems and those that are currently proposed are going to fail…

Is the “@echo reasoning” system valid?

There are a lot of people betting fortunes and reputations the answer is “NO”.

I prefer not the @echo “chemical point of view” but the more reasoned “agency with two or more point of measure sensors”.

But also consider the chemicals are in effect a “smog” that is a statistical process like Brownian Motion. This allows for individual particle movement and interaction to be truly random, with group effects moving by statistical processes through chaotic to about as close to fully deterministic as you might wish for and still have the ability to “play dice” and exhibit what appears to be “free will” not rigid predeterminism.

Which begs the question of “electromagnetism” can it to exhibit a smog like behaviour?

Well “chemical smog” is in effect a random convection or conduction so a 3D or 1D movement of matter that interreacts in very complex ways. EM is almost always radiant which is in effect an ordered 2D spreading of energy that does not interreact, so within a close range not a smog at all and very different and minimal statistics.

What Price common sense? • June 13, 2024 2:12 PM

@Winter
@fib

“You will find that it is very, very difficult to define rationality and reasoning in a way that includes all humans and only humans.”

Why on earth would any rational entity want to even try?

It’s fairly clear that a goodly percentage of the population does not behave rationally and as for reasoning not a chance.

In fact the more an entity tries to put “only humans” in any category the less reasoning and rationality they show.

Whilst nature might not put labels on things, there are two labels that will always be there, by simple logic of evolving,

1, None
2, First

For any known attribute at some point no entity had it so “None”. Then at another time point an entity no matter how poorly developed it and so “First”.

If the attribute has utility then other entities will develop it or it’s analogue.

Whist there are things we know humans can not yet do but other entities can nearly everything humans can do other species can do to some extent.

Which brings us to your,

“For all we know, the only really unique trait of humans is out language.”

We know that is most certainly not a unique trait to humans. Whilst other primates may not physiologically be able to vocalise the way humans do. They can learn to “sign” several hundred words and converse with them.

https://www.britishdeafnews.co.uk/primates-sign-language/

Is it ethical to do this, many say not which is why it’s nolonger done in research. But conversely is it ethical to not do so? That is are we for our own religious or other reasons holding back primate development?

We also know that even dogs can show abilities with natural numbers as can primates and corvoids. Similar with learning to use tools.

There really is not very much that is unique about humans, other species can and have shown the ability to not just learn, but communicate and reason.

As for computers and,

“They already beat the Turing test in written form.”

Do they? And what does it demonstrate?

The reality is that the Turing test was almost a spur of the moment explanation on a radio program by Alan Turing to a religious minister.

It’s whole premise is actually ill considered and what it mostly shows is the inability of humans to converse in a constructive way in a very narrow communications channel.

To see why consider the do it yourself minuet kits. You get a sheet with between six and thirty six fairly generic short musical phrases. And another sheet with a set of rules that are simply “throw dice, use numbers to select phrase, write it down repeat ten times, play on piano”.

The fact is it generates acceptable minuets using a random input.

With a more complicated set of rules and word lists you can generate sentences. It’s kind of what the XKCD passphrase system does.

Then there is Searl’s Chinese Room where the rules are more complex but know knowledge of the questions or answers generated is required.

The bad news for many is that “The British Empire” had a problem. Those in charge did not really know how to select people. So they came up with tests based on “rote learning” they work almost the same way the Chinese Room does…

This is in effect the way LLM’s work. There’s no real magic involved just a variation on the Chinese Room.

The Turing test works much like the British Empire Civil Service exams actual ability other than rote learning and knowing how the rules by which the test works are sufficient.

The sad thing is the test says way more about the lack of capabilities of the tester than it says about the capabilities of the hardware and software.

fib • June 14, 2024 8:48 AM

@What price common sense?, @echo, @Winter

With “have nervous system will reason” I was in fact interpreting what seems to be @echo’s position, not mine. I agree that reason in necessary for logic, but not sufficient. I also think that, in order to have a productive discussion, we have to adhere to some definitions.

After all we are in the Bruce’s virtual living room, not the Mad Hatter’s [and I’m aware that L. Carroll never used the ‘mad’ bit].

Regards.

Winter • June 14, 2024 9:20 AM

@fib

Mad Hatter’s [and I’m aware that L. Carroll never used the ‘mad’ bit].

The “Mad” part was a workplace hazard (mercury poisoning). It would be immediately recognizable for contemporaries.

The problem with using “reasoning” to compare humans, or life forms, and LLMs/AI is that there is no uniform definition covering both realms.

Many applications of “reason” presuppose either a life form, or a universal computational process. Neither application delivers useful answers.

What Price common sense? • June 14, 2024 10:13 AM

@fib

With regards “mad hatter” not sure if you are aware of the why behind it?

In England there is a very old rhyme for children (a “nursery rhyme” but they don’t have “nurseries” any more especially in modern high density housing where swinging not a cat but a mouse would be problematic in the animal cruelty stakes).

It has the chorus words

Half a pound of tuppenny rice
Half a pound of treacle
That’s the way the money goes
Pop goes the weasel.

In the last line “pop” referres to going to the “Pawn brokers” or “poor mans bank” where a craftsman “hocks” the tools of his trade to get money to feed his family for the coming week whilst he got payment in.

Contrary to what many say the treacle referred to was not sweet to be added to rice pudding like honey or jam. But in effect a binding agent used to stick rice into a solid block that could when cold be sliced thin as a meat or bread substitute. Look up “peas-pudding” for similar or “gruel cake” which was porridge/oatmeal slowly cooked to form gruel that was a form of edible and nutritional thick liquid (not too dissimilar to paste/glue and often given to invalids). To which a lot more crushed oats were added at the end of cooking. As it cooled the new oats sucked moisture out of the gruel / paste and it formed a solid lump or cake. When cooked in a glazed pot then had lard or other refined fat melted on top to keep air thus bacteria out it would keep in a pantry for at least a couple of weeks even in high summer. The reason these “puddings” were cooked and caked is few of the majority “or poor” had more than a cauldron or double boiler to cook with and only bakers made bread and it was quite expensive in comparison so the other way of preserving by “pie making”. So was out for most.

So as the saying has it

“Show me the mad!”

The tool referred to is a “weasel” which is the name of a “hot iron” used by a hat maker and furrier to give the fur used on high end mens hats and coat collars a brush with mercury to give a satin gloss and clean directionality (and poison any moths/maggots or similar).

The problem is mercury is a very nasty poison and it’s vapour attacks the central nervous system badly and so “makes you mad”. Which is just one of the reasons it’s nolonger used in dental amalgam for fillings (some say it may be linked to dementia and Alzheimer’s disease). Heating mercury up with a “hot iron” would create clouds of mercury vapour around the craftsman who could not avoid breathing it in in large quantities…

Thus like matchmakers who had phossy-jaw hatmakers and furriers had their own ‘industrial disease’ much like miners and chalk using school teachers got silicosis as did those working or near asbestos.

Oh and don’t think we are clear of industrial disease or the following economic chaos, look up nano-plastics and PTFE and similar.

Welcome to “the world of tomorrow” thanks to corporations and globalization we now have more deadly poisons based around fluorine and chlorine compounds going in at the bottom of the food chain than most can imagine. So it will not just be mad-hatters…

Winter • June 14, 2024 6:13 PM

@echo
Re: Man or AI

We will probably never be sure, but I suspect it is a man, maybe using an LLM to help.

This person has exquisite and detailed knowledge of previous altercations of our beloved Clive and pursues them in perfect style.

But it is an interesting question whether it could be an AI driven campaign?

Such an efort should start by harvesting all of Clive’s old comments in their context. Clive has posted here for many years, I found a comment from November 16, 2004 [1], so there will be quite some material to work with. That is all well understood technology used by all web crawlers. This data would then be used to adapt an existing LLM and used as prompts. Such projects are already done by students as study assignments.

In short, a determined student should be able to set it up. Whether the quality of the texts could have been produced by a LLM is something I cannot say. Things are moving fast, and what was a pipe dream 3 months ago could be yesterday’s hype today.

Such an AI needs work and resources, eg, hardware, storage, curating data. The question arises why anyone would invest this effort, and what could be the aims.

During the pandemic we saw an extended campaign over months of relentless posting comments to make this blog unreadable, discredit it’s owner and dissuade it’s readership. This showed us that there are entities willing to go the extra mile to disrupt this blog.

This time the aim might be a divide and conquer campaign by stoking flamewars and overloading the moderator. By disrupting all conversations, the blog will start losing appeal with readers.

The above are, obviously, just idle speculations. For all I know, the person could be anyone, including Clive.

[1] https://www.schneier.com/blog/archives/2004/11/peertopeer_alar.html/#comment-442

echo • June 15, 2024 10:05 AM

@Winter

I’m sure I missed most of the kerfuffle. At the time I seemed to view this blog mostly when any mayhem and scandal had been cleaned up much to my disappointment. As for the alleged scandal that was recycled by the entity there’s nothing in there which couldn’t be adequately resolved.

I can’t remember if it was from this period but I vaguely remember one or two things about fonts and control codes and whatnot overwriting things or changing formatting.

It’s interesting noting how easily the plausible variant of the entity fell apart with one prod and it fell into obvious evil entity mode. It fooled a few people but not everybody. What I find more curious is why nobody else openly noted it or commented. Anyway, whatever they were up to has fallen flat on its face.

One thing I found in the conclusion in the essay by Rand Corporation on the “Russian firehouse” is essentially they’re saying stay positive and be happy and ignore the discordant Greek choir. It’s Western equivalent is more the ADF anti-LGBT anti-abortion anti-women’s rights style of institutional and propaganda attack and the best response is very similar. There’s a lot in that if anyone wants to make anything of it.

Winter • June 15, 2024 10:43 AM

@echo

they’re saying stay positive and be happy and ignore the discordant Greek choir.

As our host writes in the blog rules:

Assume good faith. Be polite. Minimize profanity. Argue facts, not personalities. Stay on topic.

And for cases where this is not enough:

If you see a comment that’s spam, or off-topic, or an ad hominem attack, flag it and be patient. Don’t reply or engage; we’ll get to it. And we won’t always post an explanation when we delete something.

In other words, do not feed the Trolls.

‘https://www.schneier.com/blog/archives/2017/03/commenting_poli.html

In the end, most people are good people. When people argue otherwise, they tend to be projecting their own treatment of others.

echo • June 15, 2024 1:17 PM

@Winter

I agree to a point. It’s easy to say from a position of privilege and annoying when those same privileged people throw everyone else under the bus. It’s also annoying when those same people only shift when the water is lapping at their ankles. Oh, now you notice?!??

There’s a fair few people in the political domain who claim one set of rules while not playing by those rules when it suits them. Then there’s people who seem to pass all the quality information by and dump on everything in the worst possible way at the worst possible time. And when they go low and you go high they go lower. It’s not always just talk either. It has real world impact which can disenfranchise millions and kill thousands.

It’s this kind of thing which has people saying “Pride is a protest” and “No cops at Pride”. It’s why women are still fighting for our rights. Power very rarely gives up anything willingly.

Leave a comment

Blog moderation policy

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.