Schneier on Security

It’s not surprising that the press would pick up on an Existential Risk that is nebulous and sounds far-off or difficult to imagine. Climate Breakdown (and its associated problems, such as disease vectoring and zoonoses) is increasingly killing folks, and AI poses a great, abstract, and, mostly, apolitical distraction from our by-now very good knowledge on what the future holds for Climate Breakdown. The press knows that it has to make people feel engaged and help them defuse their building existential terror, FWIW...

@Moderator

comment-422570 Julian Kris • June 2, 2023 10:58 AM

Unsolicited advertising

Whoa, cyberattacks are getting intense! Montenegro’s experience sheds light on the growing threat landscape. Makes you wonder how nations can bolster their defenses! what colors can deer see

@ Anonymous, ALL,

Re : Agency of entities.

“That sounds a lot like…”

Whilst inanimate object do the damage, the question of who or what caused the object to do the damage.

Law follows the path back from the object to either a directing or negligent mind that has “agency”. If they can not then the old “act of god” or “accident” is invoked instead.

The fact you are holding an object that causes harm, does not automatically make you guilty of any crime etc...

@LeeC

Software is not inherently risky — it all depends upon how HUMANS choose to use it.

That sounds a lot like “Guns don’t kill people; people do”.

I certainly think one of the risks is people mistaking chatbots for intelligence (or sentience, a word that soneone upthread used about AI).

@ modem phonems, Winter,

Re : What is truth?

“Even Popper has to implicitly or indirectly acknowledge truth.”

Truth unfortunately has so many meanings it is ambiguous, and worse gets different meanings in different knwoledge domains, that makes it Orwellian in nature.

It is said that,

“If something is not true then it is false”

Which is actually both true and false as a statment.

It is true in “bivalent systems” but not in other systems. We see this in Scottish law where there is not just Innocent or guilty as verdicts, but also “Case not proven”. The meaning of the latter depends very much on who you talk to. In the official view it is the same as “innocent” in the MSM and more general eye it means “guilty” but with insufficent ecidence presented to meet the burden of proof of “beyond doubt”...

@ modem phonems, Winter,

“Medical science now recognizes that from the moment of conception there is a complete human being which integrally directs its own development. There is no point after conception where anything but growth occurs, no later point or interval at which there occurs a “not-human” to “is-human transition.”

The same argument applies to “malware” such as worms and viruses, and any other self replicating automata such as Conway’s codes for his “Game of life”. So unless you are arguing such code also has “a soul” then what is it’s relevance as an argument?...

@ Coolest Hottie,

Re : Irony

“yet bragging about knowing everything about everything just makes me laugh hysterically. “

Two things for you to consider,

1, I base what I say on reason, something that most should be capable of.
2, Many who don’t reason are simply holding up a mirror and saying what they see.

Thus each time, you make one of your gaslighting attempts, who are you actually burning in the flame?...

@ Winter,

“Not in the country of the “pro-life” crowd.”

Douglas Adams indirectly refrenced this in one of his books. Where he imagined someone so sick of the insanity he saw around him, he built a metaphor. That is of an inside out building and invited people to come in outside of the asylum…

@ Winter, modem phonems, ALL,

“Given that Aristotle wants to know the one and only truth of the universe,”

He was not the only ancient philosipher with what now appear to be very odd notions.

There was Plato and his “One perfect form” notion[1].

His argument being that for any conceivable object there was “the one perfect” form/example, be it a grain of sand or a woman.

To us in this more modetn age it’s obviously a false argument, because to decide if something is the perfect example you need atleast two things,...

The only thing that is scarier than the rogue AI drones would be those designing them
IF they happen to have an EGO that is soooo BIG that they do not care to write code
that runs the drone’s OS even though they know that they CANNOT SPELL for shyte.
Blah blah blah – I’m a superstar in a comment section of a liberal blog.
Not knowing basic things such as correctly spelling words in English, in an English-speaking forum/online community – yet bragging about knowing everything about everything just makes me laugh hysterically. Thank you for that, oh you almighty clive robinson...

@ Austin,

Re : Junky Hypothetical.

“A simulation using AI decided to kill the operator so it never got “no” to missions.”

Firstly and importantly the claim is “no such test happened”.

But hypothetically lets assume something similar did happen.

Imagine if you can –and such people do exist– a drug addict / junky with mental disorders equivalent to what used to be called a psychopath.

Humans like many biological systems run on a pain-pleasure reward process for various actions...

https://www.foxnews.com/tech/us-military-ai-drone-simulation-kills-operator-told-bad-takes-out-control-tower

@modem

Even linear models have this problem. The model should be adapted to and reasonable for the data it is attempting to reduce

The weight updates occur automatically, in analogy to natural learning, inside the model, without recourse to out of model update schemes like backprop.

I am sure you will have a brilliant career ahead of you when you demonstrate such a network learning, e.g., automatic speech recognition using one of the existing speech corpora...

@ Winter

there are currently not that many ways to feed a big neural network a lot of data and get it to learn something.

The multi-layer deep learning networks and back-propagation weights update are a kind of general purpose ad hoc nonlinear analog of linear multi-factor models. It’s not surprising that they tend to learn inefficiently. Also the mere fact that they eventually model training data adequately is not a guarantee of their soundness. Even linear models have this problem. The model should be adapted to and reasonable for the data it is attempting to reduce [1]...

@modem

He claims his account bears on reality. Otherwise, why does he say anything?

No, he claims his account bears on observed reality. That is, the observations all reasonable persons can agree upon.

So some truth is always available.

But this truth is unobservable, and therefore, reasonable persons can, and do, disagree about these truths. In fact, they never ever end disagreeing.

So what is this unobservable truth that every reasonable person can agree upon? It is just as elusive as the real and truthful single curve connecting the known points...

@modem

Although nature provides a beautifully adapted environment as the typical environment for the infant, the infant could in principle survive outside the mother if arrangements were made for its nourishment and other necessities.

That is still to be seen. However, it can already be done for individual organs. If a liver or heart can be kept alive outside the body, is it a creature? If an embryo is a creature, why force a woman to carry it against her will?...

@ Winter

If it is a complete human, it can survive outside the body of another human. If not, then it is part of that body.

The parts of the containing body (the mother) are organs like hands, lungs, blood, liver etc. They are integral to that body. Clearly the infant human in the womb is not of this type. Its presence inside the mother is purely at the accidental level, not integral to the mother. Although nature provides a beautifully adapted environment as the typical environment for the infant, the infant could in principle survive outside the mother if arrangements were made for its nourishment and other necessities. And the integrity of the mother and the infant are preserved either way...

@ Winter @ Clive Robinson

How can you know the truth

Even Popper has to implicitly or indirectly acknowledge truth. He claims his account bears on reality. Otherwise, why does he say anything?

Regarding the mathematical illustration involving points, it is true that some curves go through the points. If one were to claim or look for a “true” unique such curve, mathematics would truthfully point out there is no such thing...

@modem

Medical science now recognizes that from the moment of conception there is a complete human being which integrally directs its own development.

That is literally how every organ develops and works. If it is a complete human, it can survive outside the body of another human. If not, then it is part of that body. As an embryo has an even shorter time of surviving outside the body as a liver or heart, it is certainly not more of a complete human being...

@modem

Real neural structures resemble certain classes of parametrized recurrent dynamical systems whose parameters’ time evolution in training moves without supervision to a fixed state, after which system output is a stable convergence based on initial data [1].

Stephen Grossberg’s ideas and models are certainly interesting. The problem is that there are currently not that many ways to feed a big neural network a lot of data and get it to learn something...

@ Winter

part of an entity

Medical science now recognizes that from the moment of conception there is a complete human being which integrally directs its own development. There is no point after conception where anything but growth occurs, no later point or interval at which there occurs a “not-human” to “is-human transition. By continuity this human being is then a person just as it is at any later time. It is for a time dependent on the mother for nourishment, but that does not distinguish it in any essential way that would bear on its humanity from the dependence of all human beings on obtaining nourishment...

@Clive

The use of “creature” implies a “whole entity” capable of independent existance, not something that is just “part of an entity”.

Not in the country of the “pro-life” crowd. There a “part of an entity” that cannot exist outside a body can even have personhood.

@Clive

There is already an alleged case of a lawyer using an LLM to make a legal argument and getting caught out.

Apparently he was a defence lawyer. Now ask yourself if the same questions which led to this LLM use would have be made of a prosecution lawyer?

It was a civil case, no prosecutor in sight.

The point here was not that the lawyer used an LLM. The point was he submitted the brief with “Six of the submitted cases appear to be bogus judicial decisions with bogus quotes and bogus internal citations,”...

@modem

In fact the classical account is not about “systems” but about “what is” and truth.

How can you know the truth?

There is a simple mathematical equivalent to Popper’s falsification argument.

You are given a plane with a finite number of points on it. Say, these are measurements of a certain phenomenon in a graph.

You know these points lie on a line in the plane, a graph or function. You want to know which function describes the position of the points you have, and an infinite number of other, currently unknown points on the line. That is, you want to know a theory that describes the phenomenon...

Federal Court of Australia judge dismissed the entire defamation claim brought by former celebrity SAS soldier Ben Roberts Smith. The court ruled, the two journalists and the publishers were telling the truth by claiming the soldier was a murderer and criminal.
The trial was the longest running and most expensive defamation (or, even civil?) in Australias history. Estimated to cost 35 million dollars (!)...

@Clive Robinson

Thanks for your comments.
What you discuss about natural persons, applies to the legal system.
Ultimately however the legal system is fiction. By this I refer to the two fundamental jurisdictions of the Private and the Public.
The Private is the living. The Public is the dead. Private is substance. Public is fiction

Things in the Private : Living men and women. Money of substance eg precious metals versus fiat currency of the Public...

https://www.vice.com/en/article/4a33gj/ai-controlled-drone-goes-rogue-kills-human-operator-in-usaf-simulated-test

I can’t wait to hear your take on this. A simulation using AI decided to kill the operator so it never got “no” to missions.

@ lurker

physical limit to what these machines can do, and it’s their efficiency, or lack thereof

Federico Faggin remarked in a talk on neuromorphic computing that if the human brain were as inefficient as computers, it would vaporize itself the moment it was “turned on” 😉 .

@ modem phonems, Winter, ALL,

Re : The flight of an arrow.

“I don’t know, as I don’t understand Popper. In trying to read him I can’t see how he adds anything of central importance to the classical account of science as knowledge via causes.”

Consider “A cause” has “an outcome” and is repeatedly testable by anyone that choses to carry out the experiment.

The controling conditions are found by changing the starting causes untill the effect is measurably different. That is it has been falsified...

Dangers of AI:
* use in warfare, e.g. drone swarms learn to attack better with each attempt
* use in phishing, so that the AI designs emails or texts, learns which wording works better, and constantly improves
* surveillance of a large population using AI, supercomputers, and data from our digital lives
* use in hacking to find new vulnerabilities
* use in investing; an AI might find a way to make a large profit while also crashing the stock market...

@ Erdem Memisyazici, modem phonems, Winter, ALL,

Re : A part is not the whole.

“A neuron is a living creature”

It’s not.

The use of “creature” implies a “whole entity” capable of independent existance, not something that is just “part of an entity”.

That is unlike a bacteria, a neuron is not capable of the basics we consider necessary for a living creature.

That is it does not,

1, Exist as a single cell...

@ Louis Conover, ALL,

Re : Who to watch most, the corporates or the self entitled?

“Corporations and bureaucratic organizations that pay for AI in order not to pay humans to make decisions aren’t going to be paying anyone to run a parallel decision making process that can be queried or challenged.”

You left out “guard labour” of Law Enforcment, judiciary, and to a lesser extent the millitary, MIC and IC...

Essential hardware tech to cool those overheated AI brains:

“The 2.8 × 27.5 × 41.5mm widgets – imagine a slightly elongated SD card and you’ll get the picture – contain what the maker describes as “tiny membranes that vibrate at ultrasonic frequency.”

https://www.theregister.com/2023/06/01/frore_systems_airjet_cooling/

I think it poses a similar risk to pandemics and nuclear war—which is to say, a risk worth taking seriously, but not something to panic over.

I don’t think you are using parity classifications. Machine Learning (which is all we’re talking about, and is a far cry from AI) is just a broad tool, whereas the other topics are identifiable threats that come from other tools (medicine and physics). ML is only a “risk” in the same way that any other tool is a risk: when it is wielded as a weapon by humans...

Re: neural networks

The popular AI seem to deserve the characteristic “neural” minimally, as they resemble natural neural structures only to the extent that they have “lotsa connexions. Real neural structures resemble certain classes of parametrized recurrent dynamical systems whose parameters’ time evolution in training moves without supervision to a fixed state, after which system output is a stable convergence based on initial data [1]. These are understandable and have little of the instability in presence of new data and statistical optimization “black box” inscrutability of current AI designs. See Stephen Grossberg’s work...

Any harm that AI does (and there are no inherent practical limits on such harm) will be because humans believe it. This is a problem that in principle has a solution.

I am still very concerned on the privacy aspects. Large AI leads to the Panopticon of surveilling everyone on the planet.

just having the word “pregnant” in a post or email is going to put you on the list to watch soon, and i think it’s still a 3 link chain authorized on your contacts. Even if you have legal abortion in the state or country you live, that word going to trigger surveillance to make sure you are taking vitamins, or your friend who just bought a test kit, is going to link you from her contact list...

@ &ers, SpaceLifeForm, ALL,

Re : Scott Shapiro about his new book.

What can I say about what the ARS tecnica interview[1] that I’ve not said on this blog before?

He sounds like he and I think alike to what many might consider an alarming degree (even to the joke about “whom made gods” and it’s implications).

Though I don’t hold Alan Turing as the person who proved Security is impossible, as you know I nominate some one slightly earlier in the 1930’s Kurt Gödel. Though going from what Gödel proved to the same conclusion is a lot harder than using either Turing or Church’s proofs...

There’s another physical limit to what these machines can do, and it’s their efficiency, or lack thereof. The human brain runs on a few tens of watts. Up to a couple of hundred watts more are used for the fuelling mechanism. Current “AI” machines consume megawatts. Where’s this power going to come from? Will the AI just write a cheque?

@ Winter

can it be falsified

Can any cause inferred from effects be falsified ?

I don’t know, as I don’t understand Popper. In trying to read him I can’t see how he adds anything of central importance to the classical account of science as knowledge via causes.

Popper says in The Logic of Scientific Discovery

“In other words: I shall not require of a scientific system that it shall be capable of being singled out, once and for all, in a positive sense; but I shall require that its logical form shall be such that it can be singled out, by means of empirical tests, in a negative sense: it must be possible for an empirical scientific system to be refuted by experience.” ...

@modem

It’s presence is inferred from what we observe in the activity of the living. This activity can not be explained solely as an accumulation of material properties.

The question is, can it be falsified?

@ Winter

Unless you can come up with something actually observable, this is nothing but magic in disguise.

The soul is indirectly observable. It’s presence is inferred from what we observe in the activity of the living. This activity can not be explained solely as an accumulation of material properties.

Modern science in this regard is in the situation illustrated by Sidney Harris’s famous cartoon about explanation...

@modem

• 1 It doesn’t answer any fundamental question.
• 2 But there still has to be additional form that activates this bodily potential.

Do you see how 2 does not help with 1?

But there still has to be additional form that activates this bodily potential.

Unless you can come up with something actually observable, this is nothing but magic in disguise. Magical thinking has not brought much progress to human knowledge...

@Ismar

What, the gummint interfere in commerce? willing seller, willing buyer …

And as others have said, our own spooks need to get into our gear.

[…] vulnerabilities and reviewed Eclypsium’s findings. He compares the state of affairs to the Sony rootkit scandal of the mid-2000s. Sony had hidden digital-rights-management code on CDs that invisibly put in itself on customers’ […]

@ Winter

Humans have created viruses and a bacterium using the components. … Also, take any living thing and when you mess with its components it changes in predictable ways. … electrodes …

Cf. [1], [2]

It seems various replacements by synthetics parts or subsystems is so far all that has been achieved.This just says that if you change the body you change the range of potential activity. It doesn’t answer any fundamental question...

@-*
“I can see an AI geting fooled, but an aware human?”

Can you be sure of finding an aware human when you want one? who actually cares about what is happening?

@Erdem

A neuron is a living creature.

So is a E. Coli bacterium. But that has no relevance for whether GPT can make intelligent decisions. I think the level of intelligence displayed by E. Coli bacteria is fairly limited. The same can be said about individual neuronal cells.

Cellular technology – the Internet in most people’s pockets – has aready started a transformation on the scale of Gutenberg and Luther. For the same reason: mass communication became available to the masses.

In the era of George Floyd, accountability for systemic violence has increased dramatically.

LLMs are an extension of the revolution that is already in progress. For better or for worse, but eventually, for better...

@Louis

It’s not possible to query the system or ask the system to explain itself in detail.

Explainable Machine Learning investigates how AI can be constructed for which humans can determine how decisions are made.
‘https://en.m.wikipedia.org/wiki/Explainable_artificial_intelligence

Contrary to common lore, scientists and users of AI are not fools nor stupid. Many professions refuse to use AI if the reasoning behind decisions are not clear. For instance, most clinical specialists [1] refuse to delegate decisions about patient’s treatment to opaque expert systems...

@Winter

“And human brains are just connected neurons with adaptable connection strengths.”

Not quite. A neuron is a living creature. GPT is just tokenized short sentences with some entropy calculation. Not even comparable.

Everyone is focusing on the wrong thing. It’s not what AI does that’s dangerous, it’s how it does it. A neural net is inherently uninterrogatable. What it knows and how it behaves is the result of a pattern over a very large number of connection weights which are individually meaningless. No one, not even the system’s creators, can examine those weights and predict or understand what they mean or how they are individually connected to the output of the system. It’s not possible to query the system or ask the system to explain itself in detail. When we hand over any decision making power we effectively surrender our ability to understand how those decisions are made. Corporations and bureaucratic organizations that pay for AI in order not to pay humans to make decisions aren’t going to be paying anyone to run a parallel decision making process that can be queried or challenged...

@modem

It refuses to notice that the properties of the living can not be accounted for solely on the basis of the material body.

Humans have created viruses and a bacterium using the components.
‘https://www.theguardian.com/science/2010/may/20/craig-venter-synthetic-life-form

Also, take any living thing and when you mess with its components it changes in predictable ways. When you account for all the components of viruses or bacteria, there is nothing left unaccounted for...

@ Winter

where in the kingdom of living things does that soul enter biolog

The Aristotelian argument applies to all living. The “hard conclusions” are what are avoided in “modern” science. It refuses to notice that the properties of the living can not be accounted for solely on the basis of the material body.

The body only provides the suitable matter so that there is a potential for life. Some additional factor, which in Aristotle is called the ”form”, that activates this potential in a unified way has to be found. In living things, the form is called the soul. It provides the characteristic activity of the living. The form can not be material because the material aspect or potentiality is accounted for by the body...

@Erdem Memisyazici

Not too long ago we called that Big Data.

Bigger data?

I mean what we call A.I. is really just human data enumerated with some context.

And human brains are just connected neurons with adaptable connection strengths.

GPT just learns to guess the next word in a text, any text. GPT4 does the same also for pictures. But I think no one expected that the resulting LLM could write essays based on questions...

@Winter

Not too long ago we called that Big Data.

@Clive @SpaceLifeForm @ALL

hxxps://arstechnica.com/features/2023/05/is-cybersecurity-an-unsolvable-problem/

Sounds like you got caught in a soundbite. I mean what we call A.I. is really just human data enumerated with some context. I came across an author who wrote a project called nanoGPT and it really breaks down the concept for the average person as an introduction to A.I. More people understand that funnier this “open your third eye with A.I.” culture sounds. 😄

@markm

If ChatGPT is the current state of the art for AI,

The current state of the art is GPT4 which is two orders of magnitude bigger than ChatGPT. It is also reported to be much more powerful than ChatGPT.

‘https://medium.com/geekculture/gpt-4-100x-more-powerful-than-gpt-3-38c57f51e4e3

One thing I am musing about is, with sufficient knowledge of the fingerprint matching algorithm, would it be possible to generate fingerprints that have a higher acceptance rate. According to apple, the expected rate is 1 in 50,000 false positives, I wonder if it is possible to beat that…

If ChatGPT is the current state of the art for AI, the only thing to worry about is fools and lazy people taking the output seriously. It’s just ELIZA with a bigger database, still manipulating words with no idea what the sentence means.

Hi Schneier, have you considered the threat of cascade failure that might occur as a result of AI driven inequality stalling the economic cycle between product and factor markets?

Historically, when people are unable to get food, unrest ensues, and by the time its noticed it can’t be stopped. In almost every case the driving forces are not noticeable until it was too late to stop it. We also as a species are very bad about how to handle cascade failures in processes which we do not have a full understanding. In almost every documented case I’ve seen (and full disclosure I haven’t gone looking in a research capacity but I read a lot), cascade failures are generally considered non-issues until something happens. In the case of an internet spread free replacement for labor; that seems pretty hard to stop once the genies out of the bottle...

… the “AI is Scary” Chicken-Littles are an amusing but tiresome crowd of emotional fretters.

AI is somehow a big catastrohic threat to them, but they daily ride care-free in automobiles — which statistically are by far the biggest risk to their lives.

“AI” is simply an artificial set of inert Zeroes & Ones — totally risk-free to humans. … very unlike pandemic viruses & nukes.

Software is not inherently risky — it all depends upon how HUMANS choose to use it...

I’m sorry to hear that; I think the extinction risk at this point is approaching “Yes” and that lack of security mindset among leading players is playing a giant role in that. I offer to have a conversation on the topic, private or public, at your convenience.

You may enjoy reading, if nobody has sent it to you already:

https://www.lesswrong.com/posts/8gqrbnW758qjHFTrH/security-mindset-and-ordinary-paranoia...

Legit app in Google Play turns malicious and sends mic recordings every 15 minutes

hxxps://arstechnica.com/information-technology/2023/05/app-with-50000-google-play-installs-sent-attackers-mic-recordings-every-15-minutes

@Vesselin Bontchev

Don’t miss Bruce’s links to Kieran Healy 😄

@ Bruce,

Another one to read, is from “Katherine Fidler” in the UK’s Metro “give away paper”.

The thing is the printed version is quite a bit different to the online version,

https://metro.co.uk/2023/05/30/artificial-intelligence-poses-risk-of-extinction-experts-warn-18868186/

The following paragraph is in the print version (as para 4),

“The experts are worried about the threat to jobs posed by the machines –and resulting political and economic upheaval– rather than that they will make up their minds to kill us.”...

I am not afraid that AI will become as smart as (or smarter than) humans. I am afraid that humans will become as stupid as AI.

@modem

So, yes, the agent I was referring to in the case of a living thing would be the soul.

It is obvious that non-physical, or meta-physical, and unobservable things are outside the range of science. Being unobservable, they also cannot be simulated on a UTM (quantum or not) because there is nothing to base the simulation on.

If a non-material Soul is needed to make animals function, where in the kingdom of living things does that soul enter biology? Nematodes? Sponges? Fungi and Plants? Plants do move, and fungi grow in directed ways...

@ Winter

Is this “agent” of yours non-physical Vitalism or Soul under another name?

Aristotle’s De Anima is translated as “On the Soul”. This is an application of his treatment of causality and nature when it involves living things. So, yes, the agent I was referring to in the case of a living thing would be the soul. I am not familiar with the vitalism you mention.

All I can say is: “People living in a glass house shouldn’t throw stones.” NSA’s efforts keep going in the offensive… I don’t see what benefit we’ve gotten out of that, other than keeping people busy and money flowing

@MarkH, @Clive Robinson, & @Seriously:

1 U.S.C. 1 and (I haven’t read this) perhaps 18 U.S.C. 2510(6) do not endow A.I. with personhood.

A company that uses A.I. may be liable in court for what its A.I. does, and the A.I. may be all over the evidence and the arguments to judge and jury, but the A.I. itself is not liable in court. There has been a lawsuit against a car, but I think it was really against an unknown owner and I think could have resulted in seizure of the car. A car has capabilities for its safe operation, but safe operation of the car is the responsibility of the human driver, and if the driver is a robot then the owner or operator of the robot has that responsibility, and that’s who would be sued. Likewise for A.I...

As they say: Do not go to war with the people that make your phones.

Russia says U.S. accessed thousands of Apple phones in spy plot
‘https://www.reuters.com/technology/russias-fsb-says-us-nsa-penetrated-thousands-apple-phones-spy-plot-2023-06-01/

Russia’s Federal Security Service (FSB) said on Thursday it had uncovered a U.S. National Security Agency (NSA) plot using previously unknown malware to access specially made so-called backdoor vulnerabilities in Apple (AAPL.O) phones...

@ Bruce, ALL,

Re : Technology is agnostic to use.

“I actually don’t think that AI poses a risk to human extinction. I think it poses a similar risk to pandemics and nuclear war—which is to say, a risk worth taking seriously, but not something to panic over.”

As far as I’m aware no technology mankind has made so far actually poses a direct risk of human extinction, nature on the other hand…

As time goes by machines become capable of doing more and more of the tasks that once defined a individual’s profession, livelihood, and often self-esteem. Chess grandmasters and 10-dan go players are a brave and noble few. Deep Blue and Alpha Go were greeted as a technical marvel and a curiosity rather than as a doomsday threat. Only a handful of people, if any, make a living telling cats from dogs so that, quite miraculous, breakthrough ruffled few feathers. Now we have machines that can write prose and the pundit-sphere explodes. Everyone loses their minds. Well, everyone who writes prose for a living loses their mind. Present company excluded...

@modem

Some other agent brings the potentiality into actuality.

All biological systems are physical systems. There is atoms and their organized behavior. Nothing more is needed to understand and explain all biological phenomenons. And as all of reality is quantum computable, a Quantum UTM should theoretically be able to simulate any creature.

Is this “agent” of yours non-physical Vitalism or ...

@Ismar “stronger security mechanisms on SOHO routers” aka User is the enemy, trust us, we know best.

Nothing desensitizes one more to the romanticism of the Great Artificial Intelligence Takeover than spending days on end painstakingly placing squares around bulls and cows grazing in photographic images of a squalid savanna(*). In the drudgery of labeling you get to see the guts of the system. It is interesting that greater attention is not paid to this somewhat shady part of the industry, where real humans are involved, often outside the acceptable bounds of human dignity[1]. By experiencing this painful process a clearer perspective unfolds...

@ Ismar, ALL,

Re : Network edge devices as proxies.

“Wandering if there is a silver lining here where governments/ agencies will push for stronger security mechanisms on SOHO routers sold in Five Eyes countries?”

As the NSA and GCHQ are known to avoid going into any targets leaf networks where ever they can, I assume the same for the other “extended Five-Eyes” SigInt club members (other IC entities who knows). That is the SigInts will continue to infest the Internet / Cloud from behind the “first node upstream” of a target so the target nor most others can see them snooping (or Spooking depending on your prefrence)…...

@All wondering how hardware security works and how you can share a private key between multiple HSMs.
See how IANA handles root keys for the DNS system. That’s exactly how it’s done.
Hardware keys are still now in 2023 at the root of Internet.

It involves a lot of hardware packaged in tamper-evidence bags stored in keyed boxes within safes inside cages in secure buildings. And a lot of logistics and organizing to gather each time a different subset of the people who each have custody of only parts of the keys and codes that give access to all that hardware and who are dispached all over the world...

@ modem phonems, Winter,

Re : What is a machine.

“For a machine, the state of the UTM is all there seems to be evidence for. The reality of the machine stops there.”

Two things to note,

1, Many machines function without being a UTM.
2, There are machines that are rather more than a UTM.

There are two essentials we know of so far that are not in a UTM,

1, Non Determanistic behaviour.
2, Non scalar basic storage elements...

@ MarkH, ALL,

Re : Legal and Natural Persons.

“Just for reference, from 1 U.S.C. § 1”

Yup that covers the,

“Any person legal or natural”

Requirments for the Federal Government of the US.

Though the tail end of,

“as well as individuals.”

Speaks volumes as to the priorities of Congress…

@ ALL,

In the report linked to by @flash above you will find the following,

“This backdoor appears to be implementing intentional functionality and would require a firmware update to completely remove it from affected systems.”

Yes it’s intentional, BUT it is a result of a required “BOOT hole” not a “backdoor”. As a result it may not be possible to remove the “hole” without loss of required functionality...

From Wired article:” The group, for instance, uses hacked routers, firewalls, and other network “edge” devices as proxies to launch its hacking”.
Wandering if there is a silver lining here where governments/ agencies will push for stronger security mechanisms on SOHO routers sold in Five Eyes countries?

@ Winter

… the state of the neural system and the chemistry of its body [1]. That is it. Other animals are more complex, but not different.

In animals there is the body, which includes the neural system, and perhaps this total physical reality could be simulated by a Turing machine, a Turing robot. But this only provides a potentiality for the animal’s activities. It does not by itself account for the actual movement and rest and observed behavior in general of the animal...

Continued..
@modem

For the animal (including human) there is more than the state of the neural system.

But you have still not presented a workable definition of intelligence. You claim “artefacts/machines” cannot be intelligent because of some undefined “more than the state of the neural system”. That does sound to me much like the mysterious “life force”, or vitalism, from the 19th century [1]. Vitalism was decidedly unhelpful in understanding the biology of living creatures. I think your ...

@modem

For the animal (including human) there is more than the state of the neural system.

For a sea slug, there is the state of the neural system and the chemistry of its body [1]. That is it. Other animals are more complex, but not different.

[1] ‘https://www.technologynetworks.com/neuroscience/news/sea-slug-study-shows-how-the-body-learned-to-move-372637

Your argument is only valid if you can prove that a body cannot be simulated by a stochastic UTM of the required complexity...

@ Winter

Therefore, being able to be simulated on a UTM is not enough to argue AI cannot be intelligent.

For a machine, the state of the UTM is all there seems to be evidence for. The reality of the machine stops there. There is nothing further.

For the animal (including human) there is more than the state of the neural system. Animals clearly act in a way that exhibits they are aware of a reality that exists outside themselves...

@Phillip

My wife’s medical records were altered by a human. Apparently they thought it might help to give some cover to malpractice. It didn’t work. They altered her ethnicity, marital status, and place of birth. All items that were quickly discovered, and rather apparent to be incorrect.

People also do make assumptions and mistakes, or enter data incorrectly. It is not always done on purpose. One mistake can perpetuate it further...

@Clive, Nick:

Just for reference, from 1 U.S.C. § 1:

In determining the meaning of any Act of Congress, unless the context indicates otherwise … the words “person” and “whoever” include corporations, companies, associations, firms, partnerships, societies, and joint stock companies, as well as individuals

Supply Chain Risk from Gigabyte App Center Backdoor (May 31, 2023)

#

Recently, the Eclypsium platform began detecting suspected backdoor-like behavior within Gigabyte systems in the wild. These detections were driven by heuristic detection methods, which play an important role in detecting new, previously-unknown supply chain threats, where legitimate third-party technology products or updates have been compromised. Our follow-up analysis discovered that firmware in Gigabyte systems is dropping and executing a Windows native executable during the system startup process, and this executable then downloads and executes additional payloads insecurely. It uses the same techniques as other OEM backdoor-like features like Computrace backdoor (a.k.a. LoJack DoubleAgent) abused by threat actors and even firmware implants such as Sednit LoJax, MosaicRegressor, Vector-EDK. Subsequent analysis showed that this same code is present in hundreds of models of Gigabyte PCs. We are working with Gigabyte to address this insecure implementation of their app center capability...

@Clive Robinson:

Your post of 7:53a yesterday: Good, but that doesn’t change what I wrote: A contract is not enough to grant personhood to an entire type of entity that wouldn’t otherwise have it. So, if an LLP is created by a contract that creates an entity and grants LLP-hood to that entity, that’s under superior law that the type of entity that can be described as an LLP will have personhood. Personhood that is of other than natural persons is controverted in the U.S., and that controversy would become much more intense should two people be able to confer personhood by contract alone upon wholly new kinds of entities, such as A.I., alligators, ...

Part 2
@Mr. Peed Off, All

The following is adapted [2] from:
‘https://www.forbes.com/sites/lanceeliot/2023/02/26/legal-doomsday-for-generative-ai-chatgpt-if-caught-plagiarizing-or-infringing-warns-ai-ethics-and-ai-law/
(note the question at the end)

Some have been ardently complaining that students are potentially ripping off humans that have created content. You see, most students are data trained by examining data found on the Internet ...

Second try:
Part 1
@Mr Peed Off, All

In a surprising move, Japan’s government recently reaffirmed that it will not enforce copyrights on data used in AI training.

The sense of reason prevails in Japan! But I am afraid the voice of reason will be drowned out in the courts for some time to come. [1]

Did you realize that JK Rowling, Dan Brown, Lucinda Riley, and many other established authors learned their craft from reading copyrighted materials? Also, countless students all over the world scourge the internet reading all kinds of copyrighted materials to train their brains to produce seemingly new texts and works of art. But in reality, these “new” works are based on the copyrighted materials they used to train their brains...

@Mr. Peed Off:

“In a surprising move, Japan’s government recently reaffirmed that it will not enforce copyrights on data used in AI training.”

Maybe, not that surprising.

All human menories hold some copyrighted information in some form, like that tune people can not get out of their head or add jingle or much else. So much so that you could argue that a human mind usefull to society has to contain copyright information...

@lurker, all

Yes, great observations. The focus on Guam is pointed, though the mention of critical infrastructure more broadly does make me curious about the scope of the activity.

For working so intentionally to avoid detection and attribution, I wonder if these public alerts will register as a small setback for Volt Typhoon.

@Clive:

You introduced me to the CORDIC system used on early electronic calculators; I just learned a computational tid-bit you might find of interest.

The earliest inertially-guided long-range U.S. missiles used an algorithm called “Q guidance,” apparently derived by holding the time-to-target at some fixed value.

The missile didn’t use Q guidance because time-to-target was critical, but rather because the computational power that could be fitted on a rocket was so limited. The algorithm uses a “Q matrix” (or really, matrices because they vary somewhat through the trajectory)...

In a surprising move, Japan’s government recently reaffirmed that it will not enforce copyrights on data used in AI training. The policy allows AI to use any data “regardless of whether it is for non-profit or commercial purposes, whether it is an act other than reproduction, or whether it is content obtained from illegal sites or otherwise.” Keiko Nagaoka, Japanese Minister of Education, Culture, Sports, Science, and Technology, confirmed the bold stance to local meeting, saying that Japan’s laws won’t protect copyrighted materials used in AI datasets...

Why is it now called ‘Living off the Land’?

It used to be called

‘Executing system utilities at the command line interface.’

Something that goes back into the 1970’s or 1960’s with serial line access and 300baud modem access.

So it appears to just be ‘blending in’ old school style by not leaving traces on magnetic media… Activities that used to be called ‘hiding down in the grass’ going back to the early days of radar befor PPI...