Schneier on Security

vas pup • April 22, 2024 5:08 PM

Daniel Dennett: ‘Why civilisation is more fragile than we realized’
https://www.bbc.com/future/article/20240422-philosopher-daniel-dennett-artificial-intelligence-consciousness-counterfeit-people

“While complete facsimiles of the human mind may not be imminent, the way we’re
using AI to impersonate human beings has, he told me, already put us on a dangerous trajectory. He called such AIs “counterfeit people”, and told me that
rolling out such entities en masse constituted “mischief of the worst sort”: a form of “social vandalism” that should be addressed by law. Why? Because, if
convincing digital representations of humans can be created at whim, the entire
business of collectively assessing other people’s claims, experiences and actions is put at risk – not to mention essential social infrastructure such as contracts, obligations and consequences.

Hence the need for legal prohibitions, a case he made at length in a May 2023
article for The Atlantic. “It won’t be perfect,” he told me, “but it will help if we can make it against the law to make counterfeit people. We can have stiff
penalties for counterfeiting people, same as we do for counterfeit money… we should make it a mark of shame, not pride, when you make your AI more human.”

in principle, there is nothing preventing the algorithms of artificial intelligence from approaching or exceeding our own capacities; or from humans augmenting and re-engineering their minds through artificial means. Indeed, some of Dennett’s most important early work entailed defending computation’s power and potentials against those who, like the philosopher John Searle, claimed that mere calculation could never give rise to phenomena like consciousness. For Dennett, there was nothing “mere” about calculation or algorithmic processes: it was only ever a question of scale and complexity.

In this sense, the achievements of modern AIs – from their linguistic prowess and mastery of games like chess and Go to their ability to pass legal and medical examinations – are an ongoing vindication of Dennett’s insistence that human-level competence can arise from wholly uncomprehending processes (not to mention that, in our case, it did).

It’s dangerous to obsess over whether AI will achieve “general intelligence”, with all the cognitive flexibility of a human being, let alone something greater. Long before anything like this happens, he noted,

we will need to deal with the emergence of “extremely manipulative” autonomous
agents – and these will pose a far greater threat than hypothetical super intelligences (“forget about that!”).
Why? Because, much as social media has proved an evolutionary hothouse for content able to exploit human vulnerabilities, the same dynamics favor both AI-generated content and AIs able to deploy an enticing combination of
persuasion, seduction, shock and flattery.

From flawlessly glamorous artificial influencers to deepfake pornography, from
endlessly empathetic companions to romantic scams, human loves and longings are a fertile field for the refinement of manipulation. We may not (yet) be brains
in vats. But what we see, believe, belong to and do is increasingly interwoven
with countless information systems;

and many of these are more adept at delivering persuasion and plausibility than truth.

AIs are likely to “evolve to get themselves reproduced. And the ones that
reproduce the best will be the ones that are the cleverest manipulators of us
human interlocutors. The boring ones we will cast aside, and the ones that hold our attention we will spread.

All this will happen without any intention at all. It will be natural
selection of software.”

In evolutionary terms, our minds aren’t devices fine-tuned for differentiating
truth from lies. We are partial, passionate, tribal creatures: social animals linked by bonds of love and loyalty that both define our humanity and make us painfully vulnerable.

“is your thinking to be determined by the truth about what’s out there. You want to be compelled by the good evidence there is out there for how the world is. But you also want to have the elbow room to reconsider, and reconsider, and
reconsider further: your prospects, your projects, your goals. You want to be a
higher order, intentional system that reflects upon means and ends and goals.”

The “freedom” to act on the basis of manipulatively inaccurate information is
no freedom at all. By contrast, actions determined by “the good evidence that is out there” are emancipatory: open to the complexities of actuality rather than snared by untruths.”