Schneier on Security

Mike (just plain Mike) • April 13, 2015 4:32 PM

@tyr

Well – again I think that view is kind of parochial – “we’re so horrible – we just damage and destroy ourselves and everything else – there must be something wrong”.

The dull fact is that – more than likely anyway – humans and all the things they’ve made are just as natural (and fu*ked up) a product of the evolutionary process as any other species Nature’s picaresque train-wreck has so far brought forth – so Pop-Tarts and nuclear subs and microprocessors and ‘pollution’ are just as natural a product of the evolutionary process as anthills (as I understand it social ants now account for approx 1/3 of animal biomass in many equatorial forest regions – so presumably quite a few things had to go under a bus to make room for them when they first appeared) – and there was the oxygen-ification of the atmosphere – that was a long time ago – natural process resulting from new forms of life – big ecological change there! I’m not saying such things are necessarily a good thing – or ‘OK’ in any general sense – but they happen – and they are ‘natural’ according to the usual meaning.

I think this idea that humans are somehow uniquely damaged – uniquely destructive and uniquely able to do things that are ‘un-natural’ – against Mother Nature’s otherwise perfectly beautiful balanced plans – is actually a bit of an off-shoot of the religious ideas of the supremacy or otherwise special-ness of man. For sure there are special, unique, interesting new things about humans – and their potential global environmental impact is huge – but really – if you were to parachute a load of rabbits onto Lettuce Island then they’d have a population explosion – they’d rapidly eat all the lettuces – and then their population would crash and likely Lettuce Island would no longer have a rabbit population at all (and it would have to be renamed too probably as there would also likely be no more lettuces either – maybe something like Baron-Of-All-Vegetation Island). This sort of thing would apply to most species if you dropped them into that sort of situation in which they weren’t already in a reasonable equilibrium with everything else – but humans are actually one of the few examples where they at least could stand a chance of avoiding that sort of thing – “Hey guys – maybe we shouldn’t eat all the lettuces at once?” – rabbits just don’t have those sorts of conversations. I’m not saying humans aren’t a bit special – but I think actually it may be better to think about their special-ness in an optimistic rather then pessimistic way – and that way of looking at it may actually be, in some senses, more objectively accurate.

The problem with this way of thinking is that it means something can be done – so throwing one’s hands up in melodramatic despair and going on and on about how hopeless and selfish and self-destructive and doomed and specially fu*ked up humans are – and thus not even trying – is less defensible.

Mike (just plain Mike) • April 14, 2015 4:01 AM

@tyr

Wright may be a scientist, but what he’s talking about in your reference, though expressed in suitably technical language, is not science – it is informed speculation, and we can all do that.

Understanding of the brain is pretty pathetic right now. I am personally acquainted with people researching into invertebrates, and they can’t even understand how the very small number of neurons that are involved in the movement of a fly’s leg work. There’s lots of people (right across the spectrum from fascistic lie-detector salesmen to re-constructed hippies like Wright) who have a vested interest in selling their shtick by using a lot of technical language saying things like ‘the human brain works like this…’ or using speculative toy-models like the ‘left-brain-right-brain’ fairy tale – but at the fundamental neuron/wiring level it’s all still very much at the stamp-collecting stage, and at the higher level it’s all still mostly border-line phrenology (fMRI) and speculation built around behavioral and perceptual studies – some of which is good stuff (lots and lots of self-serving politically-biased-one-way-or-the-other dross though) – but all ‘mechanistic’ explanations at the higher level can only ever be speculative/toy models at the moment – and this is acknowledged by the people with integrity doing decent research in these areas.

Humans are what they are – I know that is fatuous – but the idea that there is a way they ‘should’ be, some sort of ‘naturall-ness’ from which they have deviated, some perfect state from which they have somehow strayed is, at least in the context of existing evolutionary-theory/biology not really meaningful. It’s hardly unprecedented for species to do things that are self-destructive, or for individuals to be aggressive, or for ‘mal-adaptation’ to exist – the whole idea of adaptation is a sliding scale; drop a load of continental species onto a pristine volcanic island and you’ll likely get something new after a while – they’ll radiate – get into niches – by doing so they will create further niches – after a while it will all look lovely and equilibrated, but when they first arrived were they ‘adapted’ or ‘meant-for’ that new environment? The idea that Nature is perfect – that it’s products are normally all lovely and beautiful – is evidently and plainly wrong – Nature is nasty, brutish, short-sighted, intensely pragmatic, and at root the machinery of self-replication/persistence is fundamentally selfish. One of the few places you will find lovely-hippy-fluff type behaviors between non-related individuals is in highly socially organised critters like humans. Everything else, at least from an anthropocentric point of view, is scarily heartlessly mercenary in its attitude to those that do not share at least some of their genes.

Also – I really do think we need to knock on the head this idea that early humans were in some way all lovely and in-tune with their environment. My understanding of the available evidence (you know – the science thing) is that where ever you look around the world the arrival of early humans coincides with the disappearance of large swathes of mega-fauna (there you go – authoritative sounding technical language – I can do that too!) It’s all down to that clever force-multiplication/tool-making Clive is on about, and all the co-operative organisation between un-related individuals that humans can do. Of course – after a while – you end up with groups of humans looking like they’re living in lovely Avatar-like fluff-hippy harmony with their environment, but that’s only because you’re not seeing the groups that trashed everything and wiped themselves out because they couldn’t get into equilibrium (survivor-ship bias), and the ones you do see that are persisting have already wiped out all the delicious easy-to-catch stuff, so you just don’t see those critters any more and don’t clock the preceding orgy of xenoside. Discovering new tools, trashing the environment, and wiping things out is not a new thing for humanity.

There is so much talk in your reference of what may have been ‘meant to be’. That, to me, is actually the sort of naughty ‘left-brain thinking’ they are otherwise mostly decrying. ‘meant to be’ according to who?

Mike (just plain Mike) • April 14, 2015 12:04 PM

@Nick P – please don’t feel obliged to reply on my account – my rambly rants tend to be a bit un-focussed anyway – and I’m not exactly on-topic. You need to attend to your emergency.

Mike (just plain Mike) • April 14, 2015 2:53 PM

@albert

Agreed that the adjectives I used to describe the Utterly Indifferent Mother are drawn from an anthropocentric view/value-system – but I guess I’m trying to take on what I see as an overly parochial (and negative) view of humans of themselves – so without being overly metta, I suppose I’m trying to make judgments even handedly; if people think people are selfish and un-caring then they should also grant that Nature is selfish and uncaring too – and if anything Nature and most of her critters are rather more selfish and un-caring in general than people are (or at least can be) in particular.

On human self-destruction and hubris: I think it’s actually a kind of metta-hubris for humans to think that they are even likely to perish at their own hand – another reflection of their idea of themselves as rather special. It’s almost as if they think that surely the only thing of sufficient import that could take-them-out would be their own folly. However, I think it’s actually rather more likely that humans will succumb to any one of the many-and-varied tried-and-tested routes to extinction followed by (almost) countless species before them. Not hugely easy to wipe humans out though – I’ll grant that – they’re very widely distributed – able to eat almost anything (even with a minimum of culture/technology) – though they can’t easily mitigate some of the more non-negotiable vulnerabilities of being mega-fauna.

I lack the necessary gravitas to put this as eloquently as I’d like, but if I were humanity I’d shut the fu*k up with the whining/self-loathing and start investing more in rockets and telescopes – all that spacey stuff – because I think humanity, if it is to truly advance, also needs to be less self-obsessed and self-regarding and should really get out a bit more. Soon, preferably – because The Universe is a dangerous place, particularly if you’re stuck in one place doing one kind of thing for too long (there’s monstrous perturbations lined up for everyone and everything sooner or later in the Utterly Indifferent Mother’s scarily long tail) and I suspect humanities’ rather grandiose idea of a demise at its own hand is likely the least of its worries.

Mike (just plain Mike) • April 14, 2015 3:30 PM

@tyr

Sure – the nervous system is loosely coupled task specific modules – I can accept that, as far as it goes. Many in the field do – but accepting that as a general framework of understanding is not quite the same as saying we understand how the brain works.

Yes – loosely coupled task specific modules – that sort of setup is what you’d expect FFS if these things evolved via evolution’s expedient hacks-upon-hacks. I think it’s only the noobs who think nervous systems must be like computers or centrally organised human hierarchies with a little man at the top looking at the screen deciding what to do. No – of course not. Perception into Action – and back again – a continuum.

Ramachandran is so cool isn’t he – I was very influenced by his Reith Lectures: Consciousness will likely be a bag of tricks – not one special secret magic source/spark/algorithm – just like the basic machinery of life has proved to be a bag of tricks – all enzymes and nucleic acids and lipids and ribosomes. Yes – cool – no problem with that – though again I think the more maths/CS/classic-AI types who mostly don’t think about the evolutionary angles are still hoping to find the secret sauce for making consciousness (well, they were – they’re all mostly doing Big Data now I think) – and yes – I can make a machine that does things like the leg of a fly using my current ideas about how neurons and nerves work (and I can make Big Dog that’s a bit like a Dog in some very narrow aspects too) – but at least at the moment apparently we still can’t figure out how the actual fly neurons and their actual wiring make the actual fly leg do what it does – but we will likely get there – but maybe our difficulty means we’re still missing something (which would actually be even more interesting if it is the case). It’s all very interesting – all very exciting – but there’s still a very long way to go – and I think the general public perception of how well things are currently understood is mostly wrong. Things are still not well understood – and certainly not even getting close to being well enough understood for people on The Left or The Right to be justified in spouting on about social policy based on neuroscience.

As for nature red in tooth and claw. Sure – there is some huggy-fluff too – here and there in amongst all the blood – I mean – we’re all adults here – we can cope with things not having to be exclusively one thing or the other can’t we? However, in the absence of compelling evidence to the contrary:

The World rolls round for ever
Like a mill
It grinds out death and life
And good and ill
It has no purpose
Heart or mind
Or will

😉

Mike (just plain Mike) • April 19, 2015 12:12 PM

Hello Nick

Sorry – this has ended up a bit of a monster.

I feel your pain with regard to the general attitude of most people to data security. You can tell what we have at the moment for dealing with this is bad because we still use the word technology to describe it.

Douglass Adams: Technology is a word that describes something that doesn’t work yet.

It was very interesting to read that article of Bruce’s – thanks for linking it. I think I vaguely remember it from back then (I’ve been on the train, with gaps, since the Cryptogram days). It was also interesting re-reading it as I think my position on some of the issues discussed has shifted a bit.

Yes – Nature loves heuristics – they’re cheap to implement. There’s the classic (from Dawkins’ River Out Of Eden I think) about the deaf turkey that would repeatedly murderously attack its own chicks after they hatched – it appears there is a simple turkey-rule: ‘mercilessly attack anything in the nest that moves if it doesn’t make a noise like a baby turkey.’

I’m sure humans use heuristics too. Indeed, I’m now not even sure many people think there is that clear a dividing line between a sufficiently large bag of heuristics and ‘objective rational thought’ (a lot of current AI people seem a bit coy about that at the moment.) I think my view is more that objective rational thought is perhaps something a human stands much more chance of achieving if it is supported by a mass of accumulated culture, knowledge, history and wisdom of the humans that have gone before it, all combined with active adversarial discussion with the living humans that surround it. Perhaps what we call ‘objective rational thought’ is an emergent property of a culture made of lots of monkey brains all jabbering at each other, but the monkey brains themselves are basically just running a pile of heuristics; the monkeys think they are capable of objective rational thought by themselves, but in-fact they are no more capable of this than a gate in a PC’s CPU is ‘capable’ of running COD.

Odd, isn’t it, how much time and energy humans sometimes spend discussing things on the internet xkcd://386. All this typing is not very conducive to increasing individual selective fitness is it? Shouldn’t I be using my time foraging for berries or trying to seduce women of child-bearing age? Would you say my inclination to rant on the internet is just some sub-optimal side-effect of our industrialised culture, or might it not be more of a reflection of the profound sociality of humans? Maybe things like interaction with other humans – exchange of knowledge and beliefs, ideas – gossip – arguing – status – reputation – fairness – trust – might these things not actually be really important to humans – so important that they will expend lots of time and energy – forgo food, comfort – life even. Could these things be innate – built in by evolution – the necessary hard coded machinery to facilitate the advanced level of human sociality?

I would respectfully suggest that anyone who complains about humans being bad at cost-benefit analysis or risk evaluation should be careful what they wish for. The judgement that these behaviours are ‘sub optimal’ or ‘cognitively biased’ is almost always framed in terms of their utility with regard to short-term personal/individual advantage. Perhaps a lot of this sub-optimality for the individual in the short term is about optimality in the longer term for society. One word we have for characterising someone who efficiently optimises their short-term personal gain is ‘sociopathic’. Almost by definition you can’t have a society in which everyone is constantly optimising their short-term personal gain – not even a police state. You wouldn’t expect social animals always to maximise their short term individual benefits – so although some human heuristics may indeed be ill-adapted it may well be that a lot of them make more sense once you admit wider criteria than ‘does it maximise immediate personal gain’.

Humans sometimes lay down their lives for other non-related humans. Humans spend hours and hours curating and maintaining shrines to themselves on FaceBook. Humans will often endure serious material costs in order to ‘get back’ at others who have wronged them, or to meet out retribution to those who have wronged others or are acting ‘unfairly’. These aren’t trivial phenomena on top of an otherwise individualistic species – I would argue that these social/societal behaviours are central in humans – and yet, the framing of much of the interpretation of the behavioural research (of the kind quoted in Bruce’s piece, at least at that time anyway) is along the lines of: (a) the built in stuff must surely be adapted to a lone or small-genetically-related-group animal often having to run-away from lions (sometimes tigers) and (b) some of the stuff people do doesn’t make sense if viewed in terms of maximising short-term personal gain – so this must be because of the un-natural environment of a modern industrialised world.

Such a framing, for instance, takes little account of how the behaviour choice of the subject might be perceived/judged by others – including the experimenter. People worry big time about looking stupid, mean, selfish, irresponsible or inconsistent. Also, the actual scenario is of course seldom the stated scenario – the actual scenario is usually a college kid who’s consented to take part in some research answering some researcher’s questions – it’s not actually ‘would you kill 100 people to save 1000?’ – it’s someone with a clipboard asking you that question in some office/lab because you had some free time on your hands – the only likely material consequences of your answers will be how they reflect on your character in the opinion of the researcher or anyone else who gets to see what you said (and how quickly you can get it all over with and go home). Also, I suspect that people will often take account of how likely a given scenario like ‘or you get $100 in two years’ would actually be to happen given that the real world runs on people rather than rules.

The idea that humans, if only they were perfectly adapted to their current circumstances, should be perfectly rational – that they would always competently pursue individual utility maximisation – that is insane as a model for humans – it flies in the face of the overwhelmingly obvious fact that humans are highly social. That such a view could have ever been taken seriously – and that it still often is – seems to me very parochial. When humans look at social insects – ants say – they say things like: ‘yes, the reason that ant is laying down its life for the colony is because of this or that pheromone’. Imagine an exo-exobilogist looking at humans: ‘yes, the reason that human is laying down its life is because it is being controlled by the sound waves emitted by the other humans’ – the exo-exobiologist sees humans projecting sound waves at each other and exchanging numerical tokens – organising themselves – specialising – deferring rewards and doing things that aren’t to their immediate individual advantage (an extreme example being knowingly risking death in war). The humans, of course, think of all of this in terms of concepts like language, money, duty, patriotism, honour, respect, justice etc. The exo-exobiologist just sees lips shaping sound waves and the resultant behaviours – and any competent biologist knows that this is the sort of thing you would expect to see in social animals: lots of interaction and information exchange, lots of machinery for context dependent control/modulation/coordination of behavior between individuals.

Also – it is important to be disciplined with evolutionary arguments. Evolution does not select for the short term gain of the individual. Evolution doesn’t give a sh*t one way or the other about the individual. Generally Nature is not trying to make individuals happier or their lives longer or better (ask a salmon) – it is just doing what it does – which is physics at root – the unavoidable realities of the replication game played out in our physically constrained universe. Darwin’s great insight – evolution by natural selection – is an essentially simple idea, but it has many complex and counter-intuitive consequences when you follow it through (like chess perhaps – or Go) – but its core simplicity leads many people to think they understand it and its implications better than they actually do. The generally accepted rule – which follows from the basic mechanism of how life reproduces – is that if you can’t explain why a given thing (trait, behaviour etc.) might lead to the preferential persistence of the gene or genes which encode for it then you have a serious problem in invoking evolutionary theory as an explanation/support for why that given trait/behaviour exists. For sure – things which benefit genes sometimes also benefit individuals, groups or species – and sometimes they don’t – but the gene-centric view is the first point of call for most evolutionary biologists – not because of reactionary dogma – but because it is based on current understanding of the machinery of life, the mechanisms of heredity and the physical constraints imposed on replicating things in a finite system subject to the vicissitudes of thermodynamics (there is finite free energy coming in, there is finite stuff to build bodies with – not everyone can persist and reproduce without limit).

This is the really exciting thing though. There are still fascinating and open problems and arguments (many of them decades old and still extremely hand-baggy among biologists) as to how to account for the evolutionarily stable development and maintenance of sociality. This is particularly interesting in the case of humans because they show such high levels of co-operation between un-related individuals. Many of the arguments are rather counter-intuitive – and – just as in the field of security – the ‘first thing you think of’ often turns out to be wrong. I think greater understanding of how humans got to be as amazingly social as they are – and how they manage to stay that way – is likely to have a big impact.

The scary reality might be that societies – as so far evolved at least – may function at their best (best serve the genes that regulate them) when the people in these societies are not functioning at their most individually rational/well-informed. This is a depressing prospect – and I don’t like the idea – but, of course, Nature doesn’t care about that! The risk/cost/benefit of a given activity for society may not be the same as the risk/cost/benefit to the individuals undertaking that activity (heroics leading to certain death being a fairly good example).

Perhaps some of the ‘biases’ in humans are to do with getting groups of people to agree about what to do – even if what they agree on is not necessarily optimal/rational – it may be ‘good enough’ – and that may be better than the alternative if the alternative is that the people would not be able to agree (or would take too long to agree) over the truth/optimal-solution. Alternatively, maybe some of the heuristics are to do with how you build a society in which some people can specialise in finding the truth while the others will just take their word for it – or maybe it’s all to do with machinery for iterative societal problem solving – so perhaps initial over-reaction to rare but unusual/scary events might in some way be mirroring similar phenomena that seem to happen in immune and nervous systems before they converge over time on a more appropriate response.

To try to get back on (your) topic:

(1) People often seem over willing to give out passwords. If I’m setting up a machine/device for someone I’ll say ‘could you just type in your password at this point – I’ll avert my eyes’ – and they’ll usually say something like ‘oh – it’s OK – I trust you – it’s blah-blah-blah’ Maybe they’re just being stupid and/or making poor judgments because they are living in a city rather than collecting berries and frequently having to run away from tigers… or maybe part of what is happening is that people are programmed to signal and build trust with each other, and to foster a reputation for themselves as trustworthy – and this can be achieved, at some risk, by making a trusting opening move. Saying ‘no – I won’t tell you my password’ is not that far away from saying ‘I don’t trust you’. Problem being (as I’m sure you’d agree) that signalling a window of trust by temporary grating someone control of your in-need-of-repair car (and thus giving them the key) is reversible in a way that telling someone your password is not (if you aren’t capable of changing it that is, which almost by definition in this case such people are not because that’s why they needed me to find the bit of UI on their device where you need to type the password in the first place).

(2) Personal computing devices (for want of a better phrase) have been around for – what – 30, 35 years. In the early days they were very marginal – just home computers – no global network. If I plot a graph of ‘how bad will it really be if everything on my computing device(s) were destroyed/stolen/exposed-to-the-world’ against time then I will see that as the decades have passed that line has been going up pretty rapidly – even more so for corporates (though they started with IT quite a bit before individuals). I think it is taking society a while to adjust to this rapid change, and it is not helping that there are powerful commercial interests trying to persuade everyone to spatter far and wide anything and everything they have ever done/said/written/searched/thought/felt/photographed/eaten or otherwise used their computing devices to record – share, and enjoy (and Google/FaceBook etc. are probably pushing at an open door with that). Anyhow – I think that having your house broken into or your car stolen is still, for most people, many times worse than a hacker trashing or raiding one of their personal computing devices. And yet – as you imply – the physical security solutions (physical locks and keys) we have for houses and cars are much better developed and less effort to use than the security solutions we have at the moment for computing devices and remote data. It’s the wrong way round – the things we seem apparently to care least about (our computers/data) are the hardest work to secure. I think it’s beginning to dawn that having your bank-account drained is something that can happen to people – but the finance industry is still managing to keep a lid on how bad that actually is. The stories people tell me are still mostly of a friend of a friend who had their details stolen and the evil hackers took some money, and it was all very stressful, but the bank/card-company kissed it all better and refunded them, and now it’s OK.

(3) As for the corporates: As a human, the attitude/opinion of your peers is very important. Not only is it important to take into account the official rules (the rules of the leaders and the police-types), it is also important to take into account the general attitude of your non-governing peers with regard to how good/bad/serious/stupid those rules actually are (I remember school). So, in the corporate context I think it’s still true (and I guess you must know far more about this than me) that people still tend to show far less care in following corporate IT security guidelines than they would with regard to, say, setting the physical alarm system when leaving the building. I think part of that is a carry though from general societal attitudes to personal IT security ((2) above) – a cultural sense that basically everyone is a bit crap at it, and that computers and their contents are still somehow not as serious or important compared to real world physical stuff. So, if something bad happens at my workplace and it was my fault for not changing my password as often as I should, then all my peers will look at me and think ‘there but for the grace of god go I’ rather than ‘oh how dreadfully irresponsible of him – what a bad person – I’ll never give him the benefit of the doubt or do him a favour ever again’.

So – (finaly!) – I think issues of privacy and thus personal data security are most likely to require an acknowledgement of the social nature of humans when trying to interpret how/why people make the judgements/choose-the-pay-offs that they do. If you’re not a social animal then privacy doesn’t really have that much meaning – so I think interpreting behavioural studies in terms of the short term fitness benefits for an individualistic hunter-gatherer who spends a lot of time running away from lions isn’t likely to be helpful, and I’m not surprised that the insights gained into people’s ‘security/privacy accounting’ from analyses based on this perspective don’t help much. I’ve not read Liars and Outliers (published 2012 – so that’s four years I guess after the article you referenced) but I think this sort of societal and more enlightened evolutionary perspective is part of where Bruce was trying to take people with that book.