Friday Squid Blogging: Squid Populations Are Exploding

New research:

"Global proliferation of cephalopods"

Summary: Human activities have substantially changed the world's oceans in recent decades, altering marine food webs, habitats and biogeochemical processes. Cephalopods (squid, cuttlefish and octopuses) have a unique set of biological traits, including rapid growth, short lifespans and strong life-history plasticity, allowing them to adapt quickly to changing environmental conditions. There has been growing speculation that cephalopod populations are proliferating in response to a changing environment, a perception fuelled by increasing trends in cephalopod fisheries catch. To investigate long-term trends in cephalopod abundance, we assembled global time-series of cephalopod catch rates (catch per unit of fishing or sampling effort). We show that cephalopod populations have increased over the last six decades, a result that was remarkably consistent across a highly diverse set of cephalopod taxa. Positive trends were also evident for both fisheries-dependent and fisheries-independent time-series, suggesting that trends are not solely due to factors associated with developing fisheries. Our results suggest that large-scale, directional processes, common to a range of coastal and oceanic environments, are responsible. This study presents the first evidence that cephalopod populations have increased globally, indicating that these ecologically and commercially important invertebrates may have benefited from a changing ocean environment.

As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Read my blog posting guidelines here.

Posted on December 29, 2017 at 4:23 PM • 124 Comments

Comments

China USA CompetitionDecember 29, 2017 5:16 PM

China: Social Media Account Becomes Official ID

‘The WeChat ID programme was co-developed by the research institute of the Ministry of Public Security and Tencent’s WeChat team, and supported by various banks and several other government departments.

The project is expected to help deter online identity theft, as facial recognition technology is used to verify applicants before their virtual ID cards get authorised.
Those verified will be able to use their WeChat ID to register in hotels and apply for government services without the need of bringing their physical ID cards.’


No wonder Mr Zuckerburg’s 50 state swing as he positions Silicon Valley’s ruling elite for the 2020 elections. Creating an official USA government social media account must be a top priority. Imagine a world with advertisers gaining discrete access into your government records…

Little wonder republicans are furiously making up lost ground by ramrodding FCC data-mining rules and targeted tax increases. .

This huge internal conflict is actually a war between Republican Corporate vs Democrat Silicon Valley.
The opening 2017 battles:
1) allow choke-point/captive audience ISP data-mining
2) FCC approved ATSC 3.0 TV Broadcast Standard with personalized Internet return-channel
3) FCC approved net neutrality allows boardroom political deals to legally discriminate against opposition

The Ultimate Cost of Privacy
The root cause of this immense nation-changing power struggle are the legions of programmable, preoccupied heads-down Americans. These citizens willingly give away privacy to be monetized, manipulated and controlled by ruling class Big-Data corporations.
The winner’s prize is unprecedented in American history; eavesdropping with compulsory control over citizens lives. Expect enactment of a Social Credit System as an important means to regulate the economy and as a tool of governance to steer the behavior of citizens. No more crypto security measures guys. All driven by your Big-Data. Good Show Americans!

JG4December 29, 2017 5:21 PM


Thanks to all of the usual suspects and others for the continued excellent discussion. The quantum article below didn't do much for me. I commented before that the United States Constitution, purchased in the blood of our fore-bears, could be read as an operating system that seeks to divide power.

The last article linked below makes explicit a principle that the constitutional system of government should be robust against corruption. I've known several people, myself included, who found out that their military cook was selling the food out the back door and pocketing the cash. Harry Truman made his career on rooting out corruption in road-building, then in defense contracting. There should be some analogs in operating systems, computing hardware and in services, like the Five Horsemen of the Tech Apocalypse.

https://www.nakedcapitalism.com/2017/12/links-122917.html
...
Big Brother is Watching You Watch

Dirt Boxes: The Newest Government Tool for Warrantless Privacy Invasion The American Conservative

Imperial Collapse Watch

“Fat Leonard” Scandal Fallout Will Damage the Navy for Years US Naval Institute (Re Silc).

...

Quantum mysteries dissolve if possibilities are realities Science Magazine (ElViejito).

The Predator State James Galbraith, Catalyst

“The Anti-Corruption Principle” (PDF) Zephyr Teachout, Cornell Law Review (via). “While political virtue is pursuing the public good in public life, political corruption is using public life for private gain. Long, but a must-skim, at least.

...

Mike BarnoDecember 29, 2017 6:28 PM

@ Anthony Alfedi :

Bring on the squid explosion.

Sir, be careful what you wish for. Perhaps squid are on the cusp of evolving the best "psychic" collaboration ever known, and when there are enough of them to reach their critical mass, it will be Squid, not Humans nor our Computers, who become the Singularity.

Then trillions of squid will all decide you aren't eating any of them.

CynthiaDecember 29, 2017 11:50 PM

More cephalopod means fewer fish, crustaceans and other marine life these predators relish.

oh reallyDecember 30, 2017 12:50 AM

The oceans are dying, continents of plastic trash. Ice shelves are just about to slough off entirely.

So right now, they want to drill in the arctic preserve that even big oil says is just way too risky.

They want to turn Alaskan rivers that all freshwater spawning salmon use into mining tails.

They want to clear cut areas of national forest under the guise of otherwise needed fire prevention.

They want to strip-top the mountains and fill in the valleys. Rivers are for effluent transportation.

They want to lower drinking water standards and air quality standards, and have.

They want to remove protections for critically endangered species going extinct as we speak.

They want to gut public schools in favor of for-profit religious school voucher programs.

They want the poorest to pay more for health care, financial services, legal help.

They protect banks and major corporations from paying the rates of taxes you or I pay on income.

They've given 79% of 1.5 Trillion dollars of your sons and daughter's money to the ultra rich 50.

They've taken promises made to children born in America and stomped on them.

They've taken promises made to poor desperate people around the world and stomped on them.

All for politics? No.

No, I refuse to believe these people are not willfully evil, in the final analysis.

I really don't support Hillary, never did. Never would or could. But this?

Is madness.

A5/10December 30, 2017 1:27 AM

Much progress has already been made towards solving organized crime and terrorism. Briefly:

1. Silver and gold were replaced with currency designed to enable the Intelligence Community (IC) to more easily track criminals and dissident types by means of a unique identifier on each bill/note of debt.

2. When the Internet ushered in the so-called "Golden Age of Surveillance," IT started incentivizing debit cards, credit cards and loyalty cards, since its more convenient to simply allow anyone with a high school diploma to have instant, unfettered (technically some checks and balances were thrown in to pacify those extreme Libertarians who still insist on honoring the 4th Amendment, but there have always been enough loopholes to avoid annoying processes such as the procurement of warrants) access to perfect records of every transaction that anyone has ever made. This has made it far easier to mop up terrorist sympathizers, protesters, and other undesirables than it ever was through good ol' fashioned detective work. And so, the world became a safer place.

3. To overcome a limitation of the cards (those evil dissidents could leave their phones behind to at least be anonymous in between transactions), phone payments are slated to replace cards. By blocking payment apps from running on phones with operating systems that have less tracking,such as Copperhead and Replicant,subversive behavior is reduced.

4. Watches are next, starting with the Apples Watch Series 3 with GPS and cellular, since people are far less likely to leave their watches behind their phones. But some radicals will, even once all watches are required to have this security feature, which brings us to....

5. For purposes of national security, everyone should be required to receive a free, safe, painless microchip with integrated payment chip, GPS and 3G mobile communications technology.This will also enable IC to responsible parents to ensure their children can't simply give their smartwatches to their friends at school to trick their parents into thinking that they are safe at school when in reality they're getting drugs and having sex while listening to rock 'n roll. Think of the children and ask your lawmakers to mandate these safety-chips and to prevent anyone from being able to buy or sell without them (who besides terrorists wouldn't want to show their patriotism by getting these chips anyway?).

Since most people take better care of the right hand and forehead than the left hand, everyone should be required to get one in the right hand or forehead. There should be an unforgettable holographic tattoo over the implant to allow peacekeepers to readily distinguish terrorists from patriots, at a glance. Stickers, even gold ones, are no substitute for this.

TomDecember 30, 2017 1:44 AM

@oh really
It's not really about who you support, but about which program you vote for.

Careful observers have noted that the GOP has been building a machine that discards anything in the American tradition, anything in The Constitution, that does not serve their Lords and Masters. Their exact identities are somewhat unclear, but they're the ones who own pretty much everything.

The GOP has for decades now dealt in hate, lies and the disregard of any sort of fact, including those found in history and science. They openly call for assasination or violence, their preferred term is "second amendment solution."

(Why does my iPad provide solution as a choice in that last phrase above, one I've never before written?)

All of this was entirely predictable if any GOP candidate became President. Except the Munsters-style freak-show. We shouldn't be surprised.

oh reallyDecember 30, 2017 2:30 AM

The GOP candidate became president because there are that many "gullible" (kind of me) and furthermore ANGRY people willing to buy into the soup sandwich of bullsh*t that Trump proposed because he promised violence against their political enemies.

They were bought cheaply, as was he. Massive corporate-criminal agendas do not sleep.
They don't care if southern white males get those coal jobs, or health care, water.
Trump's promises are as worthless as the university credentials that bore his name.

Why are we here?

Because Hillary Clinton is also corrupt. And we have only 2 parties to choose from.

What the hell are we doing about the 2 party system? Two unaccountables is not less evil!

Jonathan WilsonDecember 30, 2017 4:14 AM

The problem is not the two-party system, the #1 problem in US politics (at every level) is the way special interests (everyone from Comcast to Monsanto to Boeing to Walmart to the Catholic Church) have more say in how things are run than the people who elected the politicians in the first place.

Its the same in other countries like the UK and Australia (although not to the same degree as the USA)

If we get special interests (be they corporations, unions, religious groups or otherwise) out of politics and get politicians doing what's in the best interests of everyone and not what's in the best interests of the vested interests, the world will be a better place.

echoDecember 30, 2017 6:21 AM

The UK government have released a number of previously classified docuents. This release is as interesting for what it hides as much as it reveals. The media ask questions often raised on this blog about the legitimacy of the classification system and the reasons.

I am unsure about whether the rise in diabled people's suicide is an appropriate topic for this blog. However, I believe there are questions about the information and processes behind policy such as: excessive secrecy by the psychiatric profession, undocument standards leading to prejudice and abuse and treating citizens as a threat, abuse of the Freedom of Information Act by either failing to disclose or lying in court about unofficial managment directives to meet targets and covering up of hasty processing by none expert poorly trained and overworked staff, failure to consider decisions with full regard to both science and the law, and "outsource loss" while simultaneously cutting access to the law creatign a vicious cycle where political decisions are not held to account or verified or even confirming whether the policy outcome in the real world meets the policy claims alleged.

I note this statistical rise is similar to the levels among transgender people denied medically necessary healthcare. The overall pattern of beaurocratic authoritarianism, contain and control, and cost cut into none existance is very similar.

So documents about Thatcher have been declassified. What about the rest?
https://www.theguardian.com/commentisfree/2017/dec/29/national-archives-thatcher-documents-scott-inquiry

It’s either depressing or cheering, depending on whether you’re a glass half empty or a glass half full person, to see how great the relevance of these 25-year-old documents is to our situation today. Iraq, dodgy international banking and most of all, Europe, remain painfully tied into daily lives. Of course, the papers are also released or withheld under the auspices of a Conservative government that one can scarcely believe has legitimacy at all, mired as it is in incompetencies, narrow escapes and plain falsehoods.

Restricted release of UK secrets raises eyebrows
https://www.ft.com/content/0c34b992-e1ad-11e7-8f9f-de1c2175f5ce (Restricted viewing. Google search headline to click through and read.)

While some of the withheld documents have implications for national security — one is about the “physical security aspects of Downing Street” — there is no obvious reason for the non-disclosure of others.

Attempted suicides by disability benefit claimants more than double after introduction of fit-to-work assessment
http://www.independent.co.uk/news/uk/home-news/disability-benefit-claimants-attempted-suicides-fit-to-work-assessment-i-daniel-blake-job-centre-dwp-a8119286.html

Attempted suicides among out-of-work disability benefit claimants have more than doubled since the introduction of fit-to-work assessments in 2008, The Independent can reveal.

ThothDecember 30, 2017 7:14 AM

@Clive Robinson

I have decided to sit down and investigate the hype around QR Code payment modes and I have decided to take a stab at "EMV ® QR Code Specification for Payment Systems (EMV QRCPS) - Merchant-Presented Mode".

The MPM mode to put simply is the merchant prints out their permanent transaction account details in QR format as supplied to them by their payment service providers and they paste a sticker or image of their QR code and allow people to scan it with their mobile phones to make payments.

The sad thing comes when one reads the standards. Security IS NEVER ENGINEERED INTO THE ENTIRE PROTOCOL.

The message integrity checking mechanism is .... CRC32 .... yes ... CRC32. No digital signatures, no MAC codes, no crypto .. nothing.

And to make things worse, the Merchant Name (which you could name yourself) can be done in Unicode. There are numerous cases where falsification of identities can be by via using a Unicode character very close to the actual name character so that when it shows up on the consumer's mobile phone screen, it looks almost legit and the consumer usually doesn't check the Merchant IDs and all that and click the Pay button which can be routed to a false entity.

So here's my first stab at any QR code payment and my first theoretical attack on QR code payment in two steps.

1.) Register an entity with almost the same Unicode character with the payment mode set to 'Static Paymment (Mode #11)'.

2.) Find a way to swap out the QR code display and tamper with the CRC32 checksum.

Also, note that there is also a rather loose format and there is possibility of adding URLs since there are no restriction and if it appears on the consumer's mobile phone and the QR payment app does not filter it properly, it is going to be a possible way to gain access to the user's device.

All in all, the better mode to choose is Mode #12 which is the Dynamic Mode where users have to step through the payment method procedures but here's the problem ... the weakest link is the human in any security chain and humans usually click through things without checking so Mode #12 Dynamic Mode is also theoretically broken because the checksum it uses is CRC32 without any digital signatures or robust cryptographic attestation.

JG4December 30, 2017 11:28 AM


Thanks for the great ideas and links.

via Drudge. the spooks there may be as crazy as the spooks here. they are paid to think outside of the box, but you'd like your country to have some basic ethical rules

Papers reveal 'MI5 plot to kill Irish PM'
http://www.bbc.com/news/world-europe-42510529

from the usual compendium

https://www.nakedcapitalism.com/2017/12/links-123017.html
...
What Would You Pay to Keep Your Digital Footprint 100% Private? HBR
https://hbr.org/2017/12/what-would-you-pay-to-keep-your-digital-footprint-100-private

fingerspitzengefühl is another word for "sensor fusion." Rommel had it. intelligence is the ability to connect cause and effect in a way that leads to survival. observe (including all sensor inputs), orient, decide, act. easier said than programmed. trying to anticipate all of the sensor inputs that might occur on an operating vehicle isn't so different from trying to anticipate all of the attacks that might be made on your system of backdoors

Wired: Self Driving Car Hype Crashes Into Harsh Realities
https://www.nakedcapitalism.com/2017/12/124762.html
Posted on December 30, 2017 by Yves Smith
For some time, Lambert and I have been pooh-poohing the idea that self-driving cars, particularly true self-driving cars (as opposed to ones that have humans lurking in the background to take control) would be here any time soon, much the less in the widely-ballyhooed time frame of 2018 to 2020.
A new Wired article, describing the newfound sobriety in the self-driving car development community, confirms our long-standing views. ...
The big problem is that the people engineering these systems have yet to come close to mastering basic design requirements. They think they know how to get there, but that is sort of like being able to describe what it would take to sail across the Pacific solo and actually doing it.
One set of problems is that the self driving car creators have apparently settled on using three different types of sensors and then integrating the inputs. The types of sensors individually don’t appear to be able to operate at the required performance levels.
...
https://www.wired.com/story/self-driving-cars-challenges/

CallMeLateForSupperDecember 30, 2017 12:37 PM

@A5/10 (or should I address you as Fake News?)
"1. Silver and gold were replaced with currency designed to enable the Intelligence Community (IC) to more easily [yada-yada]"

Nope. Metals went bye-bye because they wear out pants pockets and are a bi... bummer to otherwise transport. It was a practical matter; IC played no part in the transition from metals to currency, which got underway decades before IC as we know it.

"2. When the Internet ushered in the so-called "Golden Age of Surveillance," IT started incentivizing debit cards, credit cards and loyalty cards [...]"

Nope. Oil companies were "pushing" credit cards to fresh h.s. graduates a-way back in the 1960s, well before the internet, IT and debit cards were things. It is retailers who actually incentivize loyalty cards; IT just slurps up data from their use and resells it. And you should point at financial institutions for dreaming up and pushing debit cards.

I don't understand the rest of your post.

oh reallyDecember 30, 2017 12:38 PM

@ Jon Wilson

2 parties is so easy to manipulate it's a human tradition since the dawn of civilization. It is the reason we have the sellout laws that allow special interests to buy their constituency. If we had 4 or 5 independent and similarly funded parties, big money would find their job much less predictable.

A diverse coalition of parties would be forced to do battle on the merits rather than just rely on the "us or them" dipole.

I think the American people are starving for an unbought candidacy. Bernie came close.
The other parties got single-digit support, but don't count that out either.
That is millions of people. They rejected the big 2 FOR A REASON.

We need to get serious about this before it's literally tastes great vs. less filling.
It's bud lite either way.

CallMeLateForSupperDecember 30, 2017 1:46 PM

From "Dirt Boxes: The Newest Government Tool for Warrantless Privacy Invasion", pointed to by JG4:

"If passed, [Building America’s Trust Act] would require unmanned drones to scour the [U.S./Mexico] border 24 hours a day, five days a week."

Alrighty then.... So, for the best experience, migrate during a weekend, Got it.

Your fingers must be tagged and watched at all timesDecember 30, 2017 7:10 PM

It's only madness if?

Comprehend.

Fathom.

TatütataDecember 30, 2017 8:37 PM

The German Computer Chaos Club held its 34th annual congress from 27 to 30 December at a new venue in Leipzig.

https://media.ccc.de/c/34c3

According to my estimation, there are as of now 164 different presentations, with more than 100 hours of video. I don't know whether all videos have been uploaded, but my count has remained stable for the last several hours.

Many presentations intersect with the themes of this blog, and are (available) in English.

Some examples (which I haven't watched yet):

Internet of Fails -- where IoT has gone wrong

Uncovering British spies’ web of sockpuppet social media personas

Der PC-Wahl-Hack -- Analyse einer Wahlsoftware (Hacking the election reporting software)

1-day exploit development for Cisco IOS

etc. etc. etc.

A few weeks ago there was a blog entry regarding the adoption of legislation allowing the implantation of spyware on computers. I expressed skepticism in my comments, as I couldn't believe that it was possible to propose and implement this in an election year. I must stand corrected, it actually happened, and the media hardly spoke about it.

I am presently watching the CCC 2017 retrospective (in German). Starting at ~45 minutes there are about 10 minutes explaining how this was sneaked through parliament.

Work went on over several years of the previous legislature for renewing the Strafprozessordnung (Code of Criminal procedure). As the text was pretty much cast and ready to vote on, cryptic low-key "linguistic" amendments were "suggested" by the ministry of justice last summer. There was very little time left for debating, and the amendments were presented as innocuous "patches" (the panel calls them "diff") with jumbled wording cross-referencing other articles. It was difficult to grasp what was meant by this these. The CCC had been invited to comment on the law by a letter dated 29 May for appearing before the parliamentary commission on 31 May,.

The (usually tame) Federal Commissioner for Data Protection even wrote to parliament to express his disagreement with the proposed changes, which he learned about in the press around 17 May.

Since the government was ready to vote on the text, and the federal elections were quickly approaching, no one rocked the boat and the garbage went into the book like a charm. The threshold for allowing intrusive measures was through these cryptic amendments considerably lowered, from "terrorism" to pretty much any crime in the book.

My contempt for politicks isn't decreasing.

Wesley ParishDecember 31, 2017 12:14 AM

Came across this via Slashdot
ht tps://tech.slashdot.org/story/17/12/30/0247231/neuro-cyber-slaughter-emerging-technological-threats-in-2017

Neuro, cyber, slaughter: Emerging technological threats in 2017
ht tps://thebulletin.org/neuro-cyber-slaughter-emerging-technological-threats-201711378

Neuroscience—and the new weapons of the mind
ht tps://thebulletin.org/neuroscience%E2%80%94and-new-weapons-mind11229

The Russians used to have a saying: "In the News (Izvestia) there is no Truth (Pravda); in the Truth (Pravda) there is no news (Izvestia)". I've been writing about this sort of thing for a wee while:

Malaita: Confession Time
ht tp://pandora.nla.gov.au/pan/10063/20140508-0000/www.antisf.com.au/the-stories/malaita-confession-time.html

Malaita: In This Context
ht tp://pandora.nla.gov.au/pan/10063/20151004-0003/www.antisf.com.au/the-stories/malaita-in-this-context.html

Wesley ParishDecember 31, 2017 12:26 AM

Should also add that Phillip K. Dick was writing a lot of stuff during the 70s that directly relates to this above topic, of direct neuropharmacological manipulation (use of drugs to manipulate people). The VALIS "trilogy" is one of those; A Scanner Darkly is perhaps the best, though Flow My Tears, The Policeman Said is also very good.

AlejandroDecember 31, 2017 7:54 AM

I know most of the visitors here run something like Ubuntu or Debian, but for those stuck with Windows 10 Pro or Enterprise I stumbled on a good hack to cut down a tremendous amount of MS phoning home attempts. I was specifically looking for something that would address the daily thousands of attempts to contact "login.live.com".

This works but also seems to reduce other creepy and intrusive contact attempts:


--------------------------------------

Choose "Run" from the Windows menuu.

Type, "gpedit.msc" then OK.

Local Group Policy Editor will appear

Left Click:

Computer Configuration/Security Settings/Local Policies/Security Options

In the right panel, double-click "Accounts: Block Microsoft Accounts"

Choose down arrow

Left Click, "Users can't add or log on with Microsoft Accounts"

Click OK, restart.

---------------------------------------

I assume this breaks windows store or something, but it can be easily reversed if necessary.

JG4December 31, 2017 9:19 AM


we can guess why they don't just block them in country. it should make them vulnerable to a lawsuit in a friendly jurisdiction.

@Wes - Thanks for the link to your writing. Nicely done.

https://www.nakedcapitalism.com/2017/12/links-123117.html
...
Big Brother IS Watching You Watch

Facebook Says it is Deleting Accounts at the Direction of the U.S. and Israeli Governments Intercept (martha r). Glenn Greenwald.

Airport Face Scanning Skates on Thin Legal Ice—and Doesn’t Work Too Well MIT Technology Review

Lawyers for Standing Rock’s Red Fawn File Final Motion for Discovery Evidence Truth Against the Machine (martha r)

...

echoDecember 31, 2017 2:22 PM

@moderator

The use of '@echo' in the name field scans like I am posting them. Will the person doing this please use this blogs features correctly?

p@ssiveDecember 31, 2017 2:53 PM

Why don't you just ask them to echo? Pick a less common-word name if you're really concerned.

TatütataDecember 31, 2017 4:31 PM

The MPM mode to put simply is the merchant prints out their permanent transaction account details in QR format as supplied to them by their payment service providers and they paste a sticker or image of their QR code and allow people to scan it with their mobile phones to make payments.
The sad thing comes when one reads the standards. Security IS NEVER ENGINEERED INTO THE ENTIRE PROTOCOL.
The message integrity checking mechanism is .... CRC32 .... yes ... CRC32. No digital signatures, no MAC codes, no crypto .. nothing.

I have a bigger beef with IBAN. Whoever dreamed up this monstrosity? If they (ECB?) had merely slapped up national account identifiers (DE: Bank ID, BLZ, Kto, whereas other countries already had unified systems providing all that info in one shorter identifier) with DELIMITERS instead of a "fixed"-length system with an excessive number of padding zeros, that would have been already much simpler for people to remember. The system results in IBANs so long that paper-based transfers are hardly practical anymore, as the probability of jotting down without error a number with dozens of positions is rather low. (Maybe that was the point?)

Then why did German banks insist for so long an additional SWIFT code even for national transfers, when other countries figured out how to derive that info from the IBAN? And then, there are those merchants that insist in printing out their IBANs in one huge block without delimiters, making transcription difficult. And banks providing user interfaces splitting out the entry field in blocks of four or so, without any facility for accepting a longer string, making pasting a chore. At that point I begin to dream of a QR code, but then, there's indeed the problem of validating the authenticity of the data.

ThothDecember 31, 2017 6:55 PM

@Tatütata

I dont think IBAN would be use for Merchant ID. It might be altogether another format as the document left it open ended on Merchant Names and IDs. Because the EMVCo QR code format is so new and was introduce in Mid 2017, they are all figuring out more details.

Will find time to start picking away at the Consumer Presented Mode once I have time and not gonna be surprised there would be more problems. CPM mode is where your phone creates a QR and you let the cashier scans your QR for payment.

Also, Happy 2018 to all.

Oh snapDecember 31, 2017 6:56 PM

http://www.mcclatchydc.com/news/nation-world/national/article191857354.html

-Jailed Russian says he hacked DNC on Kremlin’s orders and can prove it

A Russian national in jail for hacking the Democratic National Committee says a data signature proves he acted on the Kremlin's orders.

Konstantin Kozlovsky, a jailed Russian who claims he hacked the Democratic National Committee, now says he can prove Russian intelligence ordered him to steal emails released during the 2016 US presidential election.

Earlier this year, Kozlovsky made headlines when his confession to hacking the DNC on Russia's orders was made public. He was arrested on a separate charge this year, as an alleged member of a hacking group that stole more than $50 million from Russian bank accounts.

In an interview with a Russian television station made public Dec. 27, Kozlovsky reported more details on what he said was an operation led by the Russian intelligence agency FSB to hack the DNC. He claims he planted a string of numbers -- his Russian passport and visa number to visit the island of St. Martin -- in a generic .dat file. The idea was to give himself a safety net in case those who directed the attack turned on him, he claims.

In other details released this week, Kozlovsky said he collaborated with the FSB to create computer viruses. These were first tested on large Russian corporations and later used on multinational businesses, according to a published McClatchy report.

CallMeLateForSupperJanuary 1, 2018 8:44 AM

Happy New Year, all. The beat goes on.
One more way in which scripts enhance our web experience.

------------------------------------------------------------------------------------

November 15, 2017 - "Exfiltration of personal data by session-replay scripts"
https://freedom-to-tinker.com/2017/11/15/no-boundaries-exfiltration-of-personal-data-by-session-replay-scripts/

"This is the first post in our “No Boundaries” series, in which we reveal how third-party scripts on websites have been extracting personal information in increasingly intrusive ways."


"[...] the list of sites with session-replay scripts, and the sites where we’ve confirmed recording by third parties. https://webtransparency.cs.princeton.edu/no_boundaries/session_replay_sites.html
The list in CSV format, ZIPped, is here:
https://webtransparency.cs.princeton.edu/no_boundaries/data/sr_site_list.csv.zip

Many familiar faces, e.g. Adobe; Adidas; BritishAirways; CapitalOne; Comcast; Comodo; Costco; Experian; Fidelity; Hewlett Packard; Intel; Kaspersky; Lenovo; LexisNexis; Redhat; symantec; T-Mobile; Walgreens;

------------------------------------------------------------------------------------
December 27, 2017 - "Web trackers exploit browser login managers"
https://freedom-to-tinker.com/2017/11/15/no-boundaries-exfiltration-of-personal-data-by-session-replay-scripts/

"In this second installment of the No Boundaries series, we show how a long-known vulnerability in browsers’ built-in password managers is abused by third-party scripts for tracking on more than a thousand sites."


RachelJanuary 1, 2018 11:25 AM

Wesley Parish

hello my Antipodean comrade.
I note from your other recent comment you are in New Zealand not Australia as I thought.
How do you experience awareness of InfoSec/CoSec in NZ? Is there a 'scene' ? Any aspects that may be progressive shoulders above the more likely contenders for progression?
I'm aware that Auckland and the rest of the country aren't really comparable either.
It's not security related so much but do the locals care one way or the other about Kim Dot Com ? I can kind of imagine, actually. I note he recently received a multimillion dollar award for being 'swatted' .

hmmJanuary 1, 2018 1:33 PM

@ Supper

That list is massive. I feel cold... wait, I'm completely naked and on camera the whole time?!

Thanks Princeton. Ignorance was bliss and you've RUINED IT.

Clive RobinsonJanuary 1, 2018 6:47 PM

@ Rachel, Wesley Parish,

I note from your other recent comment you are in New Zealand not Australia as I thought.

Can I say "me too"? :-)

I must admit I am curious about Kim Dotcom. It's very clear he is a "marked man" from not just the US Gov, but parts of the NZ Gov as well, seeing as how he has upset their political system. After all their sending in a lawyer into court --to fight for the extradition-- to not just lie[1] but grandstand about it as well for the Newspapers does not bode well for justice in NZ...

Further if what is reported is correct to carry on with the pretence that Kim commited a criminal act thus could be extradited is shody at best. Prof Lawrence Lessing showed it was not a criminal act even in the US, It would appear that the Hong Kong judiciary --where the alleged crimes were supposadly committed-- agrees as well, and a NZ court has made it clear that he can not be extradited on that either... kind of nails the lid on that asspect.

The fact that from what has been reported all the other charges stem from the supposed copyright infringment being a criminal act you would have thought showing it was not a criminal act would make it all a moot point.

As for,

I note he recently received a multimillion dollar award for being 'swatted'.

I was under the impression it was paid under a confidentiality agreement. From what I've read two other people at the birthday party have already received settlements above half a million dollars, and Kim Dotcom being the householder and most prominent would be expected to receive a commensurately higher award.

Of course it does not help the police case when they make statments about shotguns to make it sound like their behaviour was justified and that Kim Dotcom was a dangerous fugitive with a gun in easy reach. When in fact he had simply run away from "terrorising" people who had failed to properly identify themselves... Again piecing together from news snipits it appears the shotgum in question was apparently not readily to hand but in a locked gun cabinate and from what has been said Kim did not have any keys on him. Also apparently the gun belonged to a security consultant dealing with personal protection (AKA body guard).

Oh and their appears to be a major discrepency over the FBI's role in the entire event. If the NZ police are to be believed the FBI took no active part in the event... However it appears from court documents they did take an active part and ended up illegaly stealing copies of hard drives etc. Apparently the warrant had been quite deliberatly made over inclusive to enable such activity and an NZ judge was most unhappy about it. So that also adds "fruit from the poisoned vine" issues to the case as well...

Then there was the little FBI/DoJ stunt over Kim's defence paying for "expert opinion" outside of NZ they made it clear that they would seize any payment which is a prima facie case of "striping of rights". Which is why Prof Lessing did it pro bono.

If the speculation is correct then President Obama not just pushed the case forwards but instigated it "for friends" in Hollywood and Silicon Valley which is actually not realy that unlikely, bassed on his other behaviours.

Thus it can be seen that as Obama is no longer in office "the driver is absent from the throttle" and there is no "deadmans" to bring the runaway to a halt. Thus there are now careers on the line at both the FBI and DoJ which means common sense is not going to get a look in and more money and resources will be thrown wastefully into the case. Hence it's become a war of atrition with the US trying to "starve him out" of NZ.

But... There is another issue, Kim Dotcom claims to have knowledge that the DNC hack was as many suspected an inside job. What the truth of of this is unknown currently, but Kim does have "previous" on making claims that his evidence does not support (Warner Bros and allegedly faked Email).

If nothing else the whole process makes for occasional entertainment which unfortunatly also brings in his Ex-wife and Five children needlessly (as I understand it the kids are NZ citizens by birth and thus should have protection under NZ law).

[1] This was over what was apparently a so incompetently translated illegal recorded phone conversation that was so benificial to the lawyer and so prejudicial to the defence that you have to question if it was incompitence or malicious...

hmmJanuary 1, 2018 10:48 PM

They had him on copyright violation BS, that's an easy warrant.

Then they found out he had secret rooms.

You know they love to bust people with secret rooms.



Wesley ParishJanuary 2, 2018 12:21 AM

@Rachel, Clive Robinson

If there is an InfoSec/CoSec "scene" in NZ it's mostly with the FOSS people. And mostly with the German IT migrants, too. Most Kiwi IT people don't seem to care. But then they don't seem to be aware of much else besides the ubiquitous Microsoft software.

As far as Kim Dotcom goes, I'm delighted to see from your comment, Clive, that the case seems to have run into difficulties. I was less than impressed by the carte blanche "warrant" NZ police were given to raid his property - general warrants aren't supposed to be legal in New Zealand any more than they're supposed to be legal in the UK, France or anywhere else. And judging from what I remember reading the US Constitution, the US law enforcement aren't supposed to issue general warrants either.

As far as I can see, the case against Kim Dotcom falls over with that, that the US law enforcement asked the New Zealand courts for a general warrant, and the New Zealand courts gave a general warrant to the New Zealand police. I think a case could well be made that the relevant US law enforcement authorities have thus put themselves outside the protection of the law in the United States and internationally. They should surrender ASAP for their own protection. Anyone know who I should contact at Interpol?

Wesley ParishJanuary 2, 2018 12:42 AM

FWIW, the RIAA head honcho had made statements well before Kim Dotcom got arrested, well before the general public in New Zealand knew anything about him, that made it obvious that his chances of getting a fair trial in the United States were practically non-existent. Statement that "copyright infringement" was worse than terrorism, things like that.

New Zealand is supposed to have a law that it is illegal to extradite someone to an unjust trial.

Bong-Smoking Primitive Monkey-Brained SpookJanuary 2, 2018 2:01 AM

@ Wesley Parish:

Researchers Fooled a Google AI Into Thinking a Rifle Was a Helicopter

It's just learning.

Now I wait with bated breath for the AI faecal recognition software that correctly identifies President Trump as a piece of (censored).

No comments.

Clive RobinsonJanuary 2, 2018 3:35 AM

@ Wesley Parish,

New Zealand is supposed to have a law that it is illegal to extradite someone to an unjust trial.

Yes as are nearly all countries that inherited the English legal system.

However in England and Wales some idiot called Tony Blair along with his old living partner Lord Falconer blew a thousand years of jurisprudence out the window and along with that he signed a very unequal extradition treaty with the US. Which basically boils down to "They demand we hand over, we ask and we get the bird". The judiciary are noticeably fighting back but it's a slow process, and that so called "Special Relationship" means that outside the IC FiveEye cabal over democratic government, the democratic leaders and citizens are at the head of the que for the "Drop your trousers" treatment, which was why during the "Cold War" London and most of England was unjokingly called "Ash City". Because it's where the nuclear war would happen in a similar way that Korea and Vietnam got near genocide levels of chemical warfare agents from the US in their little "proxie wars". It's all part of American Exceptionalism, have a look at what level of response they have legislated for should some US MIC/Politico gets draged of to a human rights court, most would call it a primary act of war...

It's why "Dirty" US IC/LEAs/soldiers/politicos etc will never stand trial in another country even when they have gone totally rogue (see history of US involvment of Iraq where the Abu Ghraib torture and prisoner abuse scandal that was the tiny tiny sprinkling of snow atop the iceberg). If international preasure gets sufficient then they simply throw some lowest ranker under the bus in a military or similar court where they can not reasonably defend themselves and all the others who knew "look the other way" and get a promotion etc as a reward...

The thing is most other Western Governments know it won't be long before the US decides throwing second world countries under the bus, to make a power statment will soon not be enough. And that some first world nation will get the "bombed back to the stoneage treatment" in the near future from the US just to keep the rest in line... It's also why rational actor governments of second and third world nations that are in effect issolated started developing their own Nuclear, Biological and Chemical weapons and delivery systems. Because it's the MAD language of power that the US kind of respects, thus gets the country off of the current US list of "make a point" bombing targets. It's one of the reasons the EU is important and countries on the Russian boarders want to sign up. They can see the US starting another "prove who's boss" proxy war with Russia, and they want "mutual protection".

History shows that individual mutual defense pacts did not realy work in Europe hence the problems that led up to the two world wars. What people are hoping is that one "common voice" mutual pact will give the US and Russia pause for thought. However times have changed in that many in Europe now see the US not Russia as the aggressor thus primary danger, hence the regression back to "Reds Under the Bed" propaganda etc. But from my view point I see equal danger from both Russia and the US as they square off again for another round of "Mine is bigger than yours".

Clive RobinsonJanuary 2, 2018 6:29 AM

@ BS PM BS,

Now I wait with bated breath for the AI faecal recognition software that correctly identifies

Is the "bated breath" due to the excitment or holding your nose at the test subjects (we have a cople of "the great unwashed" in London and they do tend to smell of dog poop and rotting feet even when you are upwind of them).

But speaking of scanning things I think it was the third episode of Futurama "A Fish full of Dollars" where Fry goes into the Big Apple Bank and the cashier says "We don't have your retinal or rectal scans on file... Do you remember your PIN?" it still makes me whince at the thought of it[1].

The last paragraph of my reply to Danial Wood in Aug 2010 still applies though,

https://www.schneier.com/blog/archives/2010/08/skeletal_identi.html#c455853

[1] There was an opening "advert" in a later episode by a drain cleaning robot, with fast rotating hands, where his chirpy drain message ended with a "barbershop" sing of "we can also help with an impacted bowel"... If memory serves it was the episode with the "aqua-exercise pool" with a mothers to be session that Dr Zoidberg interupted in full mating mode and causes a panic which as he leaves is accompanied by a sound track of ploping noises and baby first cries as it becomes a mass birthing pool.

The reason I remember these is not because I'm an avid fan, but my then young son became one and he would watch Futurama over and over and over... Oh and for some reason the episode that first got him hooked was first played on UK terrstrial TV one Xmas morning, and his mother who is a nurse laughed like a hollow drain with tears of mirth at both scenes...

AnonJanuary 2, 2018 8:09 AM

An as yet undisclosed problem is apparently the source of such a significant security bug, at least one Linux kernel and the latest updates to the NT kernel are already implementing patches, with more on the way.

Background here: http://pythonsweetness.tumblr.com/post/169166980422/the-mysterious-case-of-the-linux-page-table

It's great that it is being addressed, but why the sudden urgency, and why implement a patch so quickly as to significantly hurt general compute performance (we are not even talking edge cases, but just general processing)?

Why is this bug so critical it is worth a possible 50%+ hit to processing time to fix? I can't see how the performance problem can be solved, as all mechanisms for performance are deliberately being trashed as part of the fix.

Clive RobinsonJanuary 2, 2018 1:33 PM

@ Anon,

An as yet undisclosed problem is apparently the source of such a significant security bug

Yup it's real and it's not software solvable, only partial mitigation is going to work, and then probably not for long based on previous research time lines...

I read this paper whilst convalecing away from home,

https://arxiv.org/pdf/1710.00551

And it opens a whole world of nightmares for those that know how to look at things even slightly hinky...

To understand it the first thing you have to get firmly fixed in your head is the computing stack model from the programmer/user accessable Instruction Set Architecture or ISA down to the transistor level and some of the basic physics below that...

This is territory few ever venture into these days and your chance of meeting someone below fourty who has worked down there during their working career is slim. It just so happens that having been around for a long time ;-) I worked on the designs of bit slice computers where the "CPU" is something you can see as individual chip parts (see 74S181[1] and AMD 2900 and later family[2])

Without going into all the details, what the programer or user actually sees is the ISA which is a long way above the registers and ALU you might get taught at school. At the register level of the stack it uses a Register Transfer Language or RTL CPU hardware specific programing language. In between is the Instruction Decode and pipeline control where the the ASM to RTL microcode exists and things like instruction look ahead, and some cache control are also found. But importantly there is a lot to the side of the ALU that rarely gets talked about and a whole load sat underneath that likewise does not get talked about like Dynamic Memory Access (DMA) etc used for I/O and other low level system related events.

One of the things that sits below most of it is memory, which is mostly Dynamic RAM[3] of some form these days. Simplistically DRAM memory bits are a FET used as a switch to multiplex a large capacitence that stores the individual bit information as a charge onto an internal data line.

By large capacitence I mean with respect to the normal gate capacitence of a similar sized FET. In practice the capacitance is directly related to the area it takes up on the chip and is thus made as small as is possible to get as many memory cells on a chip as possible. It is the size of this capacitor where all the problems start, and has been known about since the first multiplexed refreshed DRAM chip the Mostek MK4096 4 Kbit DRAM (which was engineer weekly wages priced back in the mid 1970s).

The problem with DRAM is that capacitors discharge over a period of time based on the effective resistance across them. Thus in one CR "time constant" period the charge will drop to around 2/3rds of what it is and be virtually gone by 5CR. Thus the refresh trick is required to correct this.

Essentially refresh is where you put/take charge back into/out of the capacitor. There are two basic ways you can do this, "slow and accurate" by measuring the voltage then pulling the capacitor up or down as appropriate, or the "fast and dirty" way. In practice it's a bit of both which makes ubderstanding it harder than it should be. So simplistically If the capacitor and effective resistance values are predictable then you could just dump 1/3rd of the charge into the capacitor "sight unseen" as this would top the 2/3rds up to the full value and only pull a zero up one third of the way. Thus if the "data sense amplifiers" are set to read at say 3/6ths of the full charge value then you will get the correct data reading. In practical implimentations things are rather more complex and use various tricks such as using the self capacitance of the internal data multiplex traces to store a 3/6ths comparison value via the "pre-charge" phase. Thus there is often considerable "slop" in the design which gives attackers potential wriggle room just as it does in physical security like lock picking.

It's the slop in the "pre-charge" and feedback sense amps that alows the attacker to force the charge on a memory cell capacitor that RowHamner uses. It's inherent in the design of DRAM which makes it a problem untill such time it is ever fixed. It's an issue the much faster but larger and more current hungry Statiic RAM does not have.

When I started playing with DRAM and Z80 CPU chips the MK4116 had got a lot cheaper due to the introduction of the 4164 64Kbit DRAM chip which cut PCB real estate dramatically, thus Z80 CP/M PCBs were as small as a couple of playing cards. However 8bit computers like the 6502 and 8080/Z80 had a problem, their memory was small (64Kbytes) and making more than one program run on them was a real difficulty due to having to define memory areas for each program as well as task and stack switching from a hardware interupt. In mini computers such as the DEC VAX and through to the main frames such as those from Burroughs, IBM, ICL etc this memory issue had been solved by first a software interupt and a form of segmentation then what we now call a Memory Managment Unit or MMU that gives us what we now call Virtual Memory.

Simplistically what you do is put the equivalant of a second very fast stripped down CPU between the main CPU and the physical or "core" memory. It's job is to hide the real physical memory address from the CPU so at it's simplest you could make software physical memory address independent. It does this by "address translation" by simplistically mapping a main CPU virtual address to a real physical memory address. The mapping values are held in one or more regions of core memory labled as "Page Tables", thus they have to be in a known location in memory, which are often duplicated in cache memory inside the main CPU.

It's important to note that the MMU is a Turing compleate computer in it's own right with the "Tape" being page table memory, and that as it sits between the main CPU and the core memory it can do all sorts of things that the programmer/user can not stop or even see, which makes it a nice target for certain types of hacker.

In Intel IAx86 chips because they originally put segmentation in the CPU then much later added an MMU things are way way over complicated and it's been shown how you can turn the mess into a "virtual CPU" that runs below the CPU ring protection methods thus "owns the kingdom"... To make it worse it probably also alows the SGX security mechanism to be breached as well which is what I've been thinking about off and on since first reading the paper.

The only real question is "How do you get at physical memory from the ISA?" to change it. Well in theory you can not because there was no designed method to "reach down" around the protection the security the MMU effectively provides.

However theory and practice do not always align and you have to start thinking about what is there by consequence not just design. Thus that known but not much talked about defect in DRAM in effect opened the possability of a time based side channel around the MMU, and that is where RowHamner works. In essence you overload a defect in the refresh logic for the capacitor in a memory cell and thus change it's state. In most software etc this would not have much in the way of security implications as the most likely result would be a program crash... However some parts of memory are very critical such as the "Page Table" for the MMU that effectively controls the security settings the MMU provides which with process orientated OSs is most of it...

As long as the DRAM issue exists and it's fairly fundemental to it's design the side channel will exist and thus "reach down" attacks from the ISA around the MMU will be possible as long as the attacker can "flip the bits" in the desired place to go further with the attack.

From what is written it appears that the "location" part of the --as yet not publically divulged--attack works by a "Cache Timing Attack". These have been known about for quite a long time as they are a subset of the broader "time based side channels" that the likes of the UK and US SigInt agencies have exploited since before WWII (ie before GCHQ and the NSA had been thought of let alone chartered). The attacks worked against very high security Super Encryption Machines for teleprinter circuits, that used One Time Tapes such as the systems that predated Ben "Pat" Baylys's "Rockex" machine[4].

One of the valid complaints about the AES competition was that the process actively encoraged time based side chanbels to be built in by default in most implementations. As the NIST competition was technically directed by the NSA and they were very well aware of time based side channels, why did they alow it to happen... Some say it's actually what they wanted to happen. Thus if that's the case they got a good ten years or so out of it and probably still do...

The only temporary solution on the software side people can see is to split out the U-area and K-area page tables from each other which means they are in effect "out of cache" with respect to each other and that is where the big performance hit comes from...

The fun bit is that it appears the attack has been developed in a high level interpreted language that nearly every web browser runs by default... Hence the platform / OS independence of the attack...

[1] https://www.wikipedia.org/wiki/74181

[2] https://en.wikichip.org/wiki/amd/am2900

[3] https://en.m.wikipedia.org/wiki/Dynamic_random-access_memory

[4] http://jproc.ca/crypto/rockex.html

CallMeLateForSupperJanuary 2, 2018 2:57 PM

How to get people to gladly hand over their most personal data: make them pay handsomely and give them a trinket.

"Between Black Friday and Cyber Monday, leading personal genomics company AncestryDNA sold about 1.5 million testing kits designed to provide insights into your ethnicity and familial connections.

"[...] some policymakers and public health officials [are] concerned about the pace with which people are blindly giving away their genetic data to these types of companies, who can turn around and sell it to third parties."

https://www.wired.com/story/ancestrys-genetic-testing-kits-are-heading-for-your-stocking-this-year/

This service could have been easily set up double-blind, to protect identities. It wasn't. Gosh... I wonder why not.

CassandraJanuary 2, 2018 4:14 PM

@Clive

That technique is so ancient it has fluff on it. Of course, that does not mean it is not effective, but I thought that it was basic knowledge that you never, ever, share your source document: there are just too many ways it can be made identifiable and trackable. The KGB kept a typed reference copy of the full individual character set of every registered typewriter in the Soviet Union (you were in deep trouble if caught with an unregistered typewiter). The point was that by comparing the text of document with the library of typewriter scripts, you could determine which typewriter had produced the document, and go and visit the registered owner. It worked due to the small differences and imperfections in the castings of the letters, and the differing spacing between letters and lines. Methods of distribution of samizdat make for interesting reading. Unregistered typewriters were prized, as was access to a 'Banda' machine. A good overview of the techniques can be found in the history of the Polish Underground here: Duplicator Underground - in these days of the Internet, the sheer difficulty of mass-producing and distributing text is underestimated, so it is as well to know the old techniques.
These days, typewriters are not registered, but most colour printers are identifiable by the yellow dots embedded in all output.
Methods using pre-arranged synonyms of key words can be used both to identify documents, and as a low-bandwidth code. Roget's Thesaurus can be used as a code-book in plain sight.

Sigh. I'm sure you know all this, but as ever, I think it is worth re-hashing common knowledge every so often for new readers. I hope you are fully recovered, and on your way to having a happy, and healthy, New Year - and I wish the same to all readers of Bruce's blog.

Cassie.

WaelJanuary 2, 2018 4:39 PM

@Clive Robinson,

"Document canaries/fingerprints"

Interesting way to identify sources of information. Trick is how to install these characters on sets of computers.

Clive RobinsonJanuary 2, 2018 7:38 PM

@ Wael,

Trick is how to install these characters on sets of computers.

Err the same way you do most other fonts on any given computer...

But something tells me that is not actually what you mean. What I suspect you are realy asking is how to make a document that contains a font with the charecters in it, as well as computer IP address, user name and time.

The traditional answer would be "use Postscript" and embed the required program lines in the file as part of the print driver.

With word docs you could at one point embed visual basic as well as fonts in.a file (I'm assuming you still can).

However from the perspective of a "repository" of documents on a server, there is no reason the server could not "make a fingerprinted file" on the fly in much the same way a webserver does with a server side CGI program.

The real question though is what info to put in a file and when. If you download files as part of your work then obviously you want the user details and time at download. However you would also like to add the time details of when it is printed out and by whom. We know from what Ed Snowden said he did things with other users credentials in various ways he did not go into. Thus he might have being using the user account whilst doing "Support Work" for the user, he might even have modified their login script to install a RAT etc. Which means he might well have done things when neither he nor the user were ay the computer. So an extra time stamp might well be tied to CCTV footage, door/access control systems etc.

Within reason and sufficient planning and time most things are possible. So sometimes the hard part is first working out which bases you need and then covering them is somewhat easier. So you might decide "no USB" if done early enough computers can be sourced that do not have USB ports that can be used for data transfers.

WaelJanuary 2, 2018 9:29 PM

@Clive Robinson,

What I suspect you are realy asking is how to make a document that contains a font with the charecters in it, as well as computer IP address, user name and time.

That would qualify as document source identification -- not fingerprinting. Fingerprinting is statistical, probabilistic in nature, meaning it could have collisions and changes Identification is deterministic and easier to spot. What I am saying is one would hope to be able to identify the source country of the document at the very lease which would be helpful in some attribution situations. Subtle variances in fonts sent to each country could be use in combination with zero width characters to encode some information in the document that are not easy to spot.

By the way, I looked at these two lines from the article in a regular editor (Sublime Text 3,) and... no differences.

We're not the same text, even though we look the same.

We're not the same text, even though we look the same.

So I decided to use a hex editor to see what's going on, but I used vi first. In vi they look like this:

We're<200b> not the<200b> same text, even though we look the same.

We're not the same<200b> text, even though we look the same.

Moral of the story is to use a hex editor to make sure the output is what one wants. This applies to source code as well. Practice safe hex, and may the source be with you.

RatioJanuary 2, 2018 9:42 PM

@Wael,

Trick is how to install these characters on sets of computers.

You mean how to input them? The same way you input any other charcter: using a dedicated key, or using some combination or sequence of keys.

For example, the keyboard for Persian in iOS 11 (which is the version you have IIRC) has a dedicated key for ZWNJ. If there’s no dedicated key, you can input the character using its code point (U+200C for ZWNJ). Here you could also use a numeric charcter reference (either hexidecimal &#x200C; or decimal &#8204; for ZWNJ) or a character entity reference (&zwnj; for, well, you know…).

S​e‌e‍?   ⇐   S [ZWSP] e [ZWNJ] e [ZWJ] ?

WaelJanuary 2, 2018 9:55 PM

@Ratio,

You mean how to input them?

Actually how to to remove them! Fingerprinting is done by someone else trying to identify you, and you want to remove any markers that give your identity away.

the keyboard for Persian in iOS 11 (which is the version you have IIRC)

I don't use any Persian keyboards. I have an Arabic one. You recalled incorrectly.

S​e‌e‍? ⇐ S [ZWSP] e [ZWNJ] e [ZWJ] ?

That's what the spying agent will do to you, see? One should write a Perl script to strip those unwanted markers before publishing a damning document.

WaelJanuary 2, 2018 10:19 PM

I see the confusion -- my bad. When I said:

Trick is how to install these characters on sets of computers.

I was speaking about a subtly different set of characters or glyphs. And also from the spying agent perspective. That in addition to ZWNJ, which can be used to encode additional data.

RatioJanuary 2, 2018 10:26 PM

@Wael,

Fonts are collections of glyphs used for visual representation of characters. They’re not really what this is about.

Actually how to to remove them!

Using tr -cd '[:print:]' as a filter might do the trick. Depends on what (else) you hope to remove and/or normalize.

You recalled incorrectly.

I meant that IIRC you use a device with iOS 11, not that you use a Persian keyboard.

One should write a Perl script to strip those unwanted markers before publishing a damning document.

Maybe the tr command above works for this particular issue?

WaelJanuary 2, 2018 10:30 PM

@Ratio,

You recalled incorrectly.

I take that back. Precedence mistake. Your recollection refers to the OS version, and not to the keyboard type. You have indeed recalled correctly, Mister!

Perhaps it's you who has a Persian, Pashto, Dari, and Arabic keyboards. An interesting combination. Any chatter translations I can help with? Muhahaha ;)

WaelJanuary 2, 2018 10:35 PM

@Ratio,

Maybe the tr command above works for this particular issue?

Any tool would work: awk, sed, tr, ...

Btw, I noticed my mistake before you had to explain it to me (OS version / keyboard type.) I also noticed some typos but I'm too lazy to fix them.

WaelJanuary 2, 2018 10:44 PM

@Ratio,

Fonts are collections of glyphs used for visual representation of characters. They’re not really what this is about.

Wanna bet? How much is in your wallet? :)

RatioJanuary 2, 2018 10:56 PM

@Wael,

Oops, hadn’t seen your other comment. There are even more now.

I was speaking about a subtly different set of characters or glyphs. And also from the spying agent perspective. That in addition to ZWNJ, which can be used to encode additional data.

Again, I have trouble seeing how fonts or glyphs are really the issue. As the person “marking” the document, you only control the plain text, no? You have a stream of bytes that represents a bunch of code points in some character encoding like UTF-8 and that’s it.

Perhaps it's you who has a Persian, Pashto, Dari, and Arabic keyboards. An interesting combination. Any chatter translations I can help with? Muhahaha ;)

That’d be some reality, winner! (Not much Arabic in her case, though.)

As for me, [REDACTED] of [REDACTED] is [REDACTED]. ;-)

Wanna bet? How much is in your wallet? :)

Maybe we’re thinking of different situations? Are you thinking of documents with embedded fonts or something along those lines? Those markings are not gonna survive copy-paste. What do you have in mind?

oh reallyJanuary 2, 2018 10:58 PM

Can you strip out the 0-width font chars by saving a font-limited version (somehow?)
Or does the 0-width charset go with despite font-limiting and re-saving the file?

WaelJanuary 2, 2018 11:21 PM

@Ratio,

Not much Arabic in her case, though.

I read the article. She supposadly became an "expert" in Arabic within a two year span[1]. I know a thousand times more Arabic than her, and I don't consider myself an expert.

[1] More than you've seen in your wildest hallucinations, @Bong-Smoking Primitive Monkey-Brained Spook :)

RatioJanuary 2, 2018 11:22 PM

@oh really,

Some characters, such as these zero-width characters, don’t print by design. That is the issue. It’s not really about glyphs and fonts and all the rest. (@Wael may yet prove me wrong, but until that happens I’m right.)

@Wael,

Printed documents is one thing.

Now you’ve really lost me. How does that work? Isn’t this about text in electronic form?

RatioJanuary 2, 2018 11:29 PM

@Wael,

I know a thousand times more Arabic than her, and I don't consider myself an expert.

The more you know, the more you realize you don’t know.

Also, humility is a thing. (Or so they tell me.)

WaelJanuary 2, 2018 11:36 PM

@Ratio,

Now you’ve really lost me. How does that work? Isn’t this about text in electronic form?

From the article linked in this thread:

Reality folded up the document, stuffed it in her pantyhose, and walked out of the building, its sharp corners pressing into her skin.

Since when do electronic forms have sharp corners that press against skin? That's number one. Number two: I said it's gonna be tricky. I haven't thought of a way to do it with electronic media but I think it's doable :)

RatioJanuary 2, 2018 11:41 PM

@Wael,

I butchered that. It's: The more you know, the more you know you don’t know. Or: The more you learn, the more you realize how much you don’t know. Something like that.

Since when do electronic forms have sharp corners that press against skin?

Where do the zero-width characters come in? That situation is almost the dual of what I thought we were discussing.

WaelJanuary 2, 2018 11:42 PM

@Ratio,

Also, humility is a thing

Next thing you'll tell me is that crime doesn't pay. I'm the most humble man on earth :)

By the way, copy/paste isn't the only way. There is email, ftp upload, drive dumps into a USB disk, etc... oh, a Tor connection wouldn't help here, would it?

WaelJanuary 2, 2018 11:45 PM

@Ratio,

Where do the zero-width characters come in?

They don't! Two different things as I said before. Pins and needles; needles and pins...

WaelJanuary 2, 2018 11:52 PM

@Ratio,

There is a TLA who wants to be able to tell the source of documents electronic or otherwise.

Electronic, they can use the zero width stuff. Non electronic they can use small variances in glyphs. How they install these variances on all computers around the world (to fingerprint them) is the tricky part. In the past some malware was activated when certain fonts were present (Persian, if I correctly recall.) Does that make sense or have I been up too long?

RatioJanuary 2, 2018 11:54 PM

@Wael,

Oh, right. I see.

But then I don’t understand this comment:

Subtle variances in fonts sent to each country could be use in combination with zero width characters to encode some information in the document that are not easy to spot.

That’s about electronic documents, right?

And you’re right, copy-paste isn’t the only way the markings can be propagated. Wholesale copy of the document works just fine for that. The thing is that the other markings you seemed to have in mind (usage of special fonts and such) wouldn’t survive copy-paste, whereas these would.

WaelJanuary 3, 2018 12:10 AM

@Ratio,

But then I don’t understand this comment:

That comment applies to both printed hard copies and electronic documents...

Hard copies: Zero-width are non-printable by design so they'll theoretically get lost in the printer, unless they're transformed into different types of watermarks. Kinda what color laser printers do when banknotes are printed or "copied", but in a more subtle manner.

Soft copies: Variances in fonts are only visual. The character code will remain the same and will not convey the visual changes to the recipient as the recipient's rendering device will show characters properly, devoid of any visual variances the source had. Unless ... many things:

1- Soft copy is a screen capture picture or video
2- A header is embedded to identify the source variances in "glyphs"
...

Bong-Smoking Primitive Monkey-Brained SpookJanuary 3, 2018 12:38 AM

@ Ratio:

Unless ... many things:

Please tell me you're not going to buy that load of crap!

RatioJanuary 3, 2018 12:49 AM

@Wael,

Let’s see if I got it now.

Electronic, they can use the zero width stuff.

That’s one of ’em, yes.

Non electronic they can use small variances in glyphs. How they install these variances on all computers around the world (to fingerprint them) is the tricky part.

If you control the printed form of the document you can mark it that way. (This works as long as you’re looking at the printed document or optical replicas. Doesn’t survive OCR from the printed document for example.)

If you don’t, we’re talking about an electronic document that’ll be printed somewhere down the line, correct? You’d need a document format that lets you embed fonts so that the font travels with the text, unless you can make it so that your magic font is already present at the destination. (In that last case you’re not marking the document but the device.)

In the past some malware was activated when certain fonts were present (Persian, if I correctly recall.) Does that make sense or have I been up too long?

Doesn’t ring a bell, and yes.

@Mr. Spook,

Please tell me you're not going to buy that load of crap!

Those examples did sound vaguely plausible, and I count “zero, one, many”, so I’m gonna say I’m buying it. :-)

oh reallyJanuary 3, 2018 12:51 AM

" Some characters, such as these zero-width characters, don’t print by design. "

No, I get that much derp.

I'm asking can you strip those 0-width characters out by saving in a different (Custom?) charset that would exclude them intentionally and replace them with nothing?

If you could do that you could make a very simple script and clean all documents easily.

Which begs the question, would removing these chars from system files in a Win.x box break things?
Are they used in actual code?

WaelJanuary 3, 2018 1:00 AM

@Ratio,

In that last case you’re not marking the document but the device.

Devices are already marked. Perhaps paper and ink too.

@oh really,

Never trust anything not in binary format even if it's an innocent looking text document. Use a he. Editor or one of the tools @Ratio mentioned to strip "markers". This applies to binaries as well and was discussed a couple of years ago right here.

oh reallyJanuary 3, 2018 2:15 AM

But how do you evaluate say an email body in realtime with a hex editor, right?
(you don't really, do you? I'm impressed either way, but honesty impresses me the most)

My question - would there be a userland-simple way to strip out misc. invisible extra-charset markers using a reduction-format display copy/save into a reduced character file format type of quasi-batch script solution, or are these invisible 0-width character ubiquitous in everything including binaries or specific-language libraries, etc?

Would stripping them out altogether in a data diode sense break things, basically.

RachelJanuary 3, 2018 2:16 AM

Wael

you said you know a thousand times more Arabic than RW. I did read the article - but how do you know 'how much' she knows? Was there a Common European Framework language rating accorded to her, for which you surpass? ( not a trick question, sincere enquiry)

RachelJanuary 3, 2018 2:27 AM

Wael

calling someone 'expert' like that is in my mind lazy journalism. I'd rate terminology as per language competency in order of fluency, then native (equivalent) and some tiers beyond that lay the rarified 'expert'. Theres not many native english speakers whom deserve to he called expert.
The FSI - Foreign Services Institute have ranked languages according to the number of study contact hours for an native english speaker to approximate a Speaking Level 3 & Reading Level 3.
(Not sure what that level is defined as. It doesnt square with the Common EuropeanFramework rating I am familiar with)
Obviously the FSI rating is a guide but its considered fairly accurate.
Arabic is one of just a few in Class V ' Considered exceptionally difficult for English speakers' estimating 2200 study hours to achieve AS3 & R3

RachelJanuary 3, 2018 2:31 AM

to clarify- I intended, not many native english speakers whom deserve to be called expert in English.

For interest, FSI rates Pashto and the few Persian languages as Class IV - 'considerably difficult for english natives'. Requiring 1100 study hours.


I reckon we are seeing some holes in this here RW legend

RachelJanuary 3, 2018 2:41 AM

I dont recall seeing any references to RW sensitivity, passion or interest to/with the cultures that use the languages she is said to be expert in.
Those are highly unusual language choices. You dont just 'acquire' them. Learning just one is impressive. Having three is sheer brilliance. Further, You dont study languages like that without having some deeper cultural context immersion envelope one either before, or during learning.
I didnt read of any such empathy in RW. Only all the yoga blah blah stuff.

waitJanuary 3, 2018 3:09 AM

"I reckon we are seeing some holes in this here RW legend"

What is the insinuation here? RW didn't do her job for years without incident, or?

WaelJanuary 3, 2018 4:21 AM

@Rachel,

I did read the article - but how do you know 'how much' she knows?

Good question! If I spent 1100 hours learning French, how much more would you know more than me? How much more if you had spent many years studying various aspects of the language?

There is only that much one can do in two years, give or take a standard deviation. Perhaps she became acquainted or familiar enough with the language to be able to transcribe and translate conversations. That's probably a third or fourth grade level skill. I don't recall they mentioning her attaining a standard level.

To be an expert, in my opinion, one has to know more than that. A whole lot more. Grammar (two major schools,) vocabulary, poetry, literature, differences in Arabic languages (Arabic is an ensemble of languages - all proper Arabic) etc... basically to be an expert one needs to at least attain professor-hood level in the language, and that's still not enough!

Take for example Dr. Fadel Al Samaraie[1], he is speaking with Dr. Mohammed Khalid -- a professor of Arabic Language at Cairo university -- and makes him look like a baby. Dr. Fadel doesn't consider himself to be an expert either (and he's been teaching graduate level students for over fourty years.) He's a very humble man, but he wasn't being humble when he said that, although he is probably the top grammarian of these times. But that's only one branch.

Anyway: a thousand times more is a figure of speech. It's a fuzzy term that means a lot more. A thousand pardons ;)

[1] The name means from Samarra; an old city in Iraq.

WaelJanuary 3, 2018 4:39 AM

@oh really,

But how do you evaluate say an email body in realtime with a hex editor, right?

Many ways. Use an editor that displays both text and hex, strip unwanted characters before you copy/paste.

you don't really, do you?

I don't, although I'm tempted to do so every time I read a comment from @tyr :-) I just keep forgetting!

My question - would there be a userland-simple way to strip out misc...

Yes, a Perl script or one of the other tools mentioned above.

Would stripping them out altogether in a data diode sense break things, basically.

Of course! It'll break things for offense and strengthen defense. That's the idea, right?

RachelJanuary 3, 2018 4:59 AM

wait

I was being a bit sarcastic referring to RW as having/being a legend. but who the hell knows.
I just meant I find it implausible and convenient. Theres also no mention of having savant like attributes that may invoke certain styles of relating or motivations.
If she was a pawn that was burnt why burn someone so unique - a basically priceless employee.

Wael
Thanks for that. Very interesting & I'm on the same page with all you said. The inference also being that you are a native arabic speaker, which I didnt know.

RachelJanuary 3, 2018 5:27 AM

Wael

I reread the article. It says she started Arabic from pasion age 17. Then Dari & Farsi for two years in the military and became fluent in Pashto via special selection course 'along the way'.
Being an interpretator for 12hr shifts is -hard-. even if its recorded audio you have to complete transcribings to a certain quality for prompt submission. the ratio expected for a court recorder is 3:1 , 5min audio must be complete in 15mins.

Clive RobinsonJanuary 3, 2018 5:52 AM

@ Cassie,

That technique is so ancient it has fluff on it. Of course, that does not mean it is not effective, but I thought that it was basic knowledge that you never, ever, share your source document

Yup it makes even Methuselah look like a "teenage hippy" in comparison. But that is a big part of the problem...

Which brings me to your last point,

I'm sure you know all this, but as ever, I think it is worth re-hashing common knowledge every so often for new readers.

Yes it is a "requirment" to keep saying it almost ad nauseam...

The reason is "society"... As you note with the KGB and many other police state entities and the "disappeared" citizens. Trying to survive in such a society made knowing this knowledge a "life skill" that like crossing the road would kill or severly hurt you if you got it wrong.

As I tell people "All actions have consequences, it's a matter of degree after the event, which proper planning and response can mitigate". After all life is only a zero sum game when you die.

The important point though is it is "society" that shapes us and therefore our actions and it can be quite insidious, thus people "sleepwalk into a living nightmare" as past behavious haunt and condemn them in the present...

Some time ago now @Nick P and myself had a chat about how old attacks became new attacks every few years. What sparked it was a variation on the old basic MBR Virus attack. Back in the days of floppies being used as the "data packets" between machines, most vaguely tech savvy users were aware of "Boot Sector" viruses and how they spread. But then Ethernet and PPP became the new kids on the block in the mid 1990's and people forgot about disks and the MBR. But Flash drives came along and eventually some malware writer either remembered, read in an old book or "heard a war story" and the next thing you know people were getting "smacked six ways to Christmas" by what was in effect an MBR attack...

The point is that contrary to what most people think, technology does not improve linearly, it happens in jumps and rests with a time line like a childs drawing of a flight of steps/stairs. As there are many different technology types at any one time people move from one technology to another as it better suits their needs. They also transfer back when a technology jumps ahead again. Thus you will see data transfer moving from hand carrying a storage device (floppy) to moving down RG58 coax as 10Mb/s Thin net and dial up modems. Then swinging back with CD/R and DVD/RW, then to 100bT and early broadband etc. Eventually it came around to Thumb drives but has gone back to GigaBit and home Fiber etc.

Thus you see this fliping and flopping between two basic classes of technology, but at each flip or flop people either forget or never learn previous security issues.

Arguably these canary/fingerprint systems are a form of stenography that goes back to tattooing slaves heads and hiding scratch marks on wood that is then covered in wax to look like an unused stylus writing block. In more recent times to the likes of Francis Bacon and his binary based codes for writing in a covert channel into plaintext letters. If you have seen a copy of David Khans book you will see a pen and ink landscape picture which has grass or reeds along a bank in variable length pen strokes. If you know morse code you will be able to read a message from those otherwise innocuous pen strokes.

The point is these technology classes and their instances happened and lessons were learned at the time but then forgoton by the majority when a different technology got used.

What @Nick P and myself were supprised at was just how fast this knowledge got lost. OK the MBR trick was nearly two decades, but some others were as little as two or three "tech generations" of around 12-18months each...

Take Reality Winner, she either forgot, never learned or over trusted, so she sent the original print out. I can accept that she might never have been told or over trusted. After all "thinking hinky" is not something most people ever get to do and if they --outside of certain fields of endevor-- start asking what are actually valid security questions they will get accused of paranoia or weird / suspicious behaviour by everybody else. You can tell just how bad society in general has got when you see the "See Something Say Something" "Four S" adverts.

What I find totally unforgivable however is the Intercept Journalist that due to compleat and utter incompetence sold her down the river. Investigative Journalism is one of those fields of endevor where "thinking hinky" is not just a requirment it's an essential way of staying alive...

So if people call me paranoid etc atleast they cannot accuse me of witholding it from them when they make a mistake.

By the way it was not just the KGB keeping typrwriter print outs the FBI did and presumably still does, likewise for photocopiers etc. Whilst it will not get you down directly to an individual machine it will get you a manufacturer and model and even a production run. Which is often enough...

What might appear to fly in the face of the Reality Winner story is my "mantra" that I tell people which is "Paper Paper Never Data" when dealing with legal types as electronic discovery involves a wealth of hidden metadata that can be exploited by them. Atleast with paper you see what you are giving them.

This mantra is because of an assumption which is that the people I tell are basicaly law abiding, and have fallen into the Shark pool that litication is all about. Which is mainly about appearences not facts or what I would call evidence.

It's when that assumption changes to that of IC/LEO involvment for political reasons over a "sensitive document" that I start telling them about "thinking hinky" and the what to do of OpSec.

In most cases for a "sensitive document" it's "retype as plain ASCII text remove all but basic punctuation and change words and/or their order, change figures for words etc". I would then take them through making a clean machine to do it on or how to do it with pieces of paper you then burn and break up the ashes befor flushing. I have other advice for redacting etc. One little box of tricks I realy miss is the Overhead projector and loose heavily creased bed sheet screen to project it on and then photograph with an old style black and white film camera which is slightly out of focus on large grain film stock, then over develop it. You end up with what looks like the old micro-dot photographs with bled out edges on the type face etc.

If you are printing pamphlets / flyers one old trick I've never tried is the old oil/bromioil idea but with PCB etching and then silk screen printing. After you etch the PCB you can then "age" it with a sponge with etchant on this will add random surface pitting and all sorts of other issues including under and over etching etc, thus hiding many fingerprint features. My father used to use a Roneo spirt printer for lecture note handouts, and I used to see him make corrections with amongst other things a razor blade and a liquid corrector that used to smell of pears.

Basic OpSec is a quite thoughtful and for your own safety usually introspective task, as in police states (which various western nations are becoming) people will sell you out for the price of a beer.

The funny thing is that what the KGB did with registered typwriters was not stop disident behavior, but kill the economy. Put simply the basic tool of commerce is the spoken and written word, sent quickly and efficiently. Any check or hinderence on those has very significant cost implications thus kills not just profit but competitive behaviour.

What many in the US do not realise is that the USG is justvas stupid. For instance one thing that has amused me in the past is the US fixation on the worthless polygraph. It's been long known how to beat it, thus rather than see the light and give up the "techno toy" they try to make it illegal for people to tell you how to beat it. The same stupidity applys to TEMPEST techniques, which are based on basic physics and the laws of nature. Then of course is the more modern and equally as detrimentl to the economy Cyber-Offence prioritization over Cyber-defence. Recent malware from leaked NSA cyber-weapons should have been a major red flag wake up call... But no it's kind of hard to tell which is worse, the US IC/LE organisations or the Australian Prime Minister...

The thing is the Western IC/LE organisation War on Terror and Cyber-Offense policies where and still are a major cancer on the economy of the respective nations. Worse whilst the FiveEyes nations are very vulnerable, other Super Powers and first/second world nations are not so vulnerable and becoming less so as more dirt gets exposed in the US.

JG4January 3, 2018 6:32 AM


Had a Digikey shipment held up by the name-matching problem that Dave Jones described on EEVBlog. Given that the fastest cheapest way to project intent uses electronics, the general approach probably isn't wrong.

The discussion of fonts and ascii code boils down to filtering, which also is required in C-v-P and various other security measures. You could think of OCR as a filtering step, and there should be a trivial filter that renders all ascii characters into a limited range of the lower case letters. Pattern recognition in OODA also is a filtering step. I may have forgotten to mention that there is no phase information in the DC bin of a discrete FT, nor in the Nyquist bin. I think that I pointed out that the Nyquist bin is blind to any signal at quadrature to the sampling grid. The FT is just a filter bank.

I think that Biomimicry made an appearance here months ago via nakedcapitalism. I saw an article yesterday about bird migratory navigation (optimized for energy expenditure, not distance traveled) that made me think that it would be clever to look for biological Kalman filters. Speaking of Biomimicry, the next Youtube video that queued up was Janine Benyus (sp?) receiving an award from Bioneers, presented by Paul Hawken. He launched into a story about "chemical-free" carpet. I haven't pulled that thread all the way yet, but I am interested in escaping from the endocrine disruptors that are poisoning us.

https://www.nakedcapitalism.com/2018/01/links-1318.html
...
Big Brother Is Watching You

‘Kernel memory leaking’ Intel processor design flaw forces Linux, Windows redesign The Register (E. Mayer). E. Mayer: “Existing ‘flaws’, a.k.a. NSA-designed backdoors which have not yet known to have been discovered and exploited by third parties, will presumably remain operative. Carry on!” And for the extemely geeky: The mysterious case of the Linux Page Table Isolation patches python sweetness. “I would not be surprised if we start 2018 with the release of the mother of all hypervisor privilege escalation bugs.”

Google Maps’s Moat Justin O’Beirne (GP). “[T]he satellites seem to be outpacing the Street View vehicles.” Well worth a read.

Ad targeters are pulling data from your browser’s password manager The Verge

Clive RobinsonJanuary 3, 2018 6:49 AM

@ Oh Realy,

But how do you evaluate say an email body in realtime with a hex editor, right? (you don't really, do you? I'm impressed either way, but honesty impresses me the most)

I don't do personal Email any more but yes I do spend a lot of time at the CLI filtering file formates and reading text in a hex editor. Most of which display ASCII text without peoblem on the right hand side of the page, with "." where non printing charecters are.

The use of the "Mark 1. Eyeball" via a *nix "od" or hex editor is usually the first step with a forensic look at an unknown file format. A look for "magic numbers" is also a part of it but most *nix distributions have a "files" tool to help with that. If those fail a statistical analysis across the file will show encrypted or compressed areas or even byte code or similar. Experience with working on PDF files will give you an idea of just how much of a headache recovering plaintext can be.

If I understand you correctly you want somethin a little more limited which is a program that will read in a good non compressed or encrypted file and find any "text" and then white lists it to remove or even replace those suspicious charecters.

This is kind of a multi step problem with modern file formats because of all the other crap that is in there. When I pull documents back and put them together for people my basic steps back in the 1990s was pull the file onto a *nix box then use "tr" and similar to strip out control charecters and the like, then use "strings" to get most of the document back into a realy rough text file then use that to search and find basic formating information and manually apply it. The user could then ussually load that back into their document editor and reformat it proof it and fill in any blanks etc.

That does not work as well these days for a variaty of reasons the most annoying being Unicode, it is without doubt an abomination, however the likes of Perl and Python scripting languages will deal with it better than most humans. Aside from that the process would be similar, except you would just change the "blacklist" charecters to a special visable char. Having done that you just pull it back into the document processing program and manually remove them etc befor saving to a new file.

The problem is that some document formants alow for embedded programs and fonts, this suggests printing out to a new file format such as Postscript that you can then edit with a specialized tool.

One way might be to print it out as an image file then load that into an OCR program then manually find and correct errors. As I've mentioned before a similar technique but actually with real paper works for "energy gap" crossing.

But at the end of the day I find getting everything in seven bit ASCII text files works for me, but then I work with "English" so it's an "easy option" compared to many languages.

Some modern OCR systems can deal with Chinese / Japanese / Korean standard charecters and convert them into Unicode or equivalent, which on theory means it's jus the white/black list issues you have to deal with.

I guess that people could write several books on this sort of thing, and I already know of a couple for programers doing "Internationalisation" that precede the major use of Unicode.

Bong-Smoking Primitive Monkey-Brained SpookJanuary 3, 2018 7:34 AM

@JG4,

Had a Digikey shipment held up...

Must be @r at it again. It's called interdiction.

Big Brother is Watching You Watch

Poke him in his good eye.

WaelJanuary 3, 2018 9:28 AM

@Rachel,

I need to take a break...

The inference also being that you are a native arabic speaker, which I didnt know.

That'll remain to be an inference. You still don't know :)

bttbJanuary 3, 2018 12:09 PM

For people interested in current events

A Reporter at Large
January 8, 2018 Issue, New Yorker
"Making China Great Again
As Donald Trump surrenders America’s global commitments, Xi Jinping is learning to pick up the pieces."
By Evan Osnos
https://www.newyorker.com/magazine/2018/01/08/making-china-great-again

Currently Osnos is live on NPR's Fresh Air
https://www.npr.org/programs/fresh-air/ ; streaming or on the radio, perhaps, near you
https://en.wikipedia.org/wiki/Fresh_Air

oh reallyJanuary 3, 2018 2:28 PM

Bannon just CONFIRMED that Trump definitely knew about the Russian meetings with Jr.

He said it was TREASON. STEVE BANNON just accused TRUMP of "definite" treason.

Good luck with the whataboutism, Dan H! Lol

Clive RobinsonJanuary 3, 2018 2:45 PM

@ hmm,

my nuclear button is bigger

Yup, "mine's/it's bigger / longer / harder / etc" appears to have been the main excuse for a ruckus since before we swung down from the trees and learnt how to get a good swing with a branch or rock etc... God alone knows how many millennia ago that got hardwired into our lizzard brain... It's apparently an "Alpha male charecteristic". But in social groups it can give rise to "security" for the group by ensuring preferential acces to limited resources.

Funnily enough there is an argument that "secrecy", actually originated not with the male of the species but with the female... It's eveloutionary later at the hunter-gatherer stage. Females with off spring had better survival prospects by not letting other rival females and males know where a tree with ripening fruit etc was.

There is another argument put forward by a few that the reason the female is seen as the better communicator is based on being able to detect when another female is hiding information.

Whilst the first argument is more or less excepted by all the second has not been sufficiently researched (though how the heck you would do this is an interesting question).

The third argument is seen as a bit contentious by a number of people kind of along the lines of "If it feels right it's probably wrong", ie it's to obvious / simplistic / trite / etc.

However all three can be fun conversations if you want to see people trying to defend what are probably untenable positions ;-)

The fun thing about this was many years ago now I got chatting with Desmond Morris[1] at a party in Oxford back when he was doing research for "Peoplewatching". He made an interesting observation that "Meat eating was social and fruit eating antisocial" as well as the former being primarily male and the latter female activities in the hunter-gatherer existance and subsequently. Which if you think about it are in line with common "maleness perceptions" but not female. But he also noted fairly early on that vegetarian behaviour was predominantly a female behaviour. He thus questioned if this would spread anti social behaviour in society. Judging by the way we tend not to live in family units untill a lot later in life these days he may have a point that would bear a lot further investigation if it was not seen as being "non PC".

[1] https://en.m.wikipedia.org/wiki/Desmond_Morris

holy_cowJanuary 3, 2018 7:17 PM

A couple people already mentioned it, but this deserves a full post once more details are in.

Apparently all Intel CPUs since Core2 Duo, and possibly even farther back (circa 1995?) have a serious design flaw involving speculative execution, that allows exploit code running at ring3 (usermode) to read ring0 (kernel) memory.

https://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/

All the OS vendors are doing workarounds ("Kernel Page Table Isolation") which will have performance impact. Microsoft has reportedly had insider-preview builds with the workaround/patch in them since some time in November. They roll it out to everyone on the next Patch Tuesday.

Proof of Concept exploits have been demonstrated: https://twitter.com/brainsmoke/status/948561799875502080

AWS and Azure have forced reboots of all VMs scheduled, presumably to deploy these patches for all OSes.

In *totally unrelated* news, Intel CEO sold almost half of his stock on November 29th, keeping only the bare minimum he needs to hold to be CEO:
https://www.fool.com/investing/2017/12/19/intels-ceo-just-sold-a-lot-of-stock.aspx
No insider trading going on there, no sirree!

holy_cowJanuary 3, 2018 7:34 PM

Looks like Google went public with more info about the bug:

https://security.googleblog.com/2018/01/todays-cpu-vulnerability-what-you-need.html?m=1

...

"The Project Zero researcher, Jann Horn, demonstrated that malicious actors could take advantage of speculative execution to read system memory that should have been inaccessible. For example, an unauthorized party may read sensitive information in the system’s memory such as passwords, encryption keys, or sensitive information open in applications. Testing also showed that an attack running on one virtual machine was able to access the physical memory of the host machine, and through that, gain read-access to the memory of a different virtual machine on the same host.

These vulnerabilities affect many CPUs, including those from AMD, ARM, and Intel, as well as the devices and operating systems running on them."

...

"There is no single fix for all three attack variants; each requires protection independently. Many vendors have patches available for one or more of these attacks."

Anonymous2cJanuary 3, 2018 8:48 PM

Below are some excerpts from Glenn Greenwald on Democracy Now. Republicans, Democrats, Independents, etc., might find him thought provoking. You can watch the show or read the transcript at
https://www.democracynow.org/2018/1/2/glenn_greenwald_on_iran_protests_trump

[...]

"GLENN GREENWALD: So, Iran is an extremely sophisticated and complex country of 80 million people. And I think that when it comes to analyzing exactly what’s driving the protest in Iran, we ought to defer to Iranians, people who are steeped in Iran’s civil society, and ought to avoid the sort of overnight experts who tend to pop up in the West and opine on these matters from afar without much knowledge."

[...]

"GLENN GREENWALD: Yeah. I mean, first of all, the centerpiece of U.S. foreign policy, really in the wake of World War II through the Cold War, and then even with the fall of the Soviet Union, has been to align with and to embrace and to support dictators, tyrants and repressive regimes, as long as they serve the interests of the United States. So, anybody in their right mind who ever takes seriously pronouncements from official Washington that they’re motivated by anger over repression or a defense of the political rights of people in other countries is incredibly naive at best, to put that generously.

Just this week, Juan, there was an amazing leak that Politico published, which was a State Department memo written to Secretary of State Rex Tillerson that explicitly said what has been long obvious, but usually isn’t put into words so clear, that human rights is not actually something the U.S. government believes in; it is a cudgel that it uses to undermine and bash countries that don’t serve its interests. They use denunciations of human rights abuses to undermine and weaken governments that are contrary to their agenda, like in Iran, while at the same time, this memo said—this isn’t me saying this, this is the State Department memo saying—they overlook and even sanction repressive behavior on the part of their allies."
https://www.politico.com/story/2017/12/19/tillerson-state-human-rights-304118 link in prior paragraph
"And it goes beyond the Trump administration. I mean, if you look at how official Washington works in terms of, say, the leading think tanks in Washington, the Brookings Institution, for example, which has become incredibly popular among liberals in the Trump era, is funded with tens of millions of dollars by the government of Qatar, one of the most repressive regimes on the planet. The Center for American Progress, which is probably the leading Democratic Party think tank in the United States, is funded in—one of their biggest funders is the government of the United Arab Emirates."

The show also covered Facebook censorship.
https://www.democracynow.org/2018/1/2/glenn_greenwald_is_facebook_operating_as
https://theintercept.com/2017/12/30/facebook-says-it-is-deleting-accounts-at-the-direction-of-the-u-s-and-israeli-governments/

hmmJanuary 3, 2018 9:44 PM

" Judging by the way we tend not to live in family units untill a lot later in life these days "

What did you mean? I don't get it.

Clive RobinsonJanuary 3, 2018 11:46 PM

@ Holy_Cow, All,

A couple people already mentioned it, but this deserves a full post once more details are in.

See Anon's comment above,

https://www.schneier.com/blog/archives/2017/12/friday_squid_bl_607.html#c6766938

And my reply immediately following it.

As I noted I'd read one of the published papers over the winter holiday period,

    And it opens a whole world of nightmares for those that know how to look at things even slightly hinky...

With a further description of the problem being below the ISA in the computing stack, in places few ever get to go in their career these days...

It appears that "in this case" --which is just one of many nightmares opened up-- it is both above and below the ALU and Register level in the stack. That is from the latest info[1][2] it now appears to do with the speculative look ahead circuitry that pulls in core memory into the cache by bypassing the ALU and registers and talking directly to memory at a point beyond the security --supposadly-- provided by the MMU and Virtual Memory or VM system.

It appears to be a case of "Efficiency-v-Security" that I go on about on the odd occasion or three from time to time ;-)

Basically if what has been described more recently[1] is correct, by striving for more efficiency the hardware designers have opened up a time based side channel[4] --via cache hit/miss timing-- by which memory a process should not have access to can be enumerated by the process.

This enumeration is possible due to access to very high resolution time refrences to javascript interpreted code used in all modern web browsers[3] which not only makes it platform independent, it also hemorrhages a whole host of other information leaks.

It appears one "fix option" is to lower the resolution of the time information available in the web browser[3]. History tells us that this is a dangerous and problematic fix (think back to the late 1990's to Smart Cards and powersupply enumeration repeatedly improving and ending up with DPA).

It's "Dangerous" because by the process of integrating many repeated timings the resolution can be got back (each time you double up the readings you in effect gain one more "bit" of resolution back. Likewise "Problematic" because if you make a significant change to the resolution existing programs will also get broken, possibly irrevocably so, which has liability issues (if you have payed for code that is now irrevocably broken you are going to want your money back or blood or both, which kind of makes Intel the "Insurerer of last resort" where "The buck stops"...[5]).

Whilst what I wrote in my reply to Anon above is just one nightmare case that fitted the then publicly available information, it now appears from new publicly available information to be a different nightmare proplem. Which are just two of the several "thinking hinky" nightmares that paper from last year opened up in my head.

The other thing to note is that the earlier publicly available information indicated it was Intel, AMD and ARM hardware that was effected, the latest only Intel. However my oft made point about the needless over complexity of some modern CPU designs still holds. Which gives rise to the question of "Has Intel gone to far down an evolutionary cul de sac to pull back?" and will that make Intel a real HiTec version of the alleged "saber toothed tiger" evolutionary extinction. Some appear from what is being said to think it will or be close to[5]...

I'm not yet ready to write Intel off but this is sure going to hurt just as the "Pentium Bug" did. If people can remember back, that partially kicked Intel out of an "Efficiency cul de sac" but only a small part of the way.

The problem is like that of Castle builders of the past, you run into diminishing returns problems. Yes you can make a castle taller, but you have to either accept that makes the walls weaker, or build thicker heavier walls that in turn need more massive foundations, which has other knock on effects in terms of cost and insecurity. Or you radically change the way you do things, Skyscrapers only just became possible with steel reinforced concrete. To get higher a radical rethink of how a build was made was required. In effect you got rid of heavy rigid "structural walls" which alowed glass to be used instead that alowed larger floor areas as a bonus, then there was the people moving problem that the original World Trade center resolved with express lifts etc enabling them to be even taller. The point is both changes required a complete reversal out of an architectural cul de sac, which gives you an indication of the shakeup Intel is going to have to go through to survive. It's not going to be Intel's first major change, the limits on tranister speed/heat forced them out from the single sequential core CPU into the multi-core parallel world, the next set of changes will be in effect even bigger, and at some point backwards compatability will have to go otherwise they will loose to ARM and for other reasons to AMD. One change might well be that the idea of reprogramable microcode will get extended, by the use of FPGA techniques to make lower level hardware reprograming in the computing stack area between the ISA and ALU/Register level and even below.

[1] https://googleprojectzero.blogspot.com/2018/01/reading-privileged-memory-with-side.html

[2] https://meltdownattack.com/

[3] https://blog.mozilla.org/security/2018/01/03/mitigations-landing-new-class-timing-attack/

[4] As I've noted before an engineer "skilled in the art" of both the "problem domain" and the "Secure system design domain" can get limited increases of one type of Efficiency, without reducing security. That is by not opening up timing side channels etc that significantly reduce Security. But they quickly get hit by the law of diminishing returns that makes another areas considerably less efficient. As an example think about "Pipelining" you get some improvment in through put, but not only do you get a disproportionate increase in delay, you also pay heavily with increased circuitry complexity as well as power consumption which makes you hit the "heat death" wall faster than other solutions.

[5] There are stories now starting to appear about the problem being reported to Intel in Nov last year, and that the Intel CEO cashed in a large part of his Intel shares holding also in Nov, thus implying "Insider Trading"...

ThothJanuary 4, 2018 5:25 AM

@all

The Stasi regime has arrived in full force in Singapore.

All new mobile phones in Singapore are forced to install the SGSecure backdoor and Stasi surveillance app I mentioned in the past.

The SGSecure app is a theoretical backdoor and is for the citizenry to report suspicious behaviours via the app and spy on each other just like the Communist and Nazi regimes.

Now the citizenry in Singapore cannot opp out of the total surveillance and snitching on each other and Stasi control mechanisms in place.

Clive RobinsonJanuary 4, 2018 6:01 AM

@ hmm,

What did you mean? I don't get it.

A large number of people believe --rightly or wrongly[1]-- that the bed rock of society is the family. Not just a cohabiting family in a house with suifficient rooms but a family that sits together eats together at the same time talks together etc etc thus establishing very strong bonds. A subtext of this view is that children should only leave home to marry and setup a family of their own quickly and have atleast an heir each and a spare preferabley a couple. Thus three to five children with a two to four year age gap[2].

Thus under that conservative religious view point girls are for breeding and boys for minimal wage labour, both bereft of a higher education that drives economic success. The Church gets new blood and 10% of their labours, the business men get low cost drudge labour that can be hired, fired and mistreated in any which way the employer choses, individuals rights get crushed and shaped to that of the church and employer.

And that's how it was except during major conflict when men were told to be cannon fodder untill the start of the sixties.

We are led to believe that the sixties were all about "Sex Drugs and Rock n Roll" or various hippy communes etc... It was not much of it was "self empowerment" industrialisation and relaxed post war restrictions ment 14year olds with real money in their pocket and a growing high street economy for them to spend it in. There was also plenty of cheap rooms to rebt etc, thus for girls the transition was nolonger "Dolls to marital bed and motherhood". Yes there was a relaxation in morals but not realy as much as you might think (morals were actually much looser in WWII and i n the Victorian era).

The two "scary things" for the establishment were in cities people stopped going to church did not marry and had less children and labour was actually a shortage thus high wages and even accepting women in the workplace did not cover the shortages.

But a third problem hit in hard. WWII had been a "technology war" and thus the real birth of electronics and industrial technology. That needed a skilled work force that was not there. In the UK the school leaving age was raised to 16 and technology collages and other higher education started expanding quite rapidly, especially slightly later with the polytechnics.

Thus the base of society the together familly was on it's way out in part in the short term to drive the needs of industry, but for women it ment real "freedom".

It soon became clear that those wanting early marriage was actually the "males" not the "females". Desmond Morris's point is that men are actually inherently social and eat that way (see kabab shops at pub closing for example). But you rarely if ever see people walking down the street sharing fruit let alone vegetables (though that has changed some with mothers and children).

The point which is actually more than a trend when you dig into it is that where they can women are not marrying, they carry on studying and get better jobs and social standing their own personal living space to live as they want which is in effect unencumbered. If they do decide to have children it's very much their choice of when, how many and with whom.

And yes the diet re meat-v-vegatarian is a strong correlating factor in this.

As for men, when you pull off the macho bravado struting talk, they are the ones sitting on the shelf today in the average and above IQ range.

There are two points arising from this.

1, the population size is falling and the age gap is widening.
2, it's falling and widening most in the average and upper age range.

From the conservative religious view point the world is now extreamly "anti-social" as they see it. But worse it's effecting the tax take by reducing it significantly whilst people are living a lot longer. There are significant even more anti-social effects because of this.

So the real question is being increasingly vegetarian a driver, correlation or coincidence? I've heard many arguments for each case. Perhaps the oddist is it's a natural effect population control in high density populations. There is actually a very real cause and effect relation between fertility in women and eating both dairy and meat products so it might not be as odd as it first sounds.

It's one of those things you have to do your own research on and come to your own conclusions, because there realy is not an ethical way to experiment reliably.

[1] The idea of a "together family" especially with a male "head of the house hold" who sets the thinking and policy of the family is often pushed by those with very conservative views that are often driven by religion.

[2] Again a conservative religious view point bassed on continuous indoctronation into the "raising children for god" view point which makes a religion very wealthy over time. It's also seen as good for the economy, especially by those who profit by cheap labour (see the history of "The Protestent Work Ethic" to see the propaganda in it and who benifits).

JG4January 4, 2018 6:18 AM


Thanks for the great discussion.

https://www.nakedcapitalism.com/2018/01/links-1418.html
...
New Cold War

Tony Blair ‘warned Trump’ that UK may have spied on him The Times.
...
Our Famously Free Press

The Biggest Secret: My Life as a New York Times Reporter in the Shadow of the War on Terror The Intercept. (The horrid mobile-friendly formatting goes away after the lead, so grab a cup of coffee and keep reading.)

Why I Left The Intercept: The Surveillance Story They Let Go Untold for 15 Months emptywheel
...[well covered by Clive and others]
Meltdown, Spectre: The password theft bugs at the heart of Intel CPUs The Register

Kernel panic! What are Meltdown and Spectre, the bugs affecting nearly every computer and device? Tech Crunch

Researchers Discover Two Major Flaws in the World’s Computers NYT. “Amazon told customers of its Amazon Web Services cloud service that the [Meltdown] vulnerability ‘has existed for more than 20 years in modern processor architectures.'” Which makes you wonder how long the intellignence community has known about the flaws. Eh?

Tech groups race to fix chip design flaw FT. I always regard “race to” in a headline as a bullshit tell. And in fact, the flaw is said to have been discovered in June 2017, and the entire industry has been working to solve it in concert, as paragraph 13 makes clear. Do better, FT.
...
A fantastical ship has set out to seek Malaysian Airlines flight 370 The Economist
...
GIMPS Project Discovers Largest Known Prime Number: 277,232,917-1 Mersenne.org (E. Mayer)

echoJanuary 4, 2018 6:32 AM

This is a media article covering "modern slavery". I have extracted the two paragraphs dealign solely with detection and opsec. The rest of the article discusses the situational imprisonment and psychological abuse of victims, and leadership forcing changes of attitudes within the prosecution community and also encouraging citizens to act on their suspicions and raise concerns.

http://www.independent.co.uk/news/uk/crime/modern-slavery-nail-bars-bath-case-jailed-police-staffing-vietnamese-uk-britain-forced-labour-a8137031.html

They transferred their victims to beauty parlours across England while dumping phones in efforts to evade police, sparking an intelligence operation involving the National Crime Agency. ... The detective described the group as “really savvy” in attempts to stop the girls being traced, changing phone numbers and getting rid of handsets, but a widening operation by six police forces and the National Crime Agency (NCA) repeatedly tracked them down.

Clive RobinsonJanuary 4, 2018 6:41 AM

@ Thoth,

All new mobile phones in Singapore are forced to install the SGSecure backdoor and Stasi surveillance app I mentioned in the past.

Well there there are two old saws the rest of us can shudder to,

    Comming to a place near you real soon now.

And consequently in the near future,

    You are not alone.

Isn't technology wonderfull, I bet SGScure is going to make CarrierIQ green with envy...

Getting SeriousJanuary 4, 2018 7:15 AM

It took a year the but the ‘White House Bans Staff From Using Personal Mobile Phones at Work’.

Prehaps this is due to a recent foreign top-secret document leak?

gordoJanuary 4, 2018 1:35 PM

@ Clive Robinson,

You're not only read, but quoted and comment-linked (as "a frequent and well-informed commenter on Bruce Schneier's security blog ..."). Considering the ideas, etc., of yours, of which you occasionally make note of having seen show up elsewhere, unattributed, kudos to the article's writer, Geoff Dutton:

The Internet wasn’t designed to be secure; quite the opposite, it is rife with holes in its backdoor code and protocols deliberately put there for reasons that might or might not have to do with government surveillance. A frequent and well-informed commenter on Bruce Schneier’s security blog notes (12/16/2017):
... most Internet vulnerabilities at the protocol and standards layers have been there since day one. Because they were quite deliberately built in from day zero.


It was almost certainly not done maliciously but to “solve problems within resource constraints” that no longer apply.

The thing is nobody wants to spend money to solve these problems, usually portrayed with the excuse of “don’t break legacy systems” as it’s the almost perfect “get out clause”. As well as the biggest soirce [sic] of not just technical debt but building in security vulnerabilities across the board.

https://www.counterpunch.org/2018/01/04/the-nets-good-old-boys-hacking-the-arpanet/

As always, Clive, your writing speaks for itself.

GarboJanuary 4, 2018 2:23 PM

"Because they were quite deliberately built in from day zero."

Without a source, citation, eyewitness, explanation, or specifics... you can say that, but is it so?

You can say they existed since day zero much more provably. Saying each were deliberate requires proof.

65535January 4, 2018 6:58 PM

@ Grauhut

[from multiple post - sorry was so late responding]

https://www.schneier.com/blog/archives/2017/12/friday_squid_bl_607.html#c6766825

and

https://www.schneier.com/blog/archives/2017/12/friday_squid_bl_606.html#c6766836
"ssl is --transport-- security and your browser "strips ssl" in the moment you read a page here... ;)"

I know that much. The problem of transparent SSL Stripping before I read the screen is a different story all together.

"Nowadays they try to snipe maleware and their mostly js based droppers before they reach your browser or mail application in order to get them out of the line of fire by putting proxies with sandboxes in front of them. These proxies are then the new, we all hope harder, attack surface, but the price for this is a benevolent mitm attack, these proxies need you to trust them."-Grauhut

Huh, "benevolent mitm attack" that is nice way of saying your AV reads your "Secure" communications - and as Bruce S has proven most AV vendors refuse to answer the question of "do you turn a blind eye NSA/CIA/FBI/and so on malware placed on a non-air gaped machine. This sounds like an increase in attack surface area.

This also relates to Kaspersky ex-filtrating NSA documents which ended up on their servers. I would hardly call that "benevolent" or any other nice term.

You really did not address the over-write issuer of root certificate stores in both the browser and OS. That "over-write of certificates" is very problematic and I don't really think it is benevolent. Do you think over-writing certificates is good?

Last is the AV having a "White List" of say 600 banks which supposedly it doesn't strip. As I have indicated that is trivial changed or completely removed revealing all banking transactions to the AV vendor and possible. That is not so good.

This is true when an individual reporter or dissident come to the attention of the TLAs in various governments including the IRS which considers you guilty until proven innocent... and like Iran and so on.

http://www.ilook-forensics.org/

"Don't use av software from vendors you don't trust."

From you post:

https://www.schneier.com/blog/archives/2017/12/friday_squid_bl_606.html#c6766836

What AV vendor do You Trust windows shops. Can you give us some names?

gordoJanuary 4, 2018 7:33 PM

@ Garbo,

Sure, some of it may well have been happenstance. Yet, in a tautological sort of way, if not to a ⊤, "the proof of the pudding is in the eating".

Clive RobinsonJanuary 5, 2018 2:15 PM

@ gordo,

As always, Clive, your writing speaks for itself.

Including the spelling mistakes and "fat fingerisms" ;-)

@ Garbo,

Without a source, citation, eyewitness, explanation, or specifics... you can say that, but is it so?

First of all @gordo was quoting me, and the answer is yes we can say they were built in from day zero.

I can give you the who trying to keep awake in standards meetings chat if you want, but it boils down to this,

You need to compare the ISO ISO and the X protocols with the DOD ARPA protocols, that were actually based on the work of Gordon Welchman who was one of the "Bletchly Brains" and easily the equivalent of Alan Turing.

The ISO standards were designed to be scalable reliable interoperable and all sorts of other good things we want these days. The DOD ARPA protocols were designed pragmatically to work within the constraints of the available hardware at the time (but only just). The result the DOD protocols happened, and the ISO protocols mainly did not.

But you also have to understand that all the Telecom Standards Boards are staffed by "insiders" from "industry" and MIC etc. They all know or their bosses know people from the IC especially the SigInt agencies. Thus the standards "get the fix put in". It provides entertainment for those who know the game and an opportunity to "kick t'b'stards back".

The game is usually played by derailing any move towards security onto "safety" and then framing it in "think of those poor people, lost / alone / ill in need of help" yup they can lay it on with a trowel or JCB if necessary. It's called "Finessing" and it has a numbers asspect to it. Say you as an individual try to throw a spanner in their nasty little works of evil... Well most of the international spooks know the other Five Eye spooks will back them and they will vote you down. If you push it then a word will be dropped in your ear, if that fails your bosses ear and so on. And if you tell them to their face they are a bunch of IC lickspital lackies etc then the gloves will come of in other directions.

That's the game, they play to win and the only way you can draw is to not just name and shame the individuals but get the academic community to start providing embarrassment. Which is kind of what happened with NIST and why they withdrew a standard...

As a US president observed "The price of freedom is etrnal vigilance" and the IC and SigInt agencies do not stop and do not give up, there is way to much tax money to ensure keeps flowing their way. As I said they play to win, and you can only hope to draw.

gdJanuary 5, 2018 5:02 PM

I'm the guy @ Gordo quoted quoting @ Clive Robinson. I surely don't have the credentials that most of you blokes do, but I've been wailing about abuses of power in the tech community and government a long time, both at CounterPunch and on my blog. SoS is my go-to site for clearing what is known about digital misdeeds and really appreciate being able to lurk here and occasionally pipe up. I especially appreciate Clive's insights and hope he doesn't mind the publicity I have afforded him. As I said in a little post today, "If you’re curious about what security experts say to each other over beers, read the comments his followers have been leaving [on Bruce's blog]." Cheers, and keep up the good (even if ultimately futile) fight.

ChrisJanuary 5, 2018 7:26 PM

Hi thanks for a good continuing blog!

I really enjoy reading this blog, keep up the good work.

I just wanted to show you what I started to test since december 13 as a test
i thought it would not work in real life but today i use it in all my computers.
And i am VERY happy how it works. If you have time 3 hours implement it.

So what am i talking about is whitelist approach
At the moment since its complicated and time consuming to do it in the firewall
i do it only on DNS level, i wanted to do a POC and se if it could work.
Well it actually took only about 3 hours to get 95% of the stuff working
the fine tuning is the rest of the time and every now and again i find something
new that i forgot about that had stopped working that i didnt use that ofteh.

So its a continuous approvement approach, at this time only on DNS level
The basic idea is this:
1.) network interface dns > 127.0.0.1
2.) dnsmasq listen on 127.0.0.1
3.) dnsmasq servers point to tor dns and dnscrypt
4.) port 53 block on outgoing firewall
5.) dnsmasq configuration with whitelist towards tordns and dnscrypt if domain name is in whitelist.

Just a short sample of my DNSMASQ file, it has around 50 domain names today
all the rest is going to address 9.9.9.9
that is rejected in outgoing firewall.
For simple way to find what to whitelist
tail -f /var/log/dnsmasq.log |grep 9.9.9.9

I hope someone find this useful
//Chris
---

# Needed to do groupadd -r dnsmasq since the group didnt exist
# Apparmor for dnsmasq needs to be disabled for now, and ev touch /var/log/dnsmasq.log and chown dnsmasq:dnsmasq /var/log/dnsmasq.log
# http://www.g-loaded.eu/2010/09/18/caching-nameserver-using-dnsmasq/
bind-interfaces
bogus-priv
cache-size=4096
dns-forward-max=300
domain-needed
group=dnsmasq
listen-address=127.0.0.1
local-ttl=3600
log-async=5
log-facility=/var/log/dnsmasq.log
log-queries
neg-ttl=80000
#no-negcache
no-poll
no-resolv
pid-file=/var/run/dnsmasq/dnsmasq.pid
proxy-dnssec
user=dnsmasq

# Whitelist Domains to DNS Lookup
# If not in the list send to 9.9.9.9
# Implemented 2017-12-13

# ACESTREAM
server=/linux/127.0.0.1
server=/acestream.net/127.0.0.2
server=/acestream.net/127.0.2.1
server=/dht.transmissionbt.com/127.0.0.2
server=/dht.transmissionbt.com/127.0.2.1
server=/router.bittorrent.com/127.0.0.2
server=/router.bittorrent.com/127.0.2.1
server=/torrentstream.info/127.0.0.2
server=/torrentstream.info/127.0.2.1
server=/torrentstream.net/127.0.0.2
server=/torrentstream.net/127.0.2.1
server=/torrentstream.org/127.0.0.2
server=/torrentstream.org/127.0.2.1
server=/tracler.coppersurfer.tk/127.0.2.1
server=/tracker.leechers-paradise.org/127.0.2.1

# Block The Rest
address=/#/9.9.9.9

Clive RobinsonJanuary 5, 2018 10:01 PM

@ gd,

I especially appreciate Clive's insights and hope he doesn't mind the publicity I have afforded him.

As our some of our antipodean colleagues say "No Worries".

The important thing as I said is the message not the messenger, as long as it gets out as wide as possible, hopefully people will learn from it, and "whilst climbing up on my shoulders reach up to do better things still".

Although my shoulders may be getting old, they are still broad, just as long as they don't pull my beard on the way up, I need it to keep my chest warm ;-)

Leave a comment

Allowed HTML: <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre>

Photo of Bruce Schneier by Per Ervland.

Schneier on Security is a personal website. Opinions expressed are not necessarily those of IBM Resilient.