NASA is researching new techniques for multiplexing SQUIDs—that’s superconducting quantum interference devices—for X-ray observatories.
As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.
Read my blog posting guidelines here.
Ted • July 1, 2022 10:25 PM
So Google is going to start deleting some sensitive location data, including for places like abortion clinics? Good decision.
Hopefully they’re tidying up the attack surface with a bevy of analysts and lawyers. We’ll see. When you collect HUGE amounts of user data, it’s probably good to avoid getting dragged into a fiery spotlight, particularly in connection to issues like these.
https://www.washingtonpost.com/technology/2022/07/01/google-abortion/
Also you can delete your menstruation logs in Fitbit, if you had been tracking them there.
https://blog.google/technology/safety-security/protecting-peoples-privacy-on-health-topics/
Ted • July 1, 2022 10:37 PM
Per Daniel J. Bernstein:
NIST now says it plans to announce its selections of post-quantum algorithms on “Tuesday, July 5th” (I presume 2022, not 2033)…
https://twitter.com/hashbreaker/status/1543034151544049664
I haven’t yet looked for an official source on this.
MarkH • July 1, 2022 10:44 PM
Because so much of the work of state intelligence agencies is kept secret, it’s of special interest to intelligence watchers when facets of operations rise to public view.
For the state of Israel’s intelligence apparatus, Iran is famously target number one. The tempo of successful operations against Iran has been rising recently. In addition to traditional recruitment of sources of sensitive data within Iran, actions deemed likely or very likely to have originated from Israel include cyberattacks against military/industrial infrastructure and attempted (or successful) assassinations of key Iranian personnel.
(1st of 2 parts)
MarkH • July 1, 2022 10:45 PM
part 2:
These setbacks for Iran might have motivated the recent dismissal of the intelligence chief of the Revolutionary Guards Corps, a key Iranian security organ.
It’s also reported that a General of the Corps was arrested on charges of spying for Israel. One news story about this situation:
https://www.nytimes.com/2022/06/29/world/middleeast/israel-iran-spy-chief.html
Interestingly, Israel is believed to have brought public attention to these blows against Iran (without claiming credit for them), in hopes of fomenting conflict and turmoil within Iran’s power structure. If so, this tactic may be working.
Tom Ace • July 1, 2022 11:26 PM
On June 28, @BrendanCarrFCC tweeted
TikTok is not just another video app. That’s the sheep’s clothing. It harvests swaths of sensitive data that new reports show are being accessed in Beijing. I’ve called on @Apple & @Google to remove TikTok from their app stores for its pattern of surreptitious data practices.
and posted images of the letter he sent to the CEOs of those two companies. If they won’t remove TikTok from their app stores, he wants them to explain by July 8, 2022 why they believe it doesn’t run afoul of their policies.
SpaceLifeForm • July 2, 2022 12:21 AM
A funny Observation
It’s really easy to miss stuff if you are looking elsewhere.
‘https://nitter.net/matthew_d_green/status/1542658072568545281#m
Also: I’ve been reading ASCII since I was a little kid programming BASIC and I never realized the difference between upper and lowercase letters was a single bit. I bet that saved some mainframe programmer loads of CPU.
Sorry, Matthew, but you made me LOL. Twice for different reasons.
Clive Robinson • July 2, 2022 2:22 AM
@ Ted, ALL,
So Google is going to start deleting some sensitive location data, including for places like abortion clinics? Good decision.
No, very bad decision.
1, It’s not solving the underlying problem of psychopath behaviours.
2, It’s playing into the hands of those who “terrorise” in both the old and new meanings.
Sorry but nothing but longterm harm will come of it.
History shows without any doubt trying to keep any information secret eventually fails. The result of trying to keep things secret builds up a tsunami of tyranical systrms that get abused to oppress people to gain power and control over them.
You will always find it boils down to them saying,
“For the common good”
Whilst what they realy mean is,
“Might is right”
Which logically flips to,
“You are powerless so wrong”.
SpaceLifeForm • July 2, 2022 3:08 AM
I can not disagree
https://sfconservancy.org/GiveUpGitHub/
If you have been around a while, you knew that MS would start to EEE it.
You should not be surprised.
Clive Robinson • July 2, 2022 3:48 AM
@ SpaceLifeForm, ALL
Re : A funny Observation
Of Matthew Green “cryptographer at large” and his tweet of,
“I’ve been reading ASCII since I was a little kid programming BASIC and I never realized the difference between upper and lowercase letters was a single bit.”
I wonder if he realised about the easy number conversion….
But more importantly why,
“NUL” = 0x00
“SP” = 0x20
“A” = 0x41
“a” = 0x61
“DEL” = 0x7F
A hint, it was not for much later programing languages for C etc.
As a different hint look up ITA2 Baudot Murry code and you will see a curious legacy.
Baudot’s original code was designed to minimise “hand stretch” and slightly later from a suggestion by Gauss to “balance the code” to maintain DC levrls on the line (which would later have TEMPEST issues as I’ve mentioned before on this blog).
You may remember I mentioned a while back why the electro-mechanical “Relay” got the name it did. That is it made the transmission of messages over longer distances and via crude switching more efficient thus reducing the need for “Relay Operators” who’s job was to receive a message on one circuit and “re-key it” on another circuit.
Also the “galvonometer and smoke tape” recorders got replaced with the equivalent of very early “punch tape”. In a case of cross-fertilization the Baudot system changed. A message would get punched on tape, and then sent at much higher speed down the line. Thus the operator and line became seperated by a tape punch and reader. This also enabled easy resends and even editing of the tapes.
When Murry came along the mechanics had got to the point where operator hand fatigue was not an issue but the new paper tapes were, so he picked a different way. If you
look at the punch tape perforations and remember “A SIN TO ER” being the most frequent letters used in english text. You will find they have the minimum number of holes in the tape. Which makes the tapes less likely to break or tear in use.
Consequently the NUL has no holes punched and the “DEL” –not the backspace– has all holes punched.
So it goes back in part to “things mechanical and hackable” but also have a look at,
https://cryptomuseum.com/crypto/uk/rockex/index.htm
In the lower sections where it explains how the Rockex works as a “One Time Pad/Tape” Super encryptor…
[1] For those who have not memorised the ASCII chart and thus can not “sight read” the HEX or Binary values,
https://www.freecodecamp.org/news/ascii-table-hex-to-ascii-value-character-code-chart-2/
[2] With regards “helping main frames”… Some who do assembler level programing might wonder why Intel made their OpCodes such a pain. Actually they did not. Back in the 1970’s the general view was “multiples of three bits” hence 12, 24, 36 bits wide memory in mainframes which became a legacy in the PIC microcontrolers. So the first Intel CPU chip instruction sets were based on Octal not Hex. This lasted as a legacy into the 8080, thus moved into the Z80. In turn the 8088/86 inherited it from the 8080 and so on… Hence the 80486 instruction set OpCodes should be viewed as Octal not Hex.
Clive Robinson • July 2, 2022 4:14 AM
@ SpaceLifeForm, ALL,
Re : You should not be surprised.
I spoke out at the time Micro$haft aquired Github…
I spoke out at the time Micro$haft aquired MineCraft…
I pointed out history and got told that was old news, old people and now it was all new and shiny and new leaves etc etc etc.
Need I remind people that when things are,
“Lower than a Snakes Sphincter in a Waggon Wheel Rut”
They are unlikely to be “uplifted” or change… In fact when the rains clean up a bit on top, down there they just sink lower in the mire…
As a friend has recently proved Micro$haft only start to behave the way they are legally required to do when you start legal proceadings against the individual officers personally, to take away their homes, bank accounts pensions etc as compensation for their misfesence.
SpaceLifeForm • July 2, 2022 4:36 AM
@ Clive
re: Consequently the NUL has no holes punched and the “DEL” –not the backspace– has all holes punched.
In his defense, I was out of High School before he was born.
So, I doubt he ever dealt with punched paper tape.
Kind of hard to backspace and un-punch holes, right?
Avoid typos. Be careful. Otherwise the tape may break.
MM • July 2, 2022 5:48 AM
Have always wondered why squid?
SpaceLifeForm • July 2, 2022 6:03 AM
@ Clive, ALL
Hole-ly Paper. Let me drop some chad into the bit bucket.
There are two good reasons why Hollerith cards are better than punched paper tape.
First is the stock. The Hollerith card is way less flimsy than punched paper tape. It is way less likely to break thru usage. They will wear out eventually. I know this from experience.
With the Hollerith card, if one breaks, or you spill coffee on it, you will probably be in position to recreate it. And this is important. You will be in position to recreate it by Visual Observation. Assuming you have some blank stock and an IBM-029 handy.
Whereas, paper tape is flimsy. If it breaks, it will be tricky to do a scotch tape repair. Note also, it is a serial byte stream.
So, that brings us to the concept of records.
Hollerith cards are 80 byte records.
Records on Hollerith cards allow you to replace, insert, or delete relatively easily.
Whereas, punched paper tape, being a byte stream, requires that you must recreate the entire tape if you want to make a change.
So, with a 80 byte Hollerith card, if I need to make a simple change, I just create a new card with the change, and put it into the stack of cards, throwing away the old card. Delete or Insert is obvious.
Or is it?
Well, it turns out that it is not that simple. Nothing is.
What if you drop your stack of cards on the floor?
You see, the cards (the 80 byte Hollerith records), must have an order to have meaning.
A whole bunch of consecutive 80 byte records are, in fact, a byte stream. Just like a punched paper tape. But much easier to edit.
But, what happens when you drop your stack of cards on the floor, and they get all mixed up? They are now out of order. Do you panic? Do you have a backup card set?
No. You do not panic. Recovery may be slower than you would like, but if you plan ahead, you can recover from the dropped card deck.
And how did this work you may wonder.
Planning.
Any large card set (source code) would be created with the first 6 characters being numeric. That is the ordering.
Typically, only the first 4 would be used, with the last 2 being zeroes.
That allowed for easy insertion. For example, cards with 234500 and 234600 leaves room for cards to be inserted, for example 234510 and 234520.
You do not need to re-sequence the entire card deck to make edits.
Now, you may wonder, what does this have to do with a dropped card deck?
Well, the beauty of a Hollerith card is that it is both machine readable and human readable at the same time!
Hopefully, when you dropped your card deck, they stayed in groups. And on the cards, you will be able to visually Observe the 6 digit ordering numbers. So, you will be able to re-sort the deck by Observation and hand.
Speaking of card decks. Did you know that if you perfect shuffle a standard 52 card deck 7 times that nothing changes?
JonKnowsNothing • July 2, 2022 6:58 AM
@SpaceLifeForm, @All
re: Did you know that if you perfect shuffle a standard 52 card deck 7 times that nothing changes?
For those who would like to test this:
You will notice that as the colored card moves around within the deck it eventually returns near its initial starting area.
Not a very precise method but easy to check visibly.
JonKnowsNothing • July 2, 2022 7:18 AM
@ Ted, @ Clive, @ ALL
re: Closing the barn door after the fire..
Way before technology and computers, people knew where things were.
Sometimes they had maps, although in some countries maps were considered Military Secrets and possessing one defined you as a spy and you got shot. Still, people knew where things were within an “area”.
While some folks cannot get down the street without a GPS blaring away their travel time, route, mileage and proximate eta, there are still lots of people who can find their way to the burger stand or grocery or clinic without one. We know where things are with in a certain range.
When we didn’t know, the classic “ask that person how to get to …” question evolved into a common joke. Some people were less inclined to ask directions and a great number of detours took place along with passenger discord.
The locations are known and can be discovered because a lot of it is in the public domain; certain tax records, business filings and business licenses can be found (or used to be found) on microfiche at the business license office in the area.
As for the targeted 50% of the population, it’s not too hard to track them either. China did it for a long time with their One Child Policy. Grannies and Aunties assigned to notice things and report on them.
Then there is Bluffdale… They won’t be purging their database anytime in the next century. Their data storage capacity has more than enough space for everything.
Clive Robinson • July 2, 2022 7:54 AM
@ SpaceLifeForm,
Well, the beauty of a Hollerith card is that it is both machine readable and human readable at the same time!
Just as paper tapes were if you put in a little learning…
I used to know both ASCII (7chan) and ITA2 (5chan) well enough back in the 1980’s to read them out.
With BASIC programs you could with ASCII quickly scan down the tape and read the line numbers…
There were times with both types of punch tape, sticky tape and a sharp knife and punch tool I used to do “hand edits”.
Often though I used a back to back cable and two terminals that had both tape readers and punches.
It’s easy to do with “program tapes” and “Five Letter Word Ciphertext tapes” and sometimes a lot faster than you can type for small corrections.
Back in the 1980’s if you wore the green and wanted to be a telegraphist as a trade you had to demonstrate your skills.
If you wanted to rise up in the ComnCen to be a Sgt or Cipher Op then you had to also demonstrate skill with the blue 6 hole later 8 hole tape, used for KeyMat on BID kit… Some of which such as the BID/950 were accepted by NATO and were still knocking around into this century.
http://jproc.ca/crypto/bid950.html
The ability to hand edit punch tape proficiently is most definately the sign of a misspent youth 😉
Mind you I’m in my 7th Decade and not sure how old Matthew Green is so cannot comment if I was programing before he was a twinkle… But it may be that I have some punch paper tapes sitting in tabbaco tins that are older than him… One tin has hand digitized map of the world projected as 1/10th of an inch to the degree that took ages to do along with a program that read it in pushed it out to a ploter of graphics terminal correctly scaled and then plotted a satellite orbit with time marks for a given date and time based on a database of “orbital elements”. That was back in the mid 70’s before “Home Computing” in the UK got going… The funny thing is the rubber bands have long since perished and the ball point ink faded so only the impression on the leader is still just visable… But I’ll be honest with you I still have the urge to “cut and stick” punch tape like I used to… I’d probably slice the end of my finger off these days without a decent magnifing light but hey is it like “riding a bike”…
Ted • July 2, 2022 9:40 AM
@Clive, All
Re: Google deleting sensitive location data
No, very bad decision.
Bold opposition 👍
I think it’s at least good for Google. These aren’t the circumstances I would have wanted for a few more privacy measures. At least Google isn’t oblivious to the risks this ruling poses to them.
JonKnowsNothing • July 2, 2022 10:34 AM
@Ted , @Clive, All
Re: Google + location + data privacy
FYI It’s security theater. It’s not real. It’s not going to change anything. It’s not going to help anyone.
The only value is that if people “think” Google is doing “something” they will forget about the rest of the field that does the same stuff.
Every vendor, every OS, every phone, every app and every street camera, every RING-thing, every cop car, every tow truck, every pharmacy and store with “shrinkage cameras”, every repo-dudette.
The only “good for Google” is they are hoping to piggy back the wave for $$$.
Any state, city or DA can send an official or unofficial request and get the data that they are supposed to be deleting. It’s part of the system.
It is not safe and it will not be safe to think otherwise.
Ted • July 2, 2022 11:58 AM
@JonKnowsNothing, Clive, All
It’s not going to change anything.
I get your sentiment. The ubiquity of surveillance far exceeds that of what it once did. I’m sure it won’t be too long before we start seeing some cases come up. I predict there’s going to be lots of research on how tech and tech policy interface with this issue.
JonKnowsNothing • July 2, 2022 1:15 PM
@ Ted, @ Clive, @All
re: it won’t be too long before we start seeing some cases come up
From what I’ve read, this is already in play.
An interesting side note in the UK:
It seems that terminations were made legal in the UK @1967. An older law making terminations a criminal act was not removed from the books. This older law has been activated 67 times in the last 10 years in England and Wales to bring criminal investigations. Often times the instigator of the claims are health care services and hospitals.
===
ht tps://www.theguardian.c o m/world/2022/jul/02/women-accused-of-abortions-in-england-and-wales-after-miscarriages-and-stillbirths
(url lightly fractured)
lurker • July 2, 2022 1:19 PM
@Clive Robinson
Thank you for the detailed explanation. Mine was too terse to post:
ASCII was designed by typewriter mechanics for typewriter mechanics – think about Murray code.
vas pup • July 2, 2022 4:10 PM
Recent drills with US Navy missile subs show how Marine special operators would get to the frontline in a fight against China
https://news.yahoo.com/recent-drills-us-navy-missile-113000806.html
” Recent US Navy-Marine Corps drills highlight an overlooked method of getting operators on target.
The drills involved Marines launching from submarines, which can be done on the surface or underwater.
These kinds of submarine operations are harder for enemies to detect but come with inherent risks."
Interesting details inside.
vas pup • July 2, 2022 5:01 PM
US Navy, Air Force running ‘capstone test’ of new high-power microwave missile
https://news.yahoo.com/u-navy-air-force-running-110000431.html
“AFRL is also making progress on a more advanced version of its Tactical High Power Operational Responder (THOR), which uses HPM technology to disable drone swarms that pose a threat to military bases. The next-generation platform is named Mjölnir as an homage to the mythical god Thor’s hammer. AFRL awarded Leidos a $26 million contract in February to develop the Mjölnir prototype and deliver it in early 2024.”
SpaceLifeForm • July 2, 2022 5:38 PM
@ Clive
GOSUB 2600
Matthew Green was born in 1976.
We are both about 18 years older.
Thanks for the Rockex link.
Jenifer • July 2, 2022 6:58 PM
So Google is going to start deleting some sensitive location data, including for places like abortion clinics? Good decision.
So… Alice took a long drive, disappeared off the face of the earth near an abortion clinic, then reappeared a couple hours later and did the same drive in reverse. She’d better hope there are some other plausible destinations nearby. And that she was careful with phone records; a state might be able to get intra-state call records (presumably, as with long distance billing, these are regulated differently than inter-state calls).
Jon • July 2, 2022 7:10 PM
@ efficient encoding:
Of course, Morse code (as a serial stream) was designed to use the fewest ‘signal times’ for the most common letters (yet they blew it).
The actual frequency of letters in English, in order from most to least, is
“ETAOIN SHRDLU” (omitting the rest). Given that every symbol must also include a dead time between (or after) symbols (defined as one ‘dit’ time, while a ‘dah’ is defined as three dit times) the letters have the following ‘transmit times’, in ‘dit’s:
E = 2
T = 4
A = 6
O = 12 (they blew that one, dunno why)
I = 4
N = 6
S = 6
H = 8
And so on down the line. Numbers (they blew that one too) are hideously expensive in Morse time: A 0 (zero) is 20 dit times! Not so good for fast remote control!
Have fun, J.
name.withheld.for.obvious.reasons • July 2, 2022 7:49 PM
Homage to the Fourth (or is it forth),
In the United States, with the upcoming 4th of July celebration, one has to think of the lionized history and customs over time that represent the centuries long experience. From the genocide of native peoples, the kidnapping and enslavement of entire populations, and the ceaseless incursions in any place, at anytime, weighted by values harbored with reflexive principals. From the foundational aspirations of a privileged society isolated from the influences of classic Euro-Atlantic power structures (monarchies, theocracies, and oligarchies) and the willingness to assert legitimate self rule, the United States is both unique and ugly.
Whatever the context, legitimacy is a commodity, but its trade is not across the space of monetary exchanges but rather the perceptive compact visible in unwritten social contracts. Civil societies do require some level of acquiescence to those that assert authority, whether deserved for forced.
Today social contracts are being nullified with blinding speed and utter contempt toward those that live within them. These actions are anti-revolutionary and injury with force both the space to negotiate their forms and to exist within them successfully. One could argue that living in a gulag is successful while one has the ability to draw breath, but in relativistic terms, those running the place faire better than those forced to occupy (just cannot come up with the correct word[s] here) it. So think of this 4th of July, you former citizens of a once quasi-democratic republic, for your experiment has ended and your nightmare begins. Have a great 4th of July, y’all. (Sarcasm included for free) You have now come out the other side.
Many here warned of this day, the anti-4th, was coming. Not that it was wanted, expected, or called for. History’s markers are everywhere, and those that can afford to look away will someday be forced to face the reality they so desperately wish to avoid. Nothing I or anyone else can say speaks to moments like this. Why, because the time to answer history’s cries have passed and only the tears of the present are left. Eventually, as the entropy and decay of overt political crimes wears down the fabric of society and the remaining prisoners of conscience realize they have nothing to lose, will those with the courage and the stamina to take power to task will a more just and civil society be possible.
SpaceLifeForm • July 2, 2022 8:09 PM
@ Jon, Clive
re: efficient encoding
For What It’s Worth
I’m pretty sure that the reason the encoding was designed that way, was to make it easier to LISTEN TO.
Stop, Hey What’s That Sound
There’s Something Going Down
name.withheld.for.obvious.reasons • July 2, 2022 8:13 PM
@ Clive,
Again, another return trip to my youth. Telex and 60 baud serial interfaces to mainframe computer allowed a programmer with balls covered in magnetic tape to take on the punch tape pros. Don’t get me wrong, I’ve done both. Programming in Fortran on cards, the language designed to do so, and using tape to store larger programs was almost fun. Even worked in the Hollerith building in LA not too long ago (a decade+).
In 1980, at home writing an HVAC application, spent five+ hours at the keyboard banging away at code (Z80 MASM) that literally was an expression of what was already living in my head–oops the power went out. Having not the foresight and temperament to periodically write this code to tape (the 7 lines of code and the time taken for the tape to home, find the EOT, and write the data (300 baud) was an imposition I could not tolerate), after the power returned and the code was literally still in the front of my brain, quickly typed away for another four hours, boom, the power went out again. My reasoning at the time went along the lines of ‘The World According to Garp’ and the likelihood of another plane plunging into a house (forgetting that if you live near an airport with a runway that parallels or intersects your location–odds are…). Oh to be so naive again–what an opulent life.
Ted • July 2, 2022 8:36 PM
@Jenifer, JonKnowsNothing, All
So… Alice took a long drive, disappeared off the face of the earth near an abortion clinic, then reappeared a couple hours later and did the same drive in reverse.
I thought of the same thing. In a letter that around 40 Congress members sent to Google in May, they mentioned dragnet geofencing warrants.
…according to data published by Google, one quarter of the law enforcement orders that your company receives each year are for these dragnet geofence orders; Google received 11,554 geofence warrants in 2020.
Maybe Google’s announcement was partially in response to that. I don’t know how Google would respond to one-off warrants for an individual’s location data. I hadn’t thought about call records though.
Re: State laws
Hopefully we’ll see more resources roll out so that people don’t have to guess how they might be affected.
name.withheld.for.obvious.reasons • July 2, 2022 8:38 PM
Always fun to make Fortran continuation statements or edit them. With 72 columns and the assumptive 7 columns of line numbering one could be quite presumptive that 10 million cards of any code (assembly or high level language) was enough headspace well into the future. Of course electromagnetic storage won out–but the longevity of punch cards exceeds that of almost all forms of EM based storage systems (given proper care).
I would be interested in seeing an archeologist finding a petrified punch card and puzzling over its meaning. But I guess that’s not uncommon, humanity seems to puzzle over much and understand little of anything.
SpaceLifeForm • July 2, 2022 8:57 PM
@ name.withheld.for.obvious.reasons
re: the code was literally still in the front of my brain, quickly typed away for another four hours
Sometimes, I would agree with your definition of quickly.
Just points out, that when you are on a coding roll, document it. Key it in. Do not document it via Morse over Shortwave.
name.withheld.for.obvious.reasons • July 2, 2022 8:59 PM
@MarkH
Interestingly, Israel is believed to have brought public attention to these blows against Iran (without claiming credit for them), in hopes of fomenting conflict and turmoil within Iran’s power structure. If so, this tactic may be working.
Color me cynical, Israeli relations consist of one overriding principal, and it is not about negotiated settlements or treaties. When unilateral assertion to authority and righteousness leave the room and make space for peace accords, let me know. Until then, the state of Israel and so many other nation states need to rethink the solution brought about by overt militarism. It is what is happening in Ukraine, Yemen, the Sudan, and more than a half dozen other countries in the world. The might makes right crowd believes in its legitimacy and righteousness (no matter who profits from it).
It is tiring to watch elite clashes of unconsciousness wield its people on and to a battlefield and both think nothing of it and pay nothing for it. Maybe humanity should be marked down, sold on e-bay, and then put in someone’s garage.
Ted • July 2, 2022 9:13 PM
@MarkH
Nice coverage of Israel-Iran.
Did you see the spark’s flying at an Iranian steel mill due to a purported cyberattack?
Just the video:
https://twitter.com/gonjeshkedarand/status/1541288345183158272
More of the story:
https://www.cyberscoop.com/iran-cyberattack-israel-hacktivist-steel-ics/
SpaceLifeForm • July 2, 2022 10:40 PM
@ Ted
‘https://groups.google.com/a/list.nist.gov/g/pqc-forum/c/7yLIZcFOMF0
BTW, thanks for digging into the e-voting DDoS issues.
At least the researchers brought up DNS and BGP.
MarkH • July 3, 2022 1:19 AM
@Jon:
etaoinshrdlu was Samuel Morse’s frequency ranking of letters in U.S. written English in the Victorian era.
Some modern frequency rankings are slightly different, but run very close to that.
Morse’s work was “frozen in” to the design of the Mergenthaler Linotype machine’s keyboard; for mechanical reasons, it was more efficient to cluster frequently used letters.
It’s the way I always remember letter frequencies …
Clive Robinson • July 3, 2022 4:29 AM
@ Jon, MarkH, ALL,
The story behind the letter frequency used by Samuel Morse is,
“One day in a printers shop, he noted the positions of the letter bins…”
Now I have no idea if it is true or false but things to note,
1, Language use in printing is not normal.
2, Some leter comninations are made into one like “Th” and “oe”
3, The positioning of the bins would be a mixture of usage frequency, alphabet order and hand distance.
Back in Morse’s era the printing of short run “hand bills” was common as “typewriters” (people) were more expensive to employ than “printers”.
The language of the “hand bill” or “adverts” was then, and still is today not, “spoken english” nor is it “written english” in any of a myriad of forms and knowledge domains.
But importantly, sometimes we don’t realise just how fast change happens…
It was not untill the work of Claud Shannon during WWII became more widely known, and “Information theory” started less than a life time ago in the 1960’s, that the likes of American mathematician Richard Hamming came up with ways to not just measure, “information content” but build tools that have given us “coding theory” that is behind most of modern communications, in the “physical layer”.
A place way down in the stack these days, where most even technically minded do not ever see or even realy know about. Or importantlt need to know/see because it is now a “done deal” that just works out of sight out of mind thanks to API’s and the like.
The likes of Reed-Solomon Codes, CRC Codes, and most other Error Detection, and “Error Correction Codes”(ECC), or even such simple ideas as parity in more than one dimension, or “Forward Error Correction”(FEC) are just words without real meaning to most these days.
But it was not always so, back in the early 1980’s a friend had a very nice Polish Girlfriend who was over in the UK doing her PhD at a well known UK University. Technically it was under the “maths” Dept, but as she and I discussed one day it realy should have been under “applied math, as relating to engineering” as her work was on coding theory and I was one of the few people in her social group who could at the time understand what her work was and why it was so important.
Now we just take it for granted but still without realy understanding it. Which is sad in a way because some of that work has made it into “Post Quantum Cryptography” yet it is largely unknown.
History tells us that the strength of “Renaissance Man” was not some fantastic ability beyond imagination. No it was simply that they were not men of a single art or domain. What they brought to the party for our benifit even today was the ability to take knowledge from one domain and apply it to another domain, thus bringing other domains up quickly to the leading edges of understanding. We appear to be loosing that cross domain ability to politics and power play games and almost religion by cult behaviours. Knowledge nolonger freely flows and is almost jealousy guarded in formalized heirarchics in academic knowledge domains, where division gives some power that is detrimental.
Ted • July 3, 2022 6:55 AM
@SpaceLifeForm
Sure thing! Oh, there was the update from NIST‘s Dustin Moody. Fingers crossed for updates on PQC this week! Thanks so much! 😀
MarkH • July 3, 2022 11:43 AM
@Jon, Clive:
I sit corrected … apparently what Morse did was find out the counts in manufactured type sets, perhaps by looking at a catalog.
The quantities for each letter were very much rounded, so there were numerous “ties” (letters in equal quantities).
However, the ordering in etaoinshrdlu resolves those ties quite accurately in frequency order, so that specific ordering probably resulted from some diligent statistical work, I don’t know by whom.
I guess you have to be really old to understand punched cards. The usual method of ensuring that you could re-sort the card deck if you dropped it waas to take a marker and draw a diagonal line across the card tops. Then you used that to reorder the deck if it became unordered.
There is another reason that Fortran (e.g.) only used 72 of the 80 card columns. It was not so that the last columns could carry a serial number. It was because the console card reader on an IBM 70x (704, 7040, 709, 7090) read cards by rows, not colums. The word length on those machines was 36 bits, and each row read into 2 consecutive words. The last 8 columns on the card were ignored and not read.
Now look up Hollerith coding of cards, and compare it with EBCDIC…
Clive Robinson • July 3, 2022 12:27 PM
@ MarkH, Jon, ALL,
Re : the ordering in etaoinshrdlu
The “English” english letter frequency is slightly different.
I remember it as,
1, “Eat on Irish Lid”
2, Remove “Two Right eyes” (duplicates)
3, Swap second from ends to the right.
So
1, So “eatonirishlid”
2, becomes “eatonirshld”
3, then “etaonirshdl”
Oh for those looking to break suspected “Caesar Ciphers” in English just look for, first the three letter “rst hump”, then the “ae gap”. It works most times and if one or neither is there suspect it is not a true Caesar Cipher, or the language is not English (so do the more labourious letter frequency to language frequency tests next).
Clive Robinson • July 3, 2022 1:30 PM
@ MK, ALL,
The word length on those machines was 36 bits, and each row read into 2 consecutive words.
Yes a multiple of “three bits not four” which was the norm back then.
Modern ICT people are so used to “Hexadecimal, nibbles, and bytes”, when they first come up against that in *nix systems with Octal and three bit file permissions and other 12bit fields (inherited from GCOS) they get thrown a bit…
Also those crossing the ISA – High Level Language divide to do assembler level programming, they often do not realise that 3bit / Octal is important to know and understand especially when IAx86 is involved and it’s inheritence from the 1970’s…
As you say,
I guess you have to be really old to understand
But it is still within living memory 😉
&ers • July 3, 2022 2:38 PM
@Clive @SpaceLifeForm @MarkH @ALL
ASCII is actually full of logic, it was designed
in that way.
with simple operation you can:
And Russian KOI8-R was designed also having ASCII letter
location in mind – you strip high bit and Russian text gets
readable!
hxxps://en.wikipedia.org/wiki/KOI8-R
&ers • July 3, 2022 2:54 PM
@Clive @SpaceLifeForm @MarkH @ALL
BTW, one of the best ASCII references. Luckily
archive.org captured it.
hxxps://web.archive.org/web/20180211083655/http://nemesis.lonestar.org/reference/telecom/codes/ascii.html
lurker • July 3, 2022 3:45 PM
British Army reported to be on the defensive following attacks on their Twitter and YT accounts. The Red-top rags are having a feast on it, much speculation on who, how, why. Watch this space.
Clive Robinson • July 3, 2022 4:32 PM
@ &ers, SpaceLifeForm, MarkH, ALL,
Re : http://nemesis.lonestar.org
First of it id funny that the copyright “do not copy” warnings have been ignored…
But the page contains a number of inaccuracies, and lack of knowledge.
First of Baudot-Murry ITA2 “5bit” is still very much in use around the world and can be found inuse even in some satellite traffic for anoungst other things maratime use. It was certainly was still in fairy solid commercial use into the 1980’s when I wrote “Telex-pump Software” just across from the Brompton Hospital in Chelsea a stones throw from the Town Hall where quite a few “celebs” got married. I was working for AT&T and I still have the gold tie pin and awful “flying postman” ties they gave me…
But missing is the early 1960’s stuff to do with Cryptography and electro-mechanical cipher systems that got pulled from WWII and later 5bit telex radio links (RTTY) used for high level diplomatic and military traffic. Interestingly it was not untill the late 1980’s that 8bit ASCII even started making it into NATO crypto kit to replace 5bit codes. For my sins I still have a set of “Trend Keys” for their various TEMPEST and Crypto terminals.
Yup there is much that the author of that page does not appear aware of…
Oh and yes I still use Maritime RTTY services out to 1000kM daily on the MF Maritime band and HF bands on a weekly schedual depending on traffic.
Yes more advanced systems such as PACTOR span the globe, at higher data rates, but the operators want serious money and well the service does not work as well as you might like.
People are looking at the likes of VaraHF as a more effective alternative and some point to point are using the likes of modified JS8 etc to send “Engineering Order Wire”(EOW) type traffic to note which bands are open so that “Automatic Link Establishment”(ALE) systems with their downsides can be avoided.
&ers • July 3, 2022 5:03 PM
@Clive
Maybe you are interested : here is Estonian
HAM history in pictures.
hxxp://www.ambur.ee/ESajalugu.pdf
Sorry, text is in Estonian, but you can browse the
images and there’s a lot of callsigns visible, maybe
you recognize some you have had QSO with 😉
73!
vas pup • July 3, 2022 5:38 PM
Body Language Misreads
https://www.psychologytoday.com/us/blog/in-it-together/202206/body-language-misreads
“Deception
Being able to spot deception is probably the most important skill gained from knowledge of body language. Most of us have an instinct that something isn’t quite right when someone is telling a lie or is withholding part of the truth. Those instincts are often correct, but it can be helpful to have some way to judge the chances that we are being deceived. There are many behavioral signs of deception, only a few of which are the following:
Frequency of eye blinking, compared to the norm for that person
Pressing together of lips, as if to stop words from coming out
Showing a number of self-soothing behaviors, which could be used to ease anxiety about deception
Not answering the actual specific question, but rather addressing a more general question
Promoting oneself as special, as if building a resume of character to prove one’s honesty.
A key point with all of these signs is that any particular one should not be taken alone as a sign of deceit; they are only meaningful when they occur in !!!!clusters. Any one behavior could easily have another explanation. For example, someone who has dry eyes might blink very frequently. A person who is generally anxious and yet honest might do a lot of self-soothing, such as massaging their own hands or shoulders. However, when several different behaviors are seen together, the odds increase that the person is being deceptive.”
Plus compare with behavioral base line.
&ers • July 3, 2022 6:12 PM
@ALL
In light of Tik Tok :
hxxps://www.datel.eu/en/news/62-huawei-as-datel-and-hitsa-partner-to-modernize-schools-in-estonia
Those devices are now in place, almost in 150 schools.
Devices are:
Firewalls: Huawei USG6300 series
Switches: Huawei S5720-L series
Wifi AP’s: Huawei AP6050DN inside
Wifi AP’s: Huawei AP8050DN outside
Everything is on central management, it’s forbidden to
operate them without central management.
This is BAD.
SpaceLifeForm • July 3, 2022 6:30 PM
@ &ers, ALL
re: ASCII is actually full of logic, it was designed in that way.
As was Baudot, Morse, and EBCDIC.
You will hear nothing . . .
For What It’s Worth
^g ^g ^g
␇ ␇ ␇
Stop, Hey What’s That Sound
There’s Something Going Down
&ers • July 3, 2022 7:17 PM
@ALL
Nevertheless ASCII history is interesting.
How they chose the locations for certain characters.
Page 233 / section 13 and UP.
hxxps://textfiles.meulie.net/bitsaved/Books/Mackenzie_CodedCharSets.pdf
SpaceLifeForm • July 3, 2022 11:37 PM
@ Ted
My reading on the NIST PQC announcement.
My hunch is that they will recommend SABER and FALCON, for encryption and signing respectively. But I can see my hunch being completely wrong.
Also, I would not count on the timing of the announcement.
‘https://www.esat.kuleuven.be/cosic/pqcrypto/saber/
‘https://falcon-sign.info/
SpaceLifeForm • July 4, 2022 3:30 AM
QWERTY and typewriter arms
You all know what that is about, right? 🙂
And then, things became easier. So you might believe.
Pure manual under finger power, or use the magic electric powered golf ball?
You decide which is more secure for sensitive information.
Here’s some interesting bits to think about.
‘https://www.cryptomuseum.com/covert/bugs/selectric/
SpaceLifeForm • July 4, 2022 3:46 AM
QWERTY and typewriter arms
Of course, I left out a link.
I’ll blame that on my typewriter ribbon drying out.
https://www.schneier.com/blog/archives/2015/10/soviet_spying_o.html
Clive Robinson • July 4, 2022 7:16 AM
@ SpaceLifeForm,
Re : SABER and FALCON,
And
“… for encryption and signing respectively”
SABER is not an encryption algorithm, but a “Key Encapsulation Mechanism”(KEM) for semi-secure “Key EXchange”(KEX).
It’s based on the outgrowth and generalisation of
“The Parity learning problem”
Into
“Learning With Errors”(LWE) ~2005
Which further becomes,
“Learning With Rounding”(LWR) ~2012
Problem. LWE was if memory serves correctly originated by Oded Regev around 2005 when he showed that the LWE problem is atleast as hard to solve as several worst-case lattice problems…
In SABER the LWR lattice is kind of used as a combined strong “one way function” and specialised function to get around many padding function deficiences (if you need a 128bit symetric key, but you need say +8kBits to get your key exchange security margin you can see there is going to be issues without a lot of care).
I can sort of explain it but without the background it quickly turns into a lot of what sounds like “arm waving verbosity”.
The thing is, it’s not necessary, for symetric key transfer, but it sure makes it one heck of a lot easier, with accepted built in proofs against QComp attacks. In effect it is a handy “building brick” towards a post QC encryption system.
But it does not do the encryption. However as a Fiestel round used in almost all block encryption algorirhms needs a OWF in theory you could build the LWR into such a round in some interesting ways.
&ers • July 4, 2022 4:03 PM
Ah, yet another case of the disappeared post.
Seems like no-one cares to fix the bugs here.
OK.
Anonymous • July 4, 2022 5:09 PM
Is this thing working?
Anonyrat • July 4, 2022 5:14 PM
Nope!
Anonymous • July 4, 2022 5:39 PM
I am seeing issues also.
Anonymous • July 4, 2022 5:56 PM
I think I found a work-around.
We are Anonymous!
Appropriate timing, no?
Clive Robinson • July 4, 2022 5:58 PM
@ &ers, Anonymous, Anonyrat
Re : another case of the disappeared post
It might help if folks gave a few more details.
You might have seen that @SpaceLifeForm, one or two others and myself have tried to,
1, Find ways to characterize the problems.
2, Make suggestions on how to avoid problems.
3, Work arounds when problems do occur.
Giving details helps with stages 1&2 which leads to recomendations for 3.
So to answer,
Seems like no-one cares to fix the bugs here.
Some of us users do care and want to mitigate issues with work arounds etc.
Anonymous • July 4, 2022 6:08 PM
@ Clive, Anonyrat, Anonymous, Anonymous
The problem may have magically disappeared in the past hour.
We shall see.
SpaceLifeForm • July 4, 2022 9:06 PM
‘https://scottarc.blog/2022/07/04/police-cyberalarm-uses-alarming-cryptography/
SpaceLifeForm • July 4, 2022 9:59 PM
@ Clive, Anonyrat, &ers
It sure looks like a Network Heisenbug.
As I was trying to gather information, the problem sure seemed to disappear.
Did you try turning it off and back on again?
My hunch is that insufficent information was collected to debug the problem, so I expect it to occur again at some point.
It has the smell of a Memory Leak.
Something useful could be to restart all of the server software every night, say at 03:00, and then monitor the siuation. I suspect it will then appear to be fixed, but in reality, it is just hiding the problem.
If that is the case, then the next steps would be selective restarts of server components, and eventually figuring out which component is the culprit.
I do not like either approach.
To properly debug this will be human resource intensive. It will require an on-call network engineer to be in position to watch the internal traffic when the problem occurs, with an outside user being able to recreate the problem. Such was certainly not the case this being July 4th Holiday weekend.
It sure smells like a Memory Leak.
I doubt it is in nginx or php, my first suspect would be MariaDB.
Someone at Pressable needs to carefully study the server logs from the past 48 hours. There may a smoking gun in there.
If nothing shows up, maybe the level of logging needs to be bumped.
This is not rocket science. It just takes knowledge and effort.
SpaceLifeForm • July 4, 2022 11:08 PM
@ Clive, Anonyrat, &ers
Between
and
I had 3 posts go to the Held for Moderation bit bucket.
None had a link, no special keywords.
I changed handles, No Joy.
The Anonymous posts were me, testing, trying to collect information. I was not Anonyrat. Obviously, they made it.
But, in that time range, there were definite problems happening. So, that would be a good timestamp window to concentrate upon when reviewing logs.
Somewhere in that timeframe, either someone manually forced a restart, or the process finally crashed and was auto-restarted. I suspect the latter because of July 4.
Magic, Memory Leak problem gone. For now.
The more I think about this, it really does smell like a Memory Leak.
So, when you see the ‘Held for Moderation’, just change that to read: ‘An internal error has occurred’.
Which is why they never escape the horizon. They never made it to the DataBase in the first place.
The actual error is hidden from your Observation.
There is a software component somewhere in the stack that just bails out upon any error.
This is likely related to the 429.
I could probably clean this mess up in a year. Or maybe a month. Well, at least figure out where the bugs are.
If it is a Memory Leak issue, that may not be quickly fixable.
Anonymous • July 4, 2022 11:53 PM
There’s something happening here
What it is, is not exactly clear
Anonymous • July 5, 2022 12:06 AM
anders
Had same experience regarding the school net kit as a comment to you. No links, short and simple.
They are on to us, they know we are in school.
There are bad teachers in the University of eh eye.
Omitting certain keywords in the hope I get a passing grade.
I think you will get my drift.
[This is one of my posts that went to the bit bucket. I just happened to have it in a tab still. I think you can see there was no reason for it to be ‘Held for Moderation’]
Anonymous • July 5, 2022 12:07 AM
It is a mess.
lurker • July 5, 2022 12:29 AM
I guess the BOFH could tell a tale or two about strange goings on in the server room on a holiday weekend.
Anonymous • July 5, 2022 12:30 AM
What is this eh i stuff eye here about?
Winter • July 5, 2022 2:53 AM
How the world has changed:
From ICM 2022 | Not Even Wrong (read to the end)
ht-tps://www.math.columbia.edu/~woit/wordpress/?p=12960
The 2022 ICM is starting soon, in a virtual version organized after the cancellation of the original version supposed to be hosted in St. Petersburg (for how that happened, see here). The IMU General Assembly is now going on, moved from St. Petersburg to Helsinki. One decision already made there was that the 2026 ICM will be hosted by the US in Philadelphia. With the 2022 experience in mind, hopefully the IMU will for next time have prepared a plan for what to do in case they again end up having a host country with a collapsed democracy being run by a dangerous autocrat.
Clive Robinson • July 5, 2022 5:36 AM
@ Bruce, ALL,
Re : PR, Press, and Research don’t mix.
I’m still chasing this story down but…
It’s gone from a paper Nature should not have published, to nonsense about dark matter and new quantum devices for room temprature Quantum Computing…
https://www.hpcwire.com/2022/06/08/newly-observed-higgs-mode-holds-promise-in-quantum-computing/
First off the original paper has nothing to do with the Higgs Field,
https://www.nature.com/articles/s41586-022-04746-6
(Pre-Print https://arxiv.org/abs/2112.02454 )
There is a mention in the first few lines of the papers abstract,
“The observation of the Higgs boson solidified the standard model of particle physics. However, explanations of anomalies (e.g. dark matter) rely on further symmetry breaking calling for an undiscovered axial Higgs mode. In condensed matter the Higgs was seen in magnetic, superconducting and charge density wave(CDW) systems.”
That is effectively “click bait” and why Nature alowed it to get through puzzles me…
The whole thing kind of shows that citations and money are now inextricably bound in US Universities, Press and Journal publishing…
Clive Robinson • July 5, 2022 5:55 AM
@ SpaceLifeForm…
Another to add to the anomolies list.
My post immediatly above has made it onto the page…
Yet looking on the 100 Comments page it has not shown yet, despite the site not having any activity.
Just add to the list :-B
Clive Robinson • July 5, 2022 5:59 AM
@ SpaceLifeForm,
And both comments pop up together on the 100 comments page…
Suggesting not a “time delay” but a holding que or similar.
Clive Robinson • July 5, 2022 6:42 AM
@ Bruce, ALL,
I forgot to include the all important “social media link” in the not a Higgs Particle or field, so nothing to do with Dark Matter or Quantum Computers,
https://nitter.net/ArindamPhysics/status/1534630899282718720#m
george • July 5, 2022 8:15 AM
@Ted
So Google is going to start deleting some sensitive location data, including for places like abortion clinics? Good decision.
There was never any real reason to keep that on their servers in the first place.
oh sorry, forgot. Shareholder value of course. Your data is their cash.
Johnny B. • July 5, 2022 8:26 AM
@Clive Robinson
@Ted
So Google is going to start deleting some sensitive location data, including for places like abortion clinics? Good decision.
Good decision.
No, very bad decision.
1, It’s not solving the underlying problem of psychopath behaviours.
2, It’s playing into the hands of those who “terrorise” in both the old and new meanings.
Sorry but nothing but longterm harm will come of it.
History shows without any doubt trying to keep any information secret eventually fails.
None of those points make the deletion of abortion data a “bad decision” or “very bad decision”.
Winter • July 5, 2022 8:27 AM
@Clive
That is effectively “click bait” and why Nature alowed it to get through puzzles me
Nature is rather sensationalist. But this I can understand. This is actually pretty interesting and useful.
First of all, this article is not about the Higgs Boson, neither are all the other press pieces.
But, the Higgs boson, or the Higgs field have quantum fields/waveforms that are ruled by a specific mathematical structure. This specific mathematical structure can be recreated in other systems. That allows us to study certain behaviors of the Higgs boson at room temperature in the lab. Furthermore, these lab systems behave like the particles, which means we can do some things in the lab that would otherwise only possible in high energy accelerators.
This is no different than that we can study Bose-Einstein condensation using electrons (which are fermions) in superconductors. Bose-Einstein condensates have nice quantum properties that are useful for quantum computing. The same quantum properties are used in superconductors using electrons. Bose-Einstein condensates are hard and short lived. Superconducting is easy and long lived.
In an entirely different field, hawking radiation has been observed in the lab in an acoustic (mathematical) analogue of a black hole. Black holes are not accessible in the lab, the acoustic analogue is.
ht-tps://backreaction.blogspot.com/2017/10/space-may-not-be-as-immaterial-as-we.html
ht-tps://www.technion.ac.il/en/2021/02/hawking-radiation-and-the-sonic-black-hole/
All in all, this is part of a wide field of analogue systems that replicated “esoteric” quantum behavior in the lab.
Zed • July 5, 2022 8:41 AM
Johnny B
None of those points make the deletion of abortion data a “bad decision” or “very bad decision”.
+1. Not Google’s job to even solve those issues listed by Clive. Issues like that are a form of whataboutism, unrelated arguments designed to cripple decision making. As in “if your vacuuming the carpet does not make the house look good outside, maybe you should not even vacuum the carpet”.
Coolio • July 5, 2022 8:49 AM
Google announced that it would “restrict” a loophole that allows developers to see which other apps are installed and deleted on individuals’ phones.
“It’s long overdue. Data brokers have been banned from using the data under Google’s terms for a long time, but Google didn’t build safeguards into the app approvals process to catch this behaviour. They just ignored it,” said Zach Edwards, an independent cyber security researcher who has been investigating the loophole since 2020.
Android allows developers to find this information through the “Query All Packages” API. This API makes it possible for an app (or a third-party packaged used by an app) to query the inventory of all other apps on a person’s phone.
Data marketplaces such as Narrative.io sell data obtained in this manner.
Source: https://www.ft.com/content/d4c9b1d6-71a3-427c-acd9-75c3ffae8c18
Winter • July 5, 2022 8:54 AM
@Johnny B
None of those points make the deletion of abortion data a “bad decision” or “very bad decision”
I wholeheartedly agree. The lives of people, pregnant women in this case, go before any political or moral principles.
The underlying problem is the GOP going the Gilead way trying to destroy Democracy and founding a Theocracy/Kleptocracy in the US. Saving whom we can is the right thing to do.
Mac • July 5, 2022 10:08 AM
An interesting story of how a mermaid site is taking over Google search in Norway
https://alexskra.com/blog/the-mermaid-is-taking-over-google-search-in-norway/
Winter • July 5, 2022 10:15 AM
Google allowed sanctioned Russian ad company to harvest user data for months
Info included unique mobile phone IDs, IP addresses, location information, and more.
ht-tps://arstechnica.com/information-technology/2022/07/google-allowed-sanctioned-russian-ad-company-to-harvest-user-data-for-months/
The day after Russia’s February invasion of Ukraine, Senate Intelligence Committee Chairman Mark Warner sent a letter to Google warning it to be on alert for “exploitation of your platform by Russia and Russian-linked entities,” and calling on the company to audit its advertising business’s compliance with economic sanctions.
But as recently as June 23, Google was sharing potentially sensitive user data with a sanctioned Russian ad tech company owned by Russia’s largest state bank, according to a new report provided to ProPublica.
Google allowed RuTarget, a Russian company that helps brands and agencies buy digital ads, to access and store data about people browsing websites and apps in Ukraine and other parts of the world, according to research from digital ad analysis firm Adalytics. Adalytics identified close to 700 examples of RuTarget receiving user data from Google after the company was added to a US Treasury list of sanctioned entities on Feb. 24. The data sharing between Google and RuTarget stopped four months later on June 23, the day ProPublica contacted Google about the activity.
csurran • July 5, 2022 10:25 AM
@Winter
Google allowed sanctioned Russian ad company to harvest user data for months
Hopefully no US senators/representatives use Android. Or rather: hopefully no US senators/representatives use Android and are addicted to p0rn. Because that would make them a juicy target for “offers” from Russian interests.
Clive Robinson • July 5, 2022 11:25 AM
@ Johnny B, Zed,
I realy do not think either of you understand,
“It’s playing into the hands of those who “terrorise” in both the old and new meanings.”
The fact something is not on the map does not as has been pointed out by others here stop women being terrorised either by State authorities –old terrorism– or the lunatics –new,terrorism– that think they are on a mission from god or other stupidity.
But it does make it in any way safer for women trying to access those services. In fact it makes it not just harder but way more dangerous.
Because at the simplest rather than do a general search the women have to do a specific search thus drawing attention to themselves, and limiting any deniability.
I could go on with a lot more, but what’s the point, you are not “thinking hinky”.
So you are not seeing how it makes finding vulnerable women so much easier. Oh and make the prosecution so much easier.
But hey don’t let me stop you hurting women by giving Google a pass on a very deliberate ploy to give them “Good PR” and “Better invasion of privacy” on women that becomes 3rd Party Business records which do not require a warrant or any judicial oversight to be accessed in the US…
Winter • July 5, 2022 12:09 PM
@Clive
I could go on with a lot more, but what’s the point, you are not “thinking hinky”.
Removing information does not make women more vulnerable.
vas pup • July 5, 2022 2:35 PM
Underwater drones seized by Spanish police
https://www.bbc.com/news/world-europe-62040790
vas pup • July 5, 2022 3:26 PM
Good short video – History of Vaccination – and article itself
https://www.dw.com/en/why-we-have-cows-to-thank-for-the-eradication-of-smallpox/a-62355834
SpaceLifeForm • July 5, 2022 4:04 PM
NIST Results
‘https://www.nist.gov/news-events/news/2022/07/nist-announces-first-four-quantum-resistant-cryptographic-algorithms
‘https://groups.google.com/a/list.nist.gov/g/pqc-forum/c/G0DoD7lkGPk
See the second link and see what @hashbreaker noted about a patent.
SpaceLifeForm • July 5, 2022 4:45 PM
@ Clive, ALL
Keep in mind that Recent 100 is not dynamically created on the fly. There is a background process that periodically creates that webpage. That webpage is also subject to batcache.
So, there are two timers involved, and they are not in sync.
So, just like any page, you must wait at least 5 minutes, and then force refresh.
If you View Source the Recent 100, at the bottom of the html you may spot the batcache comments.
Note: For any View Source page, you also need to force refresh on that also. That is not due to batcache directly, but due to browser cache.
So, you are dealing with two different timers, and two different caches.
Clive Robinson • July 5, 2022 4:52 PM
@ Winter,
Removing information does not make women more vulnerable.
Yes it does, as I said to @Johnny B and @Zed, you have to think hinky and if you don’t do so then you fall into the trap that history has repeatedly shown is there.
Google are planning to remove public references to the medical services on maps etc that are involved with “Womens Health” issues.
This has two imediate detrimental effects, because it makes it way more dificult for legitimate users of those services to find them.
So Firstly, Women in need of medical services have a significantly more difficult time trying to obtain the medical services they need. It will force them to make web searches and the like that can and will be used against them. Information Google keeps as third party business records and makes available to just about anyone as the “Priest” found out when he was found to be in the vicinity of gay bars etc, then found guilty in the court of Main Stream Media…
Secondly the medical services suffer from decreasing numbers of patients, thus either they stop offering those services or go out of business. So making it even harder to find the healthcare they need.
This in turn means that women in need of services will end up back with “back street abortionists” or worse buying the likes of “Chinese Medicine” or similar poisons that could kill them or worse do damage to both the mother and fetus. Oh and “Midnight knock and draged into the streets” if they are lucky by the police if they are unlucky by lunatics goped out on religion or worse cultish behaviour. Think witch finding and trials from history.
This is not debatable, history shows us this will happen as it’s happened before and will just happen again. Do you realy want people dressed in white burning crosses or worse again? Because it will be by people with that type of mental makup “Doing Gods Work” or what ever their mad reasoning is.
But these days there will be a new effect. For the last half of the last century it was no easy task to prevent women finding such services, or traveling early to get them in another country. Thus the authorities had difficulty catching and prosecuting women or Dr’s and even “back street operators”.
The likes of Google and it’s invasive algorithms into peoples PPI mean that the women are now easily traceable, thus can be “dragnetted up” and put on display and thus suffer not just incredible injustice, but worse irreprable harm to the rest of their lives, physically, emmotionally and mentally as well as being effectively forced into a lower socio economic position.
Oh and don’t argue this won’t happen, teenagers swaping photos of themselves in various states of undress on IM or other social media. Effectively the modern version of “Drs&Nurses” or “You show me yours and I’ll show you mine” that has been going on for cenruries or more one way or another as part of growing up. Have been turned into “Sex Offenders” with a life long stigma hanging over them, and the inability to study, work, travel etc. All so some DA can “make numbers” during the day and do highly questionable things by night.
The level of harm this action Google is proposing is horrendous and the fact people are not seeing it for what it is, even though there is plenty of historic presidence, realy realy worries me.
I think a lot of people should realy wise up, and wise up quickly.
Clive Robinson • July 5, 2022 5:07 PM
@ SpaceLifeForm,
Re : you must wait at least 5 minutes, and then force refresh.
Look at the time stamps, it’s 19mins difference at a minimum…
I’ve seen longer…
Winter • July 5, 2022 5:34 PM
@Clive
Google are planning to remove public references to the medical services on maps etc that are involved with “Womens Health” issues.
That is not what Google is planning to roll out.
ht-tps://www.dailywire.com/news/google-to-scrub-location-history-for-women-who-visit-abortion-clinics
Google announced on Friday that user data for visits to certain facilities, including abortion clinics, will be automatically deleted.
As I understand it, Google will automatically delete the location history for anyone who comes near such a facility, not just visitors.
Clive Robinson • July 5, 2022 5:34 PM
@ JonKnowsNothing, ALL
Re : BA.4 and BA.5 and mutations.
It would appear that evidence is mounting that they are both more harmful than origonaly thought, as well as more infectious.
The basic evidence is the increase of hospitalisations “due to” it as the primary cause rather than from other cause “with”…
This does not bode well.
Nor does the actions of “Big Phama” with serious suggestions to the FDA that a booster shot should still be the now effectively usless first mRNA… Oh and at increased pricing…
As for the fairly bad flu virus that has hit Australia and will start hitting the Northern Hemisphere in three months or less. It would appear that “Big Phama” is whacking up the cost and probably can not make the required vaccine…
So for anyone over 45 I would recommend going back to “masks etc” right now and minimize all social, work, or traval contact as a first step “self defense”.
SpaceLifeForm • July 5, 2022 6:04 PM
@ Bruce, Clive, Anonyrat, &ers
re: Memory Leak
Based upon limited internal information, and only network Observation, I think that my hunch is correct. MariaDB.
Supposedly fixed, but is the fixed version deployed here? I suspect not because these problems have existed before the alledged fix was rolled out in version 10.4 last year. And people are still seeing Memory Leak problems in later major versions.
‘https://jira.mariadb.org/browse/MDEV-20455
‘https://jira.mariadb.org/plugins/servlet/mobile#issue/MDEV-20455
Pressable needs to upgrade.
So, near term work-around, 03:00
Did you try turning it off and back on again?
SpaceLifeForm • July 5, 2022 7:07 PM
@ Clive
Yes, I Observed that which you Observed.
You know I pay attention to Metadata.
And I assumed you have gotten into the program by now, and do force refresh after waiting 5 minutes.
But, tell me, what is the periodic timer for the Recent 100 page rebuild?
If it is 15 minutes, then your Observation still fits.
How does the MariaDB Replication fit in?
It is not instant. In fact, it can not be, Physics and all. Do you know what your DNS gave you and which server you connected to?
There are multiple symptoms in view, and it can be difficult to unravel the spaghetti. But symptoms can lead you down a rabbit hole that have nothing to do with the underlying cause.
In this case, I stare at my prime suspect. The MariaDB software.
The symptoms are explainable, if you can accept that the underlying cause is Memory Leak in MariaDB software.
JonKnowsNothing • July 5, 2022 7:30 PM
@ Clive , @All
re: BA.4 and BA.5 and mutation recycling
I would recommend that anyone who has a choice, mask up and close up and reduce their exposures as much as possible.
Not only are BA4 BA5 more of “everything”; the retread, rebounds and constant redux is about to get big time play.
It has been clear that the HIP-RIP-LOVID economic model has won the day in much of the world, with the major exception of China, where they are not interested in what the Western Economies are offering in the way of Constant-Covid, but the HIP-RIP-LOVID economies have bet the farm on things that are not turning out “as planned”.
Early findings on reinfection are proving just as valid now as then (P1 P2 Brazil) only more so. The mutations in BA4 BA5 put paid to the idea that “You Only Get It Once”.
Reports that people are on their 3-4-5 rounds of COVID-19 and those reports do not include rebound infections.
What isn’t clear is how the HIP-RIP-LOVID economies are really planning on having significant re-infections on periodic basis. As summer vacation tourism has spawned and more tourist destinations are dumping all COVID protocols, it can be expected a huge uptick in sickness + mutation mania will be shortly arriving on the homeward journey.
The concept seemed to be
This is not working out at all.
There will be a big uptick from holiday exposure coming, along with new variants like the combinants, but also big down tick in protection from vaccines and treatments. So the aftermath will not be of short duration.
The reinfection rates are making some “take notice”, a bit like hiding under a wooden desk, since this aspect isn’t governed by vaccine uptake. With or Without, makes little difference.
SpaceLifeForm • July 5, 2022 9:03 PM
@ Bruce, Clive, Anonyrat, &ers
re: Memory Leak
I’m pretty sure the Memory Leak is occuring in glibc malloc().
But, hey, what do I know?
Since that will likely never be fixed in your lifetime, the long term solution is to restart the entire webserver software stack at 03:00 every day.
Nuke from orbit.
This is the only workaround for now.
A better stack would use PostgreSQL
and MUSL. But that is not going happen.
So, expect the problems to periodically show up. Timing will depend upon traffic volume.
Restart every day at 03:00 and everyone will be happier.
Is 03:00 US Central time acceptable for everyone if you know it is scheduled?
I think that would be acceptable.
This is the only workaround for a long time.
But, hey, what do I know?
SpaceLifeForm • July 5, 2022 9:40 PM
re: Uvalde
Mayor buys vowel
‘https://amp.cnn.com/cnn/2022/07/05/us/texas-uvalde-mayor-don-mclaughlin/index.html
SpaceLifeForm • July 5, 2022 10:12 PM
Mist in the NIST
Beware the scary CryptoGhost.
Run! Hurry to the next swamp!
‘https://arstechnica.com/information-technology/2022/07/nist-selects-quantum-proof-algorithms-to-head-off-the-coming-cryptopocalypse/
Clive Robinson • July 6, 2022 12:21 AM
@ SpaceLifeForm,
Re : I Observed that which you Observed.
The problem is you are sending out mixed messages.
If you say,
So, just like any page, you must wait at least 5 minutes, and then force refresh.
Without carefully qualifying it, some people will read it and when they find after 5mins it is apparently not true they will think you are in error.
The posting poblem has the six basic symptoms from the thankyou page saying “approved”,
1, Posts on thread and 100comm pages within a couple of minutes.
2, Posts on thread in a couple but takes more than five on 100comms
3, Posts on 100comms in a couple of mins but thread takes much longer.
4, Apparently posts on neither for quite some time.
5, Does not post and gets held in limbo from which it can be retrieved.
6, Just disappears without trace.
What causes any of the above is an open subject and unlike you, I’ve not dug below the “What the ordinary user observes” level.
Clive Robinson • July 6, 2022 1:05 AM
@ SpaceLifeForm,
I’m pretty sure the Memory Leak is occuring in glibc malloc().
It’s been a few years since I looked but the malloc() in glibc had it’s states covered.
What it did not do was “canary the data structures”.
One of the big failings of traditional malloc() was that it included the data structure with each assigned block of memory (often at the begining).
So the structure records would get easily overwritten by program code that went out of bounds.
So whilst the malloc() code would scale up in heap space it was quite fragile.
If the allocators of pages of virtual memory like sbrk() and structutes in heap memory were bring written today I suspect that they would both be written rather more robustly including having actual garbage collection as well rather than leave it to insufficiently experienced programmers to try to puzzel out or ask wuestions and get incompleate answeres like,
https://stackoverflow.com/questions/19676688/how-malloc-and-sbrk-works-in-unix
Which you will note does hint that malloc() makes an assumption about the use of sbrk() being by malloc() alone. In fact it’s a bit more complicated than mentioned but I’m not going to go into it other than to say either do not use sbrk() or write your own malloc functions yourself.
Oh sbrk() unlike malloc() keeps “house keeping” well away from the actual memory used by the programmer so overwriting the structures is way less likely to happen. The reason for this was not to “canary the structurs” but because of the VM MMU hardware dictated it for sensible reasons.
If you are going to write your own malloc() it’s something you should give consideration to. In effect drop the “infinite tape” considerations and all the troubles it has brought with it. Likewise build in a sensible garbage collector to get rid of memory hole build up.
Winter • July 6, 2022 1:24 AM
Imagine all the people
Sharing all the world
You may say I’m a dreamer
But I’m not the only one
Landmark EU rules will finally put regulation of Big Tech to the test
ht-tps://arstechnica.com/tech-policy/2022/07/landmark-eu-rules-will-finally-put-regulation-of-big-tech-to-the-test/
Imagine an online world where what users want matters, and interoperability reigns. Friends could choose whichever messaging app they like and seamlessly chat cross-app. Any pre-installed app could be deleted on any device. Businesses could finally access their Facebook data, and smaller tech companies could be better positioned to compete with giants. Big Tech could even face consequences for not preventing the theft of personal info.
As the US struggles to pass legislation to protect Internet consumers, in the EU, these ideals could become reality over the next few years. EU lawmakers today passed landmark rules to rein in the power of tech giants such as Alphabet unit Google, Amazon, Apple, Facebook (Meta), and Microsoft, establishing a task force to regulate unfair business practices in Big Tech
Clive Robinson • July 6, 2022 2:34 AM
@ SpaceLifeForm,
Re : Mayor buys vowel
My feeling based on what was indirectly said at the time is that it was a “by the book” problem.
That is the training effectively was “bureaucratitc procedural” and as is often the case written by people who think more in terms of litigation limitation than rapid situational resolution.
Whilst I can understand such “by the book” behaviour with common events that are sufficiently well understood, thus have a sufficient degree of predictability in them. As the old military maxim has it,
“No plan survives contact with the enemy”[1]
It is known that the “Desert Fox” Rommel was of the “get there and go, go, go” mentality. In his 1961 book “The Desert Generals” Correlli Barnet said,
“Rommel took Moltke’s view that “no plan survives contact with the enemy”. If his plan got him into battle, it was enough. After that, Rommel would fight by ear and eye and tactical sense, like a duellist.”
That is a “duellist” with swords spends much time gettin the small but very important small steps not just instinctive, but as close to lightening fast as they can. That is not only what keeps them alive, but also free to think on their feet to take advantage of opportunities as they arise.
Modern “combat training” by the military is still based on this idea.
Modern “street training” by the Police is very much based on procedure. Even in “riot training” procedure rather than tactics and expediency is what is taught…
Which approach should be used, is unfortunately a problem, because training for both does not realy work as things have to be “instictive” to be “effective” and generally instinct is reactive without thought.
A friend who is long since retired tells a story of how things can happen.
He was both a full time Police Officer, but reservist soldier and rugby player.
He got into a situation at work with “a member of the public” who was somewhat “emotionaly impared” due to the fact they had consumed two or three gallons of beer over a period of time. My friend went into “proceadual mode” and tried talking to the inebriate quietly and unthreateningly. Unfortunately the inebriate was a “bottle nutter” and went from mildly loud to full on psycho in the blink of an eye and stabed my friend. As my friend put it his “auto pilot jumped gears” and he went into first “military mode” then as the pain went from surprise to real hurt jumped to “rugby mode” and the bottle nutter discovered just how much force is required to smash you through a well built bar, and just how much pain 200lb of enraged muscle can inflict by using you as protective padding whilst doing so.
As my friend wryly noted, his stab wound still hurt after six months and no amount of commendation eased that ache. But he suspected the bottle nutter was going to hurt for a lot longer as half a dozen broken ribs and a drinking arm broken in three places never realy mend well.
As I like to pull peoples chains just a little in a friendly way, I enquired who the pub landlord had sent the bill to for rebuilding the bar. To which my friend replied not him, as apparently seeing him in full on “rugby mode” had caused a little dampness in the barman’s trouser department.
[1] It is a paraphrasing of the 1871 statment in Volume 1 of the official account of the Franco-German war by Prussian Field Marshal Helmuth von Moltke (not Carl von Clausewitz),
“No plan of operations extends with any certainty beyond the first encounter with the main enemy forces. Only the layman believes that in the course of a campaign he sees the consistent implementation of an original thought that has been considered in advance in every detail and retained to the end.”
Where in the modern world “layman” should be replaced with “litigation lawyer” or “idiot” depending on your choice of phraseology or context, and are by no means mutually exclusive.
lurker • July 6, 2022 2:52 AM
@Clive, SLF
“6, Just disappears without trace.”
Note that “approved” posts can be disappeared a few hours or minutes after appearing on page. Apparently a human hand working above the eh eye. So another conundrum, can this hand work during the time buffer?
Winter • July 6, 2022 3:12 AM
@Clive
Whilst I can understand such “by the book” behaviour with common events that are sufficiently well understood, thus have a sufficient degree of predictability in them. As the old military maxim has it,
The fact is, regular police personnel are not meant to be an army fighting gun fights. That is where SWAT teams come in. They should have the training and gear to actually handle such armed resistance. Obviously, the USA do not have SWAT teams or equivalent on standby within, e.g., 15-30 minutes.
In such cases, every city or town should have trained LEOs on standby who can handle such situations at least long enough to get reinforcements.
But that costs (tax) money, and Texas is well known to not wanting to spend any tax money on public safety [1].
[1] See Texas Chill
https://www.independent.co.uk/news/world/americas/exas-mayor-power-outage-facebook-b1803181.html
“Let me hurt some feelings while I have a minute!!” Mr Boyd said in his post. “No one owes you are [sic] your family anything; nor is it the local government’s responsibility to support you during trying times like this! Sink or swim it’s your choice! The City and County, along with power providers or any other service owes you NOTHING!!”
SpaceLifeForm • July 6, 2022 4:30 AM
@ Clive
Re : I Observed that which you Observed.
Good point. Obviously, I was not clear.
What I was trying to say it that, when debugging, it it critical to know what you are actually Observing.
The purpose of waiting 5 minutes and doing a force reload is to at least give yourself a chance to Observe what you would think would be visible.
If it is not visible, then that it an indication of a different problem besides timers and caches.
That said, I still believe it is a Memory Leak, and all known symptoms of what we all see as visible problems, can, in fact, be explained by Memory Leak.
Did I mention MariaDB Cache? Sorry, my bad.
SpaceLifeForm • July 6, 2022 5:40 AM
@ lurker, Clive
re: “6, Just disappears without trace.”
If it never made it to storage, then that would be a symptom.
Debugging is hard. Especially if you concentrate on the symptoms.
Yes, the ‘time buffer’ may be an issue. But you must start at the bottom and secure your way up.
See Database Write Commit.
Did the post really make it to storage?
If the system is not configured properly, it is possible that the DataBase software thinks it did the writes to storage, but the Linux kernel may not have forced to storage. And who knows what really is going on with SSD these days?
‘https://danluu.com/fsyncgate/
‘https://www.evanjones.ca/durability-filesystem.html
‘https://people.csail.mit.edu/nickolai/papers/chen-dfscq.pdf
Oh, and if a process gets killed by the kernel due to OOM (Out Of Memory) conditions, what happens to the dirty filesystem pages created by that process?
And why would OOM kick in you may wonder. Memory Leak.
Clive Robinson • July 6, 2022 6:17 AM
@ SpaceLifeForm,
Re : Arstech article on NIST PQC
I’ve had to stop reading it several times as it made me grind my teeth or mutter intemperate comments avout what they were saying.
Not least the QC “instantly” comment.
But when you look at,
“Those giant NSA data centers in the middle of the desert were basically constructed for the moment quantum computing becomes viable. Pretty crazy (and terrifying) when you think about it: they’ll be able to decrypt every piece of traffic they’ve ever saved.”
What a load of Bovine Scat… The NSA has had a policy of “Collect it all” that was started before the Agency was inaugurated and whilst part of it was for doing the supposed impossible (see VENONA) the main reason for it existing was not decryption but the much more powerfull traffic analysis that could decades in give information linking apparently disparate parties together.
In fact the NSA were “doing their vacuuming” quite a number of years before Quantum Computing was envisaged or given formal footing[1]. So the first statment in that paragraph is demonstrably not true.
As for the second, that is again untrue. Traditional symetric algorithms of sufficient complexity are secure against QC in any realistically chosen time frame.
What is not secure is the “mathmatical” asymetric algorithms used to transfer keys for the likes of symetric algorithms where no existing “root of trust” secret exists.
Thus the encrypted messages that will be recovered will be a very small percentage of those that used asymetric algorithms to establish a “shared secret”.
As I’ve said in the past to my knowledge the One Time Pad has still been in use for “emergancy key transfer/managment” back in the early 1980’s right through, to where it is still been used for that task today.
Anything sent by OTP used properly will remain secure indefinitely. And if that was the symetric keys of sufficiently complex algorithms, the the data encrypted under them will remain unencrypted for quite some time to come as well.
QC is not going to have an “overnight” impact, it actually will take years to go through even a small fraction of what the NSA and other SigInt agencies currently hold. They know it as well, therefore we can expect them to work on an ROI model, where they will work on what gives the greatest returns in the shortest time.
Interestingly the NSA have kind of shot themselves in the foot. Their symetric encryption algorithms are known to have been designed with little or no margin in the designs. Therefore if they gave an 80bit strength, you could be reasonably sure it was at best just on the mark at the then current knowledge level. Algorithms from academia have tended to put in what is a “healthy margin” based on the simple acknowledgment that what we know currently has a reasonable chance of being superseded fairly rapidly tommorow or shortly there after.
Thus DES for instance was hardly out of the box before it started looking weak and we got less than a quater century of usefull life out of it similar for other NSA involved algorithms.
The AES algorithm being of academic origin looks like it might make it to half a century of usefull life…
Yes Quantum Computing if it moves from the theoretical to the practical will do damage, but it won’t be “overnight catastrophic” damage and effect mostly asymetric systems and other KeyMan systems.
There are non technical solutions for solving the QC issues, but ultimately rely on a trusted second communications channel which the first and second third parties can exchange a masyet secret as the root of trust.
But those are generally not viable for one-off private communications such as for “Online Shopping” and a selection of “Social Media” activities.
And from the “ordinary mortal’s” percpective those are what they mostly use encrypted communications for.
Therefor much of the NSA stored encryptions are not that much use to the NSA or others. Because they can already get the data easily using third party business record access in various ways.
But importantly people have to remember that whilst the actual “message” is of little importance, the knowledge of “who is talking to who and when” is very much more important than the message, and the SogInt agencies do not need cryptanalysis, let alone Quantum Computing to give them that.
[1] There are many who claim XXX or YYY thought up the idea for Quantumn Computing, but they can not show XXX or YYY did anything more than make statments. Which is why David Deutsch is different, not only did he think up the idea he ran with it and produced algorithms and the first proof that QC would be exponentially faster than CC back in the mid 1980’s. He did other interesting QC related work but has moved on to rather more interesting work,
JokingInTuva • July 6, 2022 9:50 AM
@Clive Robinson
Re : Arstech article on NIST PQC
Some comments:
– I am not really sure that the amount of secure communications relying Asymmetric Key establishment exchanges is really so small
– It is not so easy always to base those seucre communications in pre-known secrets
– OTP per se is not enough (at least lack of authentication, beyond other difficulties: high consumption rate of protecting keys, non-repetition assurances, etc)
– So the OTP channel must be authenticated (OK, it could be based in currently available QC resistant symmetric crytpo & other pre-known keys)
– In any case, I guess that migration to asymmetric PQC is here to stay (and it may still have some Rainbow-like surprises ahead, but it is going to be quite interesting)
Clive Robinson • July 6, 2022 2:37 PM
@ JokingInTuva,
1 – I guess that migration to asymmetric PQC is here to stay
Only in as far as it replaces the existing asymetric “key negotiation” and “data signing” functions.
Actual data encryption that is not based on mathmatics which is the majority of Symetric algorithms, will not realy change, just double up on the number of “effctive” key bits (where effectctive is a little hard to define).
2 – I am not really sure that the amount of secure communications relying Asymmetric Key establishment exchanges is really so small
It is and it is not. The big lump of encrypted data using asymetric algorithms is as I indicated mostly of no inyerest to SigInt agencies. Basically web-browsing, e-commerce, social media.
The traffic they are supposed to be interested in is foreign nation Diplomatic and Military traffic, and a number of “commercial enterprise”. Large organisations with any sense would have established “shared secrets” and “Key Managment”(KeyMan) networks where the use of mathmatical asymetric ciphers should be unnessesary (except for certain “odd” functions where the use of asymeyric crypto make things easier, but it can still be done other ways).
3 – It is not so easy always to base those seucre communications in pre-known secrets
It all depends on the “second channel” that the mathmatical asynchronous algorithms try to substitute with what would be “One Wau Functions”(OWFs) if not for the difficult but not impossible to find “trapdoor” secrets.
Prior to RSA etc, the Diplomaric and military agencies of governments had little difficulty in estavlishing the second channel by which the master / shared secret was transfered between two parties. It’s still just as easy/hard for them today.
It’s only the civilians who do “online shopping” etc that realy need asymetric crypto algorithms for key exchange. As for “signing documents / data” we don’t actually do that, we “sign a secure hash of the document”. Protecting a document can be done in other ways as glorified “anti-bit flipping” techniques.
4 – OTP per se is not enough (at least lack of authentication, beyond other difficulties: high consumption rate of protecting keys, non-repetition assurances, etc)
Due to bit fliping on any bit being actually part of the proof of security of OTP systems, the plaintext in an OTP can not be protected.
However the resulting ciphertext can be protected which if done using appropriate crypto peotocols is often sufficient.
Which is what I think you are saying with,
5 – So the OTP channel must be authenticated (OK, it could be based in currently available QC resistant symmetric crytpo & other pre-known keys)
The point is that “Post Quantum Crypto”(PQC) is not realy about “data encryption” but KEM for those who can not establish a second secure channel and authentication of data by “signing” that was being done other ways prior to the early 1970’s and RSA being made public.
Neither of those is generally required for Government Agencies as they can and in some cases still are done other ways.
The NSA like most other traditional SigInt agencies were set up to go after such Government Agencies, not civilians doing online shopping, browsing the web and exchanging social information via social media. All of that “civilian” traffic can be obtained without the use of even cryptanalysis, let alone Quantum Computing. It can be obtained by accessing the “Third Party Business Records” that exists with all but properly implemented “End to End Encryption”(E2EE). As for E2EE I’ve yet to find a civilian or commercial system that actually has “Properly Implemented” E2EE so doing “end run attacks” on the available systems is somewhat trivial despite the nonsense their designers trot out.
So in a way “Post Quantum Cryptography”(PQC) is a technology that currently has nothing to protect you against, that may never actually happen in your or my life times. But even if “Quantum Computing”(QC) actually can be made to work on a thousand bits let alone 16kbits of some PQC systems it’s going to be very resource intensive, this your and my traffic is highly unlikely to get looked at with QC systems.
JokingInTuva • July 7, 2022 1:45 AM
@Clive Robinson
“Protecting a document can be done in other ways as glorified “anti-bit flipping” techniques.”
“anti-bit flipping techniques” beyond for instance regular Symmetric Authentication MAC?
SpaceLifeForm • July 7, 2022 1:48 AM
I’ve made it about half way thru this long thread
‘https://danluu.com/fsyncgate/
Just some Random Thoughts
You can not trust the kernel. The best you can hope for is that it can do the storage I/O reliably. You need to Trust, but Verify.
You must pre-allocate all storage blocks in your database files. The EOF should be locked in. Blocks physically allocated, no Sparse.
You must use Direct I/O for a robust DataBase.
The DBMS must manage all blocks. You should attempt to do at least some type of RAID-1.
Proper RAID has to happen at software level.
open() should call fsync() and if there is a write error, the open() should fail. FailFast! The underlying file directory inode should be marked as corrupt so no longer openable until some type of recovery step has occurred. This applies even if the open is readonly. The application needs to know.
close() should call fsync(), but most application code will not care even though there could be dirty filesystem blocks that may never make it to storage. But, at least, the kernel could mark the file corrupt so that a future opener can FailFast!
If the kernel can not mark the file as corrupt, it should panic.
open() must fail if the file has been marked as corrupt, even if the problem was detected on an earlier close(), which may have happened days ago. Think about it, why would an application calling close() on a file that it opened read-only even consider that there was a write failure on the file previously? The application needs to know.
‘https://man7.org/linux/man-pages/man2/close.2.html
ResearcherZero • July 7, 2022 2:26 AM
@Clive Robinson
The result of trying to keep things secret builds up a tsunami of tyrannical systems that get abused to oppress people to gain power and control over them.
Reports that are to remain classified often end with “may cause public panic” (a euphemism for the public can not know), or, “if any of this information was to see the light of day it may bring down the government”.
Eventually some parts of these reports are released, but without any context, in order to prevent the public discovering the government had the information prior. Responsibility is avoided by always being as slippery as possible, and with that comes the danger of foreign insertion, which unlike the public, is all too aware of what is being kept secret, and therefor perfectly positioned to take advantage of it.
Government always has a plan, to tell people in a timely manner, what is no longer relevant. Of course you may have to give up just a few of your rights to find out what is no longer relevant, in order for it to be possible for the government to tell you (due to the provisions of privacy laws).
Clive Robinson • July 7, 2022 3:11 AM
@ JokingInTuva,
beyond for instance regular Symmetric Authentication MAC?
There are numerous ways that are used to reliably detect changes in transmitted-text you could use chained Merkel Trees for instance.
Which you use depends on your requirments or changing circumstances.
It’s one of those subjects “that can fill a book” and infact has filled several in my dead-tree cave, ad I assume it has in others.
What you are looking for is a reliable system that has no requirment for a “shared Secret” that forms a “root of trust”.
At a high level one such construct would be a “One Way Function”(OWF) that is “Without Trapdoors”.
There are two as far as I’m aware “currently unanswered questions” that are however pertinent,
1, Do OWF’s provably exist?
2, If OWF’s can be proved, can they also be proved “trapdoor free”?
Such is the fun of human knowledge…
But the real issue that needs solving is,
To come up with an intangible information based second channel to “securely transfer” a “root of trust”.
Currently we have tagible physical based second channels to “securely transfer” a “root of trust” such as couriers hand carrying OTP’s in Diplomatic Pouches… But they all have two problems,
1, They are meet in person.
2, They require secure storage from manufacture, through delivery and through time of destruction after use or expiry.
Problems it’s hoped an intagible information based second channel system would resolve. Especially the first problem which is critical impediment on “remote commerce”.
Anyway time to “Start the daily grind” as they say with a cup of iced-tea today as the sun is being seasonaly unkind to the top of my head if I take my hat off…
Clive Robinson • July 7, 2022 3:26 AM
@ SpaceLifeForm,
Re : Just some Random Thoughts
What you are scratchingvat is the,
“CRUD dilemma”
Put simply the only way you can know your data is safely stored is to do a “full readback from media”.
Anything less is inhetently unreliable.
The problem is “write through to media” is not just hoplessly inefficient it is mind numbingly slow. Likewise “read back from media”. Thus “performance” takes a massive hit…
So the same solution as with CPU’s is used of “multilayer caching” as the caches all use “fully mutable memory” (RAM) it takes very little for data to be lost or overwritten.
So any bug along the chain will be a potential cause of data loss from the lower caches, even though the system has said the data has been saved, it only means as far as the “first cache”.
That’s the way we’ve built computers for fifty years or more. As unreliable as a bad tempered mule, but somehow managing to carry the load mostly…
SpaceLifeForm • July 7, 2022 6:18 PM
Anom decompile
‘https://www.vice.com/en/article/v7veg8/anom-app-source-code-operation-trojan-shield-an0m
SpaceLifeForm • July 8, 2022 1:19 AM
Who screamed Feedback the loudest?
‘https://www.bleepingcomputer.com/news/microsoft/microsoft-rolls-back-decision-to-block-office-macros-by-default/
SpaceLifeForm • July 8, 2022 2:23 AM
re: US v Schulte
Closing arguments were yesterday. It will go to jury in about 6 hours.
Remember, DEVLAN was NOT Air-gapped. All 200 of the devs had total access. Basically, they could have just put post-it notes on every machine with the root password. Think about it, if they are developing these hacking tools, what would be the point of trying to lock DEVLAN down? They could just hack back in with the tools. Which is why DEVLAN was referred to as the Wild Wild West.
On this link, scroll down a bit until you see July 7.
‘http://www.innercitypress.com/sdnylive110schultefurman070722.html
Ted • July 8, 2022 9:23 AM
This is going to be interesting. EU Parliament votes 324-155 to condemn US supreme court overturning of Roe v. Wade.
Does this mean Europe will offer tacit support to mail pharmacies who send abortion pills to women? Seeing that most state laws threaten abortion providers and not the woman, the prosecution of these activities gets trickier.
About half the legal abortions in the US are actually from abortion pills. That number is three quarters in Europe.
How have abortions been surveilled in the past you wonder? Cynthia Conti-Cook, a civil rights lawyer and technology fellow at the Ford Foundation, says: “Google searches, websites visited, email receipts.” Are states even allowed to track the mail?
Subscribe to comments on this entry
Sidebar photo of Bruce Schneier by Joe MacInnis.
vas pup • July 1, 2022 5:07 PM
AI predicts crime a week in advance with 90 per cent accuracy
https://www.newscientist.com/article/2326297-ai-predicts-crime-a-week-in-advance-with-90-per-cent-accuracy/
Israeli tech works to avert hospitalizations by flagging problems with clashing meds
https://www.timesofisrael.com/israeli-tech-works-to-avert-hospitalizations-by-flagging-problems-with-clashing-meds/
“Technology built and now deployed in Israel works to warn doctors if their patients’ uncoordinated treatment regimens could set them on course for hospitalization.
Significant numbers of elderly patients end up in hospital because of clashes between different drugs they are taking for preexisting conditions — or when their treatments are not adjusted based on the latest lab results. This is known in medical jargon as suboptimal polypharmacy.
Leumit Healthcare Services, one of Israel’s four healthcare providers, has deployed an artificial intelligence system developed by the Tel Aviv-based FeelBetter, which is designed to reduce instances of suboptimal polypharmacy harming patients. It was put in place in January, and there are no detailed statistics on its success yet — but a retrospective study suggests high effectiveness.”
The artificial intelligence tool generated lists of the patients who appeared, based purely on data, to be at risk of hospitalization due to suboptimal polypharmacy. Researchers then cross referenced their lists with data regarding which patients were actually hospitalized due to suboptimal polypharmacy, and
found that the AI tool identified most cases.”