Ensuring the Accuracy of Electronic Voting Machines

A Florida judge ruled (text of the ruling) that the defeated candidate has no right to examine the source code in the voting machines that determined the winner in a disputed Congressional race.

Meanwhile:

A laboratory that has tested most of the nation’s electronic voting systems has been temporarily barred from approving new machines after federal officials found that it was not following its quality-control procedures and could not document that it was conducting all the required tests.

That company is Ciber Inc.

Is it just me, or are things starting to make absolutely no sense?

Tags: , ,

Posted on January 4, 2007 at 12:06 PM • 37 Comments

Comments

Fred F. • January 4, 2007 12:52 PM

I think the biggest problem is having people understand that statistics. If there is a statistical anomaly in the votes then something went wrong.
I can understand the legal reason for protecting these companies’ IP, but there are ways to analyze the code and still protect the IP. After all there is a lot of stuff that is sealed in patent disputes, etc. It all seems very strange, unless one assumes the judge does not understand statistics and thinks it is just another case of a sore looser, or darker motives for the tin foil hat crowd.

Rich • January 4, 2007 12:53 PM

It is even worse.
Note this from the article: “The commission acted last summer, but the problem was not disclosed then.”

Davi Ottenheimer • January 4, 2007 1:03 PM

Yes, I heard about this when it first broke and I’ve been looking for the actual ruling.

What scares me is quotes like this:

“Buchanan spokeswoman Sally Tibbetts said in a statement released by his campaign. “As noted by the judge in today’s ruling, two parallel tests conducted by the state revealed ‘100 percent accuracy of the equipment in reporting the vote selections.'”

The fact that one or a dozen or even a hundred tests does not produce an error does not prove that errors will not happen. Is the judge familiar with software security flaws? Were the tests conducted properly? Source code review really is the reliable route, so to choose an unreliable answer as proof of something…

Patrick Henry • January 4, 2007 1:27 PM

the possibility of lost votes … didn’t
warrant overriding the trade secrets

I’m glad we have our priorities straight.

Davi Ottenheimer • January 4, 2007 1:30 PM

Bah, spoke to soon again. Here’s the text of the ruling:

http://electionlawblog.org/archives/ess-pdf.pdf

Note the final paragraph

“For this Court to grant Plaintiffs’ motions would require this Court to find that it is reasonably necessary for the Plaintiffs to have access to the trade secrets of Defendant, Election Systems & Software, Inc., based on nothing more than speculation and conjecture, and would result in destroying or at least gutting the protections afforded those who own the trade secrets.”

That wording is quite speculative in its own right, with disappointing analysis.

It says to me that if trade secrets may be lost in the pursuit of protecting democracy, then the court favors secrecy.

Moreover, independent source code analysis certainly does not require loss of trade secret, or such a practice could not even exist in the market. NDAs and other IP protections should be obvious controls to a court familiar with software security.

The judge also said in a prior paragraph that “Plaintiff’s have presented no evidence to demonstrate that the parallel testing was flawed and/or the results not valid.”

Likewise, you could ask what proof is there that the parallel testing was a valid proof of accuracy when (in paragraph C) 18,412 undervotes were noted? What really constitutes evidence to this judge?

I regularly work with software released to hundreds of millions of people and I can say with absolute certainty that flaws are always found after release because the money/time spent on testing to get that number to 0 is (I would say reasonably) seen as prohibitive to the business. In other words, when there is no fear of catastrophic loss due to a potential flaw (all reports are treated as potential when they come from people without access to source), the bar for testing is obviously low. On the flip side, if a flaw would be considered equal to total loss (e.g. a shuttle explosion) then a much higher bar for testing would be in place.

Seems like conjecture is allowed by one side, but not the other, no? So the bar must be pretty darn low for someone to say source does not have to be checked independently/audited. Without more info, and from a security perspective, the judge’s ruling seems specious at best.

jonny s • January 4, 2007 1:35 PM

If you aren’t rich or influencial enough to skew election results in whatever direction you deem correct, you’re obviously not intelligent enough for your vote to count, anyway.

sng • January 4, 2007 1:45 PM

While I agree that the source code for these devices should be able to be examined. I have to wonder about this trend of dispusting elections. An election is an exercise in statistics just like any other and it seems to me that since the 2000 election people have started disputing every election that falls within a reasonable margin of error. Only knowing what I’ve read in this one article this almost seems like a “you lost a close election stop whining” sort of thing.

If this is correct the judge should have said so. In more civil terms, of course. But certainly there should be a public review of the source of all of these devices.

Or better yet a return to paper ballots and doing something to stop this wave of whiners about close elections set off by Bush and Gore. As I have a really hard time thinking that in the last six years that election fraud has suddenly become widespread. And not that people learned that going to court over election results is a possibly viable tactic to win really close elections.

http://dilbertblog.typepad.com/the_dilbert_blog/2006/12/those_darned_vo.html

greg • January 4, 2007 2:01 PM

I don’t care who you are. 18000 votes goes missing and nothing happens!! You americans are crazy. I would want a revote. Period. And I would want a full independant audit. Its a crime in most western countries to be incompatent with money. Its also a crime in most countries to be incompetant with vote counting.

USA has got the most bizzar tolerance to election fraud in the free world.

denis bider • January 4, 2007 2:04 PM

And oh, by the way – how many “trade secrets” is it that you can actually have in an election machine, anyway? How is that interface any more complicated than displaying a few buttons and recording the results in some format?

It looks to me like the complexity of the software in those machines should equal something that can be done in a few weeks’ time by any capable programming student.

There are no trade secrets in there. There can’t be; there isn’t room for any.

If the manufacturer of those machines claims that there are secrets requiring protection, there can be only one type of secret – the sinister type. And that is all the more a reason to be suspicious and investigate the technology.

A manufacturer of such machines should be open about the internals and willing to prove their quality – not trying to hide the details. What HAVE they got to hide?

markm • January 4, 2007 2:34 PM

the possibility of lost votes … didn’t
warrant overriding the trade secrets

Up to six years ago, the certainty that punch card voting systems were misreading about two percent of the cards at every election didn’t warrant spending a little more for a more accurate system.

Alan • January 4, 2007 2:42 PM

More and more I feel like I am living in a banana republic run by banana Republicans. There are so many voting irregularities and even obvious fraud and nothing is done about it. No wonder young people do not vote. They know the game is rigged.

Basil Berntsen • January 4, 2007 3:22 PM

This isn’t as stupid as it sounds- firstly, on page 3 the ruling states “Two parallel tests were conducted on the subject screen systems and representatives of both Plaintiffs and Defendants were present. The test results revealed 100% accuracy of the equipment in reporting the vote selections”

Secondly, the ruling earlier stated that “The machines now challenged were tested as required by law prior to the early voting and election day voting and were found to be working properly”

If the defeated candidate had instead asked for a neutral third party to go over the code, they may have had more traction. As it is, because the defense claimed that the code was a valuable trade secret, the judge forced the plaintiffs to prove the machine was broken by testing a (probably small and statistically insignificant) number of votes on a (probably modified) demo machine in court.

Davi Ottenheimer • January 4, 2007 4:28 PM

@ Basil

well, even by your recount, it sounds pretty stupid to me. you forgot to mention that the judge dismissed the prosecution’s expert testimony as conjecture, but allowed what you called “small and statistically insignificant” as non-conjecture.

can you explain why isn’t it as stupid as it sounds?

Shane • January 4, 2007 4:35 PM

“there can be only one type of secret – the sinister type”.

I find that statement a curious thing to post on Bruce Schneier’s blog. I mean really… hehe. Honestly, I understood it to be in direct context to the prior statement regarding that specific software company’s reluctance to allow an audit of their source code, and I do agree that in such a scenario one would be hard pressed to find any benign arguments of relevance for concealing source code… but damn, what a thing to say, no?

paul • January 4, 2007 8:40 PM

Is the court accurately characterizing the results of the parallel tests? I know that the state claimed that their review of the videotapes of the testing showed that the miscounting that occurred in the parallel test was a result of user error, but I never saw an explanation or resolution of that claim. (Narrowly speaking, for example, it’s a user error if you touch the screen over A’s name and the machine selects B instead, and you don’t keep canceling the vote until it registers a vote for A. But it’s a user error that’s triggered by something fishy in the underlying code.)

Of course, the whole notion of trade secret protection for this kind of code is opposed to democracy, but even within its narrow premises this decision seems to jump to conclusions.

Matt from CT • January 4, 2007 9:10 PM

@Greg

You’re absolutely right we tolerate a tremendous amount of voter fraud in the U.S.

It’s nothing new — and traditionally was most heavily associated with Democratic Party “Machine Politics” although I’m sure the Republicans did some.

While many said it, Chicago Mayor Richard J. Daley was the most famous to say, “Vote Early & Vote Often!”

But such encouragement wasn’t in the context of Democrat v. Republican politics, but mainly internal Democrat party posturing in areas they were going to win overwhelmingly anyway.

Many if not all of the areas today associated with questions of the integrity of elections at least through the 1970s where strongly White, Democratic areas with a history of disenfranchising blacks and others. Since then they’ve switched to White, Republicans and I fear brought the chicanery to our party, too.

My area — the New England & New York — has used predominantly mechanical voting machines since the 1920s. At the time the Democrats were strong in the city machine politics, but the Republicans ruled the rural areas and State Legislatures. One way for the Republicans to check the machine politics power was to decrease the opportunity for voter fraud with stuffed or missing paper ballot boxes.

There’s other interesting ways the Rural Republicans would fight the City Democrats — Civil Service a prime example of de-politicizing a power base of the machine politicians. Even the fact that New York City’s fire department is called FDNY is a legacy of that — the Republican controlled State Assembly created the paid “Metropolitan Fire Department of New York” outside of the control of the City (only later was it made part of local government) to replace volunteer firefighting gangs run from Tamany Hall.

Unfortunately, in many parts of the country corruption of elections for internal party reasons became a common occurence, and now neither party in those areas really want it exposed.

Matt

Matt from CT • January 4, 2007 9:19 PM

BTW, I lived in what was until today the most heavily Democratic district represenated by a Republican.

With 242,000 votes cast in November, the initial victory margin for the Democrat was 167. The final margin was 91. Mostly minor math errors and the like, some interpretation of mail-in ballots that weren’t clearly marked. One town found a 100 vote error (shame on them, absolute shame) from mis-reading a dial and not having the other vote checker verify the first’s reading as they should have done. Minor errors are expected — a recount was mandatory under State Law since it’s recognized small errors in tabulating occur with a several hundred precints spread over 60 some Towns (each with their own election officials).

Some of us can run clean, tight elections without the aid of computers or questions of the integrity.

Davi Ottenheimer • January 4, 2007 9:31 PM

@ nedu

thanks for the link. interesting speculation and conjecture throughout this report.

in the conclusion, for example:

“Examination of the ballot images provides some clues as to voting patterns.”

er, some clues…

“The voters that tended to vote a Republican ballot were largely consistent with their Republican choices for county-wide races. Thus, voting patterns with respect to candidate preference does appear to be a factor that needs consideration in any statistical analysis of the 13th Congressional District race.”

ah, yes. tended to vote…largely consistent…does appear to be a factor that needs consideration…

but they move on to conclude unequivocally:

“the process of selecting one’s choices is not a measure of the voting device’s accuracy.”

say what? if a device confuses or otherwise leads a voter to choose something other than their intended candidate, that is not a measure of accuracy of a device? i would think so.

but they disagree, and say it only matters that the review screen is accurate. then they contradict themselves and say the process matters again “after making any desired changes to the vote selections.”

altogether that says to me that they believe the initial selection as irrelevant, but the next selection after review (without any further review) is relevant. oops, no?

and beyond that, there are numerous assumptions and alterations in the test that show they were testing one set of hypotheses about what might have been the cause, but they admit they were not in possession of a known process that led to documented errors.

this makes this whole test conjecture, unless i’m missing something here, and their tests may have been irrelevant to the errors experienced by real voters. page 3:

“Because documents describing voter complaints were not available for review, DOE relied solely on the published accounts bearing in mind that some of these accounts actually verified the voter’s acknowledgement [sic] to undervote the 13th Congressional District race.”

and again

“The ballot image file contains the voter selections as they appeared on the review screen at the time the voter pressed the
‘VOTE’ button. However, the arrangement of the ballot images is random. Therefore, these ballot images cannot be associated with the time that the ballot was cast.”

doesn’t that just say “we have a file with random images but with no way to recreate an experience or verify the actual vote by an actual person”?

no wonder they wanted to rule out the selection process as part of the review.

so the real conclusion of the test appears to be that if you know the voter’s intended vote (it’s scripted for them in advance) then you can resolve any anomalies later. in addition, it may be hard to figure out why people have problems with electronic voting systems by only collecting information from published reports. or, as they put it on page 8:

“There were no unresolved anomalies. In addition, attempts to replicate the published reports concerning voter difficulties in making or changing their vote selections did not materialize during this test.”

Thomas • January 5, 2007 4:18 AM

Is it just me, or are things starting to make absolutely no sense?

No, it’s just the fact that people who do not have an idea of technique decide about their reliability.

John Roberts • January 5, 2007 5:28 AM

Back here in Holland we had a test with those machines in a consument programm.
The outcome was that those machines had almost no security and were hackable/overwritable in about a minute. But the worst part was the machines were left almost unguarded in some warehouse and accessible to the undercovers from the programm.

C Gomez • January 5, 2007 8:04 AM

There isn’t any intellectual property involved in counting votes. Counting votes isn’t a serious scientific process. It’s adding. It’s not that hard. No judge should consider it intellectual property and should rule the source code to be open for review.

Voting “systems” are more about execution than they are about intellectual property. You win a voting system contract (all other things being equal) because you demonstrate you have the capability to execute on a large level on election day with minimal problems, and the ability to respond to those problems.

Really, the best thing we could do in the United States is ban any forms of electronic voting system on the voting side. All votes are cast on paper. Filling in bubbles or punching holes in non-perforated cards worked fine for a very long time, and would continue to work fine again. Obviously, some methods (filling in bubbles) tends to lead to easier judgment when manually viewing a ballot later than others (perforated cards are worthless, non-perforated could be problematic if voter isn’t paying attention).

You can use machines to tally the vote and then hand count if absolutely necessary.

Also, no more 19th century voting booths that you often see on the east coast, where no paper ballot is produced… you pull a bunch of switches and then pull some lever to have your vote “recorded”. Easy to see why these were popular. They were easily the “vote-stealers” of their day.

This isn’t a hard problem, and the simplest solution that is the least complex is probably the best one.

Scarybug • January 5, 2007 9:18 AM

I think it’s ridiculous that i.p. is involved in counting our votes in the first place. Votes should be secret, the mechanism that counts them should not. If companies are worried that the courts might let someone look at their i.p. in a disputed election, they shouldn’t be making voting machines in the U.S.

paul • January 5, 2007 9:19 AM

OK, so it is indeed an operator error not to catch and fix erroneous selections made by the machine. Good thing we don’t design airplanes that way.

derf • January 8, 2007 9:41 AM

The IP rights to the voting machines and software should be owned by the federal government and the source code made public.

Reader X • January 8, 2007 11:24 AM

The IP rights to the voting machines and software should be owned by the federal government and the source code made public.

Absolutely. Of course, according to nearly every software company, that’s tantamount to communism.

Ralph • January 8, 2007 5:25 PM

I don’t know what all the fuss is about.

Didn’t you read the license agreement when you bought the voting software?

pjp • January 9, 2007 4:34 AM

Don’t worry about it BS. Chaos never made sense, in a process of brute force/elimination of the worst approach all sorts of ways are tried. And someone has to set an example on what not to do. Best thing is to not get worked up too much over what makes sense and what doesn’t. In the end you’re right.

Leave a comment

Login

Allowed HTML <a href="URL"> • <em> <cite> <i> • <strong> <b> • <sub> <sup> • <ul> <ol> <li> • <blockquote> <pre> Markdown Extra syntax via https://michelf.ca/projects/php-markdown/extra/

Sidebar photo of Bruce Schneier by Joe MacInnis.