Extraordinary Claims, Ordinary Evidence...

The phrase is attributed to Marcello Truzzi, founding co-chairman of the Committee for the Scientific Investigation of Claims of the Paranormal (CSICOP):

“An extraordinary claim requires extraordinary proof.”

Truzzi’s quote, from his work, On the Extraordinary: An Attempt at Clarification, Zetetic Scholar, Vol. 1, No. 1, p. 11, (1978) echos ideas developed much earlier by various metaphysical philosophers. In his 1832 paper Théorie Analytique des Probabilités, Laplace wrote: “The weight of evidence for an extraordinary claim must be proportioned to its strangeness.” In his An Enquiry Concerning Human Understanding (1784), David Hume wrote: “A wise man… proportions his belief to the evidence,” and “No testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous than the fact which it endeavors to establish.”

But it’s 2016. All that old sk00l thinking is… well… antiquated. In an era where 24-hour news isn’t so much about “facts” as it is about “engagement,” the world of Laplace, Hume and Truzzi is nothing but an irrelevant memory of a more naïve time.

And just when you thought that the American election process couldn’t get stranger, it does.

Much stranger.

This week, serious people decided to throw Truzzi’s maxim out the window and just go with whatever the hell sounded plausible. “Extraordinary proof” is so frickin’ boooooring.

Perhaps it’s just me, but the theory that Russia is attempting to overtly influence the outcome of American elections seems to rise to the level of an “extraordinary claim.” Sadly, I think that fact has managed to somehow become culturally irrelevant.

As for the evidence, I’ve seen nothing so far that rises to a level that gets it past the whole “beyond a reasonable doubt” standard that I think the media used to strive to surpass. It’s filled with weasel words (ex. “likely,” “highly probable”) and - even more damning - suffers from some unflatteringly transparent contradictions (ex. self-congratulatory phraseology describing the investigation’s ability to quickly work their way past the “superb tradecraft” of the intruders while simultaneously pointing out their apparent n00b-like “blunders” - leaving metadata in edited files, etc…)

There is an almost willful blind-spot on the part of the “investigators.” It’s almost as if you can hear them say: “Look at all of these blunders by the bad guys - it just goes to prove that we’re so much smarter, taller, and better looking than them… False flags? No. These couldn’t possibly be false flags. We know false flags when we see them (and we’re lookin’ at you, Guccifer 2.0). We’re so smart.”

Having been incorporated into the mainstream political discourse, this is pretty much a done deal. Russia did it. No questions allowed.

But I have questions.

Lots of questions.

Questions that can only be answered by the extraordinary proof that we’re never going to get.

-TL Tom Liston
Owner, Principal Consultant
Bad Wolf Security, LLC
Mastodon: @tliston@infosec.exchange
Twitter: @tliston
July 27 2016

Addendum (July 28, 2016):

My friend Chris Sanders (@chrissanders88) asked me to “cite specific examples” of where I feel the evidence is lacking. Full disclosure: Chris works for Mandiant/FireEye. (I believe it is important to also note that, despite my constant urging, he has been unable to convince them that they should change the company name to FireAnt.) So… here you go Chris. I hope you’re sitting down - and while I’m happy to oblige, it’s important to remember, onus probandi incumbit ei qui dicit, non ei qui negat i.e. it isn’t MY job to make a case… ‘cause I’m not the one making the claim.

Note: Chris wasn’t saying I was wrong… he was just pushing me to develop my argument further. Also, I’m not saying that the overall conclusion (Russia did it) is wrong - I’m just saying that this is a lousy way to make an argument and, therefore, it teaches people to accept lousy arguments.

For the most part, the techinical details making the case that the DNC intruders are Russian aren’t even available (see #1 below). So while I can point out problems in the details that ARE available, the bulk of my rebuttal will be based on pointing out formal logic errors in the conclusions being drawn.

1. First of all, we need to get this out of the way: Currently, the concept of “attack attribution” on the Internet has been slapped around and then held down and forceably violated by “OpSec.” Beginning with the Sony breach, we’ve seen several high profile “attack attributions” chucked to the media by shadowy agencies who can only manage to say “X did it” before tagging on “we can’t tell you HOW we know… OpSec.” Here’s the bottom line on that bullshit: Look up at the top of this blog post - “An extraordinary claim requires extraordinary proof.” If you’re going to make an extraordinary claim (“The Russians are overtly attempting to influence U.S. elections”) then you MUST provide rock-solid, definitive proof. NO EXCEPTIONS. Otherwise, keep your Goddamned mouth shut. Even if you KNOW FOR CERTAIN THAT SOMETHING IS TRUE if you’re not going to provide us with proof, STFU.
   • NSA/FBI/CIA: Stop this crap RIGHT NOW. Argumentum ab auctoritate wasn’t cool back when our parents told us “because I said so,” and it certainly isn’t cool in this context. You’re indoctrinating the public with an incredibly bad habit (“accepting assertions of authority”) and you need to stop.
2. CrowdStrike claims the attackers are highly sophisticated (“Their tradecraft is superb, operational security second to none…”) and yet we’re - at the same time - supposed to believe that the intruders were so stupid that they “accidentally” left metadata in released documents showing that the file was edited on a machine configured with Russian language settings by someone with a Cyrillic username (Феликс Эдмундович - “Iron Felix”) that references the dearly departed Felix Dzerzhinsky, one of the founders of the Soviet Secret Police. Another leaked document included hyperlink error messages in Cyrillic, the result of editing the file on a computer with Russian language settings. Sorry, but you can’t have it both ways: either the intruders have OpSec “second to none,” or they are the Moe, Larry, and Curly of cyber-attackers. CHOOSE.
3. One of my favorite conclusions: After the “edited-on-computers-with-Russian-language-settings” cockup became public, “the intruders removed the Cyrillic information from the metadata in the next dump and carefully used made-up user names from different world regions, thereby confirming they had made a mistake in the first round.” Or… not. Perhaps - just perhaps - you’re choosing to see “confirmation” of your foregone conclusions in the actions of an adversary attempting obfuscation. Perhaps you’re seeing EXACTLY what they want you to see.
4. Follow this chain of logic: Guccifer 2.0 claims to be Romanian. Guccifer 2.0 cannot speak colloquial Romanian. Therefore Guccifer 2.0 is Russian. (Technically, this is a fallacy of four terms or Quaternio terminorum in Latin.)
5. The intruders used the same command and control infrastructure that the Russians have used in the past, therefore they MUST BE THE RUSSIANS! Uh… no. They used hacked systems on non-Russian networks for C & C. Of course it would be utterly impossible for anyone else to use those hacked systems - they’re so… er… um… secure… nevermind.
6. Finally, and this is my favorite, see if you can find the formal logical flaw here: The DNC attackers used tools A, B, and C and methods X, Y, and Z. Russian attackers use tools A, B, and C and methods X, Y, and Z. Therefore, the DNC attackers are Russian. (Hint: It falls into the same category as this: Steve is carrying a purse. Women carry purses. Therefore, Steve is a woman.)

So if your biggest, baddest “proof” (#6 above) rests on a logical flaw (and FYI, it’s known as the “fallacy of the undistributed middle” or in Latin, non distributio medii) then really: What frickin’ proof do you have?

Addendum II (July 30, 2016):

I’ve had several people ask me, “Well, if the Russians didn’t do it, then who did?” Unfortunately, I think you’re missing the point here. I’m NOT SAYING that I disagree with the conclusion - I have a problem with the fact that there IS a conclusion with no real evidence to back it up. I’m saying is NO ONE HAS PROVEN ANYTHING, so let’s stop going around acting like they have. The media has taken this particular ball and uncritically run with it: AND THAT’S A PROBLEM.

This rant isn’t about Russia. This rant isn’t about hacking or attribution. This rant is about people being taught to accept extraordinary claims without evidence. People shouldn’t be willing to accept claims like this without adequate and compelling proof. Encouraging this type of behavior is dangerous and WILL have consequences.