Lying with Statistics and Research.

For many years I’ve had a small book on my shelf called How to Lie with Statistics first published in 1954. I see that it is now in a 1993 paperback edition, with pictures.

Darrell Huff, How to Lie with Statistics, W. W. Norton & Company (September 1993).

I am reminded of Mr. Huff’s book by this entry on the Crime & Consequences blog which I follow for interesting news and usable information. I’ll quote and paraphrase extensively from the entry so you’ll understand why I am reminded of Mr. Huff. Here is Kent Scheidegger’s, The Incredible Growing Statistic.

The headline in USA Today reads, “FBI: Burgeoning gangs behind up to 80% of U.S. crime.”

The story actually says, “Criminal gangs in the USA have swelled to an estimated 1 million members responsible for up to 80% of crimes in communities across the nation, according to a gang threat assessment compiled by federal officials.” That could be read the way the headline-writer read it, but maybe not.

The FBI’s press release for its 2009 National Gang Threat Assessment says, “Criminal gangs commit as much as 80 percent of the crime in many communities, according to law enforcement officials throughout the nation.” Ah, “as much as” and “many communities,” meaning this is the peak estimate for the most gang-infested communities, a far cry from the national total implied by the USA Today headline.

Turning to the full document, and skipping past the dumbed-down summary and key findings (which are the same as the press release), what to our wondering eyes should appear but this:

According to local law enforcement information, gang members are responsible for as much as 80 percent of the crime in some locations.5————————————-5. Stated percentage is based solely on self-reporting by local law enforcement. The figures given are not meant to represent an National Drug Intelligence Center (NDIC) analysis.

So, not only is the estimate for “some locations” and not nationwide, as the USA Today’s headline indicates, it is expressly disclaimed as a federal government estimate in the very source the paper cites as a federal government estimate.

The moral of the story is, don’t believe anything you read about the results of a study until you have gone into the text and checked it out. With each step removed from the original, the results grow simpler, more clear-cut, less nuanced, and more ominous.

As we consider the outrage and doom and gloom about crime in the military, we should consider Mr. Huff’s homily and Mr. Scheidegger’s exhortation to read the underlying reports — closely.