More on Voodoo Science

Karen Franklin reports a “guest” piece, by Prof. Mnookin, Professor and Vice Dean, UCLA Law School. See actually, Jennifer L. Mnookin, Clueless ‘science,’LA Times, 19 February 2009.

The article is about the congressionally funded National Research Council report on forensic science (look to my earlier posts). Several points that have been on my mind for some years.

Bias: Doctors testing a new medicine are — appropriately — not told which patients receive placebos and which get the test medication, because that knowledge might unconsciously bias their behavior and observations. But forensic scientists are frequently exposed to information that can potentially foster bias. Crime laboratories have failed to create adequate procedures for making testing “blind.”

Why should this one be too hard for USACIL, or DCFL, or NCIS? All of the Services have set up a “blind testing” program without problems — yes, that’s right, if they can do it for urinalysis testing, why not other tests. Isn’t one of the stated reasons for blind testing in the urinalysis program to ensure reliability?

Error rates: Most of the forensic disciplines lack good information about how often practitioners make mistakes, a basic requirement of any science. Not knowing the error rate is bad enough, but some experts consistently testify under oath that their technique has an error rate of zero, an inherently preposterous claim.

I’ve yet to have a military crime lab examiner tell me the overall error rate for their lab, or for their field at that particular lab, such as biological testing, or for themselves as individual examiners. And don’t expect to get that in discovery either (a point I find interesting because they claim ASCLAD certification). Again, where can we look within the military for examples of doing this aspect of scientific reliability — yep, the drug labs. Don’t we go throught that ad-nauseum in urinalysis cases? This is even before we get to issues about the underlying reliability of the “science” involved.

Structural independence: Here in Los Angeles, the city crime laboratory is part of the Police Department, and the county’s lab reports to the sheriff. These kinds of arrangements are typical. But when the police and prosecutors pay and supervise the scientists, it stands to reason that the scientists may have difficulty establishing their independence.

Need I say more. Just look at the wiring diagram for where USACIL sits, and the drug labs, etc.

Professor Mnookin is co-author:

D. H. Kaye, David E. Bernstein, Jennifer L. Mnookin, and Richard D. Friedman, The New Wigmore: A Treatise on Evidence : Expert Evidence , Aspen Pub. 2003.