The recent pre-publication draft of the PCAST report on forensic science describes faults and fixes for many forensic disciplines, including DNA, firearms, fingerprints, impression evidence, and bitemarks. Numerous stakeholders have already criticized the report on a variety of points, including that no forensic experts were on the committee, the committee ignored large amounts of research validating the methods, and that the methods have all be accepted in courts for years. Criticism came from the National District Attorneys Association, the International Association of Chiefs of Police, and professional forensic associations, like the International Association for Identification (IAI). That prosecutors and police would cry foul of such statements only makes sense: They see some of their forensic bread-and-butter being tossed aside. Such polemics from them are expected and can be construed politically. A few statements, however, do merit some reflection and concern.

The NDAA, in their press release, say,

The determination of whether scientific evidence is reliable and therefore admissible must remain with the judiciary — while the determination of how much weight will be given to any particular piece of scientific evidence must rightfully remain with the jury.

If judges are the arbiters of what constitutes science and juries determine whether science is significant, why is the scientist there at all? What weight does the expert–the discipline, even–provide to the proceedings? From a legal perspective, one can see how this statement makes sense. From a scientific viewpoint, it is extremely troubling: Scientists are ancillary to the process of judging and interpreting their area of expertise. Science is evaluated on its usefulness to the case, not on actual validity as science.

What’s also interesting is the response from the IAI, a body that represents and certifies footwear examinations. On that point, the IAI specifically said,

The opinions expressed by PCAST…disregard large bodies of scientific evidence, demonstrate a lack of understanding of the discipline of footwear evidence examination and include statements that appear biased and that are not supported by the facts and/or research. The process included minimal effort to involve the practitioner community in the process. 

The draft report lists additional experts consulted, which includes over 30 (depending on how you want to count) forensic experts from private, state, and federal forensic entities; several of them were footwear experts. They surely provided relevant literature to the Council for its deliberations.

Sadly, this tends to be the response of forensic science to criticism: We know what we’re doing, so leave us alone. Witness the fingerprint community’s response to attacks on individualization, 0% error rates, and cognitive bias; the IAI’s statement is similar. These complaints sound like the prosecutors’ and police responses, not those of scientists seeking to improve their methods and outcomes. This response should not be surprising given that the forensic profession grew up under the care, feeding, and protection of police and the courts. What should also not be surprising, especially to the forensic community, is that this change was coming for some time.