Validation is Knowing Where to Stop

Scientific validation testing does not simply establish whether a method works. Done effectively, it defines where the method stops being valid. It tells us where to stop.

My first cases were not about abstract failures of the criminal legal system. I started my legal career working for Barry Scheck and Peter Neufeld, co-founders of the Innocence Project. The people at the center of those cases had already lost years of their lives to wrongful convictions and wanted to understand how it had happened.

We sued municipalities and police departments over the misconduct that produced those convictions. Over time, I became particularly focused on cases involving forensic failure. That experience changed the way I think about a word that can sound painfully technical: validation.

The forensic scientist John Butler has described validation, in essence, as "push[ing] the system until it fails." You test the method at its boundaries. You learn where it works, where it becomes unreliable, and where the science tells you to stop.

That last part matters.

A validated method does not simply tell us what we can do. It tells us what we cannot responsibly claim.

I have spent much of my career watching good people resist that kind of boundary-setting. Not villains. Not people trying to convict the innocent. Good people pursuing what they thought of as public safety, technological progress, or simply the belief that a useful tool should be used.

But the faces burned into my memory are the people who lived on the other side of those judgments. People damaged, and sometimes destroyed, when confidence outran knowledge.

Institutions are very good at finding reasons to use a tool. They are much less practiced at

deciding not to. In a system with the power to surveil, arrest, prosecute, imprison, and take life, validation is one of the disciplines that is supposed to impose that restraint.

Validation is therefore more than a laboratory exercise. It is a way of drawing defensible boundaries around what we know. It asks: 

Where does this method stop working?

Under what conditions?

How do we know?

And will we actually stop when we reach that line?

That is where the science becomes a question about institutions, power, and constitutional government. Because knowing the limits is only useful if we are willing to honor them.