When a Facial Recognition “Result” Can Be Edited

A facial recognition result can look like a fixed algorithmic output. But some systems allow analysts to alter the candidate list before the result is preserved or disclosed.

Here’s something about facial recognition that deserves much more attention: the results list can be edited.

When we call something a forensic “result,” what exactly are we talking about? When that result is generated by an algorithm, we tend to assume it means what the algorithm actually produced. In facial recognition, that means the ranked candidate list generated when a probe image is searched against a photo gallery.

Clare Garvie and others have already shown that facial recognition analysts can and do edit probe images before running a search.

But it turns out that the results list is not necessarily fixed either.

When facial recognition use is disclosed at all, the discovery I see is remarkably consistent: a “potential match notification” report and a screenshot showing roughly the first 13 candidate photos returned by one or two algorithms. I’ve consulted on facial recognition cases across multiple jurisdictions. I have never seen anything different. And I’ve seen courts treat that screenshot as the facial recognition result, denying requests for anything more.

But the DataWorks Plus FACE Plus user manual says an analyst can remove an image “from all result sets.” There is a “Remove” link beneath each candidate photo. According to the manual, once selected: “The image will no longer be displayed in the result set.”

DataWorks Plus’s FACE Plus user guide instructs analysts how to remove an image from a facial recognition result set. The manual states that the removed image “will no longer be displayed in the result set.” Source: DataWorks Plus FACE Plus Case Management User Guide, p. 53.

I have seen that “Remove” link redacted or cut off in discovery screenshots.

The software apparently allows removed images to be restored. That is not the point. The point is that the results list itself is editable. And the screenshot we are getting in discovery does not say when it has been edited.

Nor is rerunning the search later an adequate substitute for preserving the original result. Facial recognition candidates are ranked against the gallery that exists at the time of the search. Galleries change as images are added or removed, and the deployed algorithm, software version, or system configuration may change as well. A later search therefore cannot be assumed to reproduce the candidate list that the original search returned.

So when we receive a screenshot showing 13 candidates, what are we actually looking at?

The candidates the algorithm originally returned?

Or the candidates that remained when someone took the screenshot?

Those are not the same thing.

More critically in many cases, is the first candidate actually the image from the gallery with the highest similarity score?

Or did a human make that candidate appear first in the screenshot by removing higher-ranked candidates from the displayed result set?

If the preserved record of a facial recognition search is a screenshot of an editable screen, prosecutors, defense attorneys, courts, auditors, and the public should not assume that screenshot shows what the algorithm originally returned.

Before we ask whether a facial recognition result is reliable, there is a more basic question: is this even the original facial recognition search result at all?