Author: James To and Maria Cuellar
University of Pennsylvania
January 2026
Introduction
A black box study to test the accuracy and reproducibility of forensic fingerprint examiners was performed in 2022, and the data on the participants’ responses was made publicly available with the article on the study:
Hicklin, R. A., Richetelli, N., Taylor, A., & Buscaglia, J. (2025). Accuracy and reproducibility of latent print decisions on comparisons from searches of an automated fingerprint identification system. Forensic Science International, 370, 112457.
Study design
The study included 156 latent fingerprint examiners (LPEs), each assigned 100 latent-exemplar image pairs (IPs) to evaluate. Of these 100 IPs per participant, 20 were mated pairs (originating from the same subject and finger), while the remaining 80 were non-mated (from different subjects). Participants were not informed of the proportion of mated to non-mated pairs.
By design, examiners could not skip or reorder the comparisons. We note that not all participants completed all assigned comparisons, and there was a participant submitted several responses under a duplicate registration, resulting in a total of 106 responses from this participant. This led to a total of 14,224 responses, among which there were 2,830 responses for mated IPs and 11,394 responses for non-mated IPs.
Figure 1. Study workflow of the LPE Black Box Study 2022. Image adapted from Hicklin et al. (2025).
Figure 1 illustrates the workflow of the study. On the other hand, the fingerprint dataset consists of 150 latent prints collected from 150 distinct fingers of 23 different subjects. Initially, the study design specified 150 mated IPs and 150 non-mated IPs. However, one latent was inadvertently included in two non-mated IPs instead of one mated and one non-mated as a result of a file naming error, leading to 151 non-mated IPs and 149 mated IPs in the final fingerprint data.
Quality metrics
In addition, print quality measures for the fingerprints used in this study can be obtained from the authors of Hicklin et al. (2025). These measures include latent print quality, measured by LQMetric1 and exemplar print quality, measured by NFIQ22. Samples of image prints used in the study are shown in Figure 2.
Figure 2. Examples of prints used in the study. Image adapted from Hicklin et al. (2025).
Data access
The data, codebook, and instructions on how to use it can be downloaded from Appendix A on the article website, in the journal Forensic Science International.
1 Kalka, N. D., Beachler, M., & Hicklin, R. A. (2020). Lqmetric: A latent fingerprint quality metric for predicting afis performance and assessing the value of latent fingerprints. Journal of Forensic Identification, 70 (4), 443–463.
2 Tabassi, E., Olsen, M., Bausinger, O., Busch, C., Figlarz, A., Fiumara, G., Henniger, O., Merkle, J., Ruhland, T., Schiel, C., et al. (2021). Nfiq 2 nist fingerprint image quality.