2. Visual Processing in Fingerprint Experts and Novices
Tom Busey
Indiana University, Bloomington
John Vanderkolk
Indiana State Police, Fort Wayne
Expertise with fingerprint examiners was tested in behavioral and EEG studies. Experts show greater tolerance for noise, are unaffected by longer memory delays, and show evidence of configural processing. This last finding was confirmed in an EEG study where experts show a reliable delay of the N170 component when fingerprints were inverted, while novices did not. Configural processing may be one element that underlies perceptual expertise.

3. How Do Experts Make Identifications?
Easy Match
Hard Match

4. What Cognitive Abilities Support Expertise?
Learn relevant features or dimensions
Recognize new clusters of features (unitization)
Robustness to noise
Tune detectors to specific characteristics of features (exclude noise)
Integrate information over larger regions of space
Learn to trade off quantity and quality in decision making
Superior visual memory
Tolerate affine transformations such as rotation and stretch

6. Study Fragment
one second

7. Mask
Either 200 ms or 5200 ms

8. Test Images
Until Response

9. Testing Fingerprint Expertise: X-AB Sequential Matching Task
example stimulus pairs:

10. Reduce Matching based on Low-Level Features
Overall Brightness change
Study image is rotated up to 90° in either direction
Two image manipulations designed to simulate latent prints
Added noise
Partial masking

11. Added Noise

12. Partial Masking

13. Partial Masking Semi-Transparent Masks Fingerprint Partially Masked
original
inverse
Logical
Combination
Recovers Original
Fingerprint
Semi-Transparent
Masks
Fingerprint
Partially Masked
Fingerprints

14. Includes combinations:

15. Image Degradations at Test

16. Partial Masking Semi-Transparent Masks Fingerprint Partially Masked
original
inverse
Summation
Recovers Original
Fingerprint
Semi-Transparent
Masks
Fingerprint
Partially Masked
Fingerprints

21. Partial Images in Noise
Behavioral Data
Full Images
Partial Images
Full Images in Noise
Partial Images in Noise
Experts: No effect of delay, interaction between noise and partial masking.

22. Logic of Partial Masking
One Half
Both Halves
One Half
Logical
Combination
Recovers Original
Fingerprint
Partially Masked
Fingerprints

23. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
Correct Decision
d = likelihood of getting enough information from half of a print
      in order to make a correct decision
db: d when both halves are present

24. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
yes
(db) no
(1-db)
Correct Decision
d = likelihood of getting enough information from half of a print
      in order to make a correct decision
db: d when both halves are present

25. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
yes
(db) no
(1-db)
info from guessing?
yes
(g) no
(1-g)
Correct Decision
Wrong Decision
d = likelihood of getting enough information from half of a print
      in order to make a correct decision
db: d when both halves are present
g = getting the correct answer by guessing (high threshold model)

26. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
info from first half?
yes
(db) no
(1-db)
yes
(do) no
(1-do)
info from guessing?
info from guessing?
yes
(g) no
(1-g)
yes
(g) no
(1-g)
Correct Decision
Wrong Decision
Correct Decision
Wrong Decision
d = likelihood of getting enough information from half of a print
      in order to make a correct decision
db: d when both halves are present
do: d when only one half present
g = getting the correct answer by guessing (high threshold model)

27. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
info from first half?
yes
(db) no
(1-db)
yes
(do) no
(1-do)
info from guessing?
info from guessing?
yes
(g) no
(1-g)
yes
(g) no
(1-g)
Correct Decision
Wrong Decision
Correct Decision
Wrong Decision
Question: What is the relation between db and do?
db = do : One half doesn't influence information acquired from other half
db < do : Get less information from one half when second is present
db > do : Get more information from one half when second is present (consistent                 with configural or gestalt processing)

28. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
info from first half?
yes
(db) no
(1-db)
yes
(do) no
(1-do)
info from guessing?
info from guessing?
yes
(g) no
(1-g)
yes
(g) no
(1-g)
Correct Decision
Wrong Decision
Correct Decision
Wrong Decision
Testing:
Fit reduced model with db = do. Can we reject this model?
If we reject it, look at the relation of db to do by fitting the full model.
db >> do is consistent with configural processing.

29. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
info from first half?
yes
(db) no
(1-db)
yes
(do) no
(1-do)
info from guessing?
info from guessing?
yes
(g) no
(1-g)
yes
(g) no
(1-g)
Correct Decision
Wrong Decision
Correct Decision
Wrong Decision
Experts:
No noise: reject reduced model, db and do are significantly different
Full model: db = .841, do = wrong direction for configural processing
In noise: reject reduced model, db and do are significantly different
Full model: db = .50, do = consistent with configural processing

30. Multinomial Tree Modeling
Full Image (Both Halves)
Partial Image (One Half)
info from first half?
yes
(db) no
(1-db)
info from second half?
info from first half?
yes
(db) no
(1-db)
yes
(do) no
(1-do)
info from guessing?
info from guessing?
yes
(g) no
(1-g)
yes
(g) no
(1-g)
Correct Decision
Wrong Decision
Correct Decision
Wrong Decision
Novices:
No noise: reject reduced model, db and do are significantly different
Full model: db = .40, do = wrong direction for configural processing
In noise: can't reject reduced model, db and do are not significantly different
Full model: db = .19, do = No evidence for configural processing

31. Evidence for Configural Processing: Multinomial Modeling
To test for configural processing, we can use the accuracy rate in the partial image condition to make a prediction for the full image condition, assuming no configural processing. If performance in the full image condition exceeds the prediction, we have evidence that is consistent with configural processing.

32. Evidence for Configural Processing: Multinomial Modeling
To test for configural processing, we can use the accuracy rate in the partial image condition to make a prediction for the full image condition, assuming no configural processing. If performance in the full image condition exceeds the prediction, we have evidence that is consistent with configural processing.
Experts in noise: We predict performance in the full image condition to be about 75% correct. Instead it is around 90%. Experts are doing better with the whole image than we predict they would do based on partial-image performance. This is configural processing at work.

33. Configural Processing in Faces: The ‘Thatcher Illusion’
Features are perceived individually, image looks ok.
(Thomson, 1980)

34. Configural Processing in Faces: The ‘Thatcher Illusion’
Features are perceived individually, image looks ok.
Features are perceived in context, image looks grotesque.
(Thomson, 1980)

35. Configural Processing in Faces: The ‘Thatcher Illusion’
Features are perceived individually, image looks ok.
Features are perceived in context, image looks grotesque.
(Thomson, 1980)

36. Configural Processing in Faces: The ‘Thatcher Illusion’
Features are perceived individually, image looks ok.
Features are perceived in context, image looks grotesque.
(Thomson, 1980)

37. EEG Recording Basics Record from the surface of the scalp
Amplify 20,000 times
Electrical signals are related to neuronal firing, mainly in post-synaptic potentials in cortex.
Very small signals, very noisy data.

38. EEG Recording Basics
Average over lots of trials (200 trials per condition)

39. EEG and Configural Processing
Faces produce a strong component over the right hemisphere at about 170 ms after stimulus onset, which is called the N170. Inverted faces cause a delay of ms in the N170.
Trained objects (Greebles) show a delay in the N170 component with inversion, but only in the left hemisphere (channel T5).
Data from Rossion, Gauthier, Tarr, Despland, Bruyer, Linotte & Crommelinck (2000)
fMRI studies show IT is active whether attending to faces or not (Tarr)
No effect of familiarity (Bentin & Deouell, 2000)
target status: Is the face-sensitive N170 the only ERP not affected by selective attention? (Caquil, Edmonds, & Taylor, 2000)
Appears to be feed-forward perceptual processing of faces or other face-like stimuli
Coupled with behavioral data suggesting configural processing with faces, an advanced N170 to an upright stimulus suggests that the N170 latency differences indicate configural processing.
Data from Rossion, Gauthier, Goffaux, Tarr & Crommelinck (2002)

40. An Obvious Experiment:
Show upright and inverted fingerprints to Fingerprint examiners and novices. If experts process fingerprints configurally, we should see a delayed N170 to inverted fingerprints.
fMRI studies show IT is active whether attending to faces or not (Tarr)
No effect of familiarity (Bentin & Deouell, 2000)
target status: Is the face-sensitive N170 the only ERP not affected by selective attention? (Caquil, Edmonds, & Taylor, 2000)
Appears to be feed-forward perceptual processing of faces or other face-like stimuli
Also test faces to replicate the face inversion effect in our subjects. Test both identification and categorization tasks.

49. Summary of Experiment
Fingerprint experts demonstrate strong performance in an X-AB matching task, robustness to noise and evidence for configural processing when stimuli are presented in noise. This latter finding was confirmed using upright and inverted fingerprints in an EEG experiment. Experts showed a delayed N170 component for inverted fingerprints in the same channel that they show a delayed N170 for inverted faces. Thus they appear to be processing upright fingerprints in part using configural or holistic processing, which stresses relational information and implies dependencies between individual features. In the case of fingerprints, this may come from idiosyncratic feature elements instead of well-defined features such as eyes and mouths.