Face Detection from ATIS Spiking Output

Face Detection Task: “Is there any part of a face present or not? (not trying to localize face or identify person) NO YES NO YES

ATIS data collection Training data: 4 subjects (static cam, unoccluded face) Test data (face present) – Hand-drawn cartoon faces (slightly-moving cam, unoccluded face) – Slightly-moving cam, unoccluded face – Moving cam, face occluded sometimes Test data (face absent) – 6 hand gestures – Person walking along school corridor – Dot pattern on kitchen whiteboard – Pool game

Training / testing procedure Chop all data (training and testing) up into short segments of 5000 spikes – Primarily for practical reasons and convenience Train on 4 subjects’ data separately – Static camera, subject’s head is rotating/moving Test on various data – Vary detection threshold to produce ROC curve (True Positive rate vs. False Positive rate) – Primary metric is AUC (area under ROC curve)

Face detection algorithm (briefly) Use Garrick’s HFirst algorithm as starting point – S1 layer: oriented Gabor filters over spike input – C1 layer: local position invariance Training: – Run training “images” (all containing faces) up to C1 – Extract face templates (regions of C1 spike patterns) Testing: – Run test “image” up to C1 – Match C1 spike patterns to templates – If # matches exceeds threshold, then face is present

Results (1 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: cross-testing on the other 3 subjects (static cam, unoccluded face) – Face-absent data: hand gestures, pool game, corridor, etc. Objective: test generalization to untrained subjects Sample training segment Sample test segment Sample test segment (face absent)

Results (1 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: cross-testing on the other 3 subjects (static cam, unoccluded face) – Face-absent data: hand gestures, pool game, corridor, etc. Objective: test generalization to untrained subjects Results: mean AUC = Trained on Subject C.T. Trained on Subject G.O. Trained on Subject H.A. Trained on Subject M.M.

Results (2 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: hand-drawn cartoon faces (slightly-moving cam, unoccluded face) – Face-absent data: same as previous Objective: test generalization to artificial faces Sample training segment Sample test segment Sample test segment (face absent)

Results (2 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: hand-drawn cartoon faces (slightly-moving cam, unoccluded face) – Face-absent data: same as previous Objective: test generalization to artificial faces Results: mean AUC = Trained on Subject C.T. Trained on Subject G.O. Trained on Subject H.A. Trained on Subject M.M.

Results (3 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: untrained subjects (slightly-moving cam, unoccluded face) – Face-absent data: same as previous Objective: test generalization to moving cam Sample training segment Sample test segment Sample test segment (face absent)

Results (3 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: untrained subjects (slightly-moving cam, unoccluded face) – Face-absent data: same as previous Objective: test generalization to moving cam Results: mean AUC = Trained on Subject C.T. Trained on Subject G.O. Trained on Subject H.A. Trained on Subject M.M. (Not sure why AUC is so good!)

Results (4 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: untrained subjects (slightly-moving cam, faces occluded sometimes) – Face-absent data: same as previous Objective: test generalization to moving cam, occlusions Sample training segment Sample test segment Sample test segment (face absent)

Results (4 of 4) Training: 4 subjects (static cam, unoccluded face) Testing: untrained subjects (slightly-moving cam, faces occluded sometimes) – Face-absent data: same as previous Objective: test generalization to moving cam, occlusions Results: mean AUC = Trained on Subject C.T. Trained on Subject G.O. Trained on Subject H.A. Trained on Subject M.M.

Limitations Dataset is small and probably not too difficult Accuracy – Nowhere close to state-of-the-art for face detection in grayscale images – The real region of interest is FP rate << 10%; our TP rate there is only ~70% or lower Time – Far from real-time: takes a few seconds to process each segment (containing 5000 spikes)

End