A Multi-Spectral Structured Light 3D Imaging System MATTHEW BEARDMORE MATTHEW BOWEN

Origins of our Project Freshman Imaging Project 2011 Tasked with creating a 3D imaging system over three quarters Presented at ImagineRIT 2012 Contour distances along a person’s face gives information about the structure of that person’s trachea Uses technique known as Structured Light to scan subjects

A brief primer on digital imaging Source: Digital Photography Presentation (Jeff Pelz, Joe Pow)

How are digital images captured? Light rays pass in through an aperture Those rays are focused onto the sensor by a lens The sensor segments the light into individual boxes, known as pixels Each pixel interprets the intensity of the light striking it as a numerical value A closeup of an imaging sensor Source: Digital Photography Presentation (Jeff Pelz, Joe Pow) Individual pixel element

What is Structured Light? A 3D scanning technique Involves projecting a series of known patterns onto a subject A camera interprets the distortions in the patterns and calculates depth The series of patterns creates a temporal code for each pixel Each projected pixel is uniquely identified by this code

What is Structured Light? (continued) The camera can detect where each projected pixel falls upon the subject Interprets each pattern as a part of a temporal code Combines each part at the end of the scan to reconstruct the temporal code Depth can then be calculated Triangulation between camera and projector projector camera projected light ray reflected light ray object being scanned p Diagram source: Structured Light: The Mathematics of 3D Triangulation Presentation (Gabriel Taubin, Douglas Lanman)

ImagineRIT 2012 Prototype

360° Scanning – Obstacles Simultaneous scanning with black and white can cause the scanners to interfere with each other Left projector onlyRight projector onlyBoth projectors simultaneously

360° Scanning – Utilizing multiple spectra Left projector displaying green Right projector displaying red Camera with green color filterCamera with red color filter Instead of black and white scanning, different colors can be assigned to each camera-projector pair and isolated using color filters

Our goals Extend the original FIP2011 prototype to four cameras and projectors Allows for 360° + overhead scanning of subject (2π steradians) Utilize red, green, and blue portions of the visible spectrum Allows for simultaneous projection, keeping scan times as short as possible Decrease the overall scan time of our system Assists in the scanning of subjects, as subjects will move less

First attempt Three color cameras, each with a Bayer filter on the sensor Separate red, green, and blue channels into intensity maps (grayscale) No color filters Suboptimal quality – projectors not displaying precise colors E.g. displaying green would cause projector to display a small amount of red

Use of color filters Instead of raw Bayer filter, color filters were placed in front of camera and projector Restricts projector output and camera input to only desired wavelengths Significantly improved scanning results with three camera-projector pairs Very little interference between scanners

Adding an overhead scanner Red, green, and blue make up primary colors of light What color should the fourth scanner use? Yellow was chosen due to its distance away from any primary color relative to other secondary colors Significant interference due to breadth of wavelengths that the red and green filters cover Yellow wavelength overlaps too much with red and green to be viable

Summer 2012 Prototype

Results PUT PHOTO HERE

Status of goals Successfully implemented a multi-spectral scanner that uses red, green, and blue to achieve 360° scanning Scan time halved from ~8 seconds to ~4 seconds Addition of fourth color – yellow – not successful Colors filters not narrow enough to be useful for four scanners To achieve the goal of scanning a subject with the overhead view, the last camera-projector pair would not be able to scan simultaneously

What we’ve learned Matching the output of the projector to color filters is difficult Narrow matches are optimal Grayscale cameras paired with color filters provide optimal contrast and resolution As opposed to the color (Bayer) cameras used in our project

Acknowledgements Chester F. Carlson Center for Imaging Science Joe Pow, Advisor Maria Helguera, Advisor Stefi Baum Class of Freshman Imaging Project 2011 Gabriel Taubin and Douglas Lanman, Brown University