Lecture outline Basics Spectral image Spectral imaging systems Applications Summary Lecture material by: Markku Hauta-Kasari, Kanae Miyazawa, Jussi Parkkinen, Timo Jääskeläinen, Jouni Hiltunen, Joni Orava, Hannu Laamanen, Jarkko Mutanen

Spectral measurement

Human cone sensitivities

Chicken cone sensitivities

Spectral approach to color In spectral approach, color is represented by color signal. This causes the color sensation The signal is part of electromagnetic spectrum - in human color vision the range is nm In spectral approach, we are not limited into this human visual range

Motivation for spectral color Not to loose important color information (To avoid the problem of metamerism) To define optimal color sensors To develop better color vision models To develop novel instruments To develop spectral color classifiers and optical implementations for them

Outline Basics Spectral image Spectral imaging systems Applications Summary

Component images of spectral image

Spectral Image

Spectra from leaves in previous image

MEMORY REQUIREMENTS OF IMAGES Image size256x x512 gray-level image 65 kb 262 kb color (RGB-) image 196 kb 786 kb spectral, 20 nm resol. 1 Mb 4 Mb spectral, 5 nm resol. 3 Mb 15 Mb

Definitions Spectral image An image, where each pixel is represented by a spectrum Hyperspectral A term used for spectral images with large number of spectral components RGB-image spectrum in visible region, three components Multispectral

Image Types TYPE SPECTRAL COMPONENTS Gray-scale Trichromatic Spectral –Hyperspectral Real-time spectral Single Three >3 Numerous

Outline Basics Spectral image Spectral imaging systems Applications Summary

Spectral Imaging Devices Spectral cameras –filter wheels –light filtering –scanning systems –multi-band detectors

Optical principles Narrow band filters –interference filters, LCTF, gratings –AOTF Broad band filters –absorbance filters –e.g. gratings to implement optimal filters

Joensuun yliopisto PL Joensuu puh. (013) fax (013) Jan 02/tj Formation of the Color Signal

Filter wheel based system

Specim Spectral Camera

A spectrum sampled at 39 wavelengths

Spectral Imaging One approach: to measure the spectral data accurately  A large amount of data Other approach: to measure component images using a few optimally designed color filters  Data is convenient for storing and transmission  Spectral image can be reconstructed computationally

Color Filter Design One approach: to choose an optimized set of commercially available color filters (for example, Kodak Wratten gelatin filters) Other approach: to design optimal color filters computationally (our approach)  adaptive to various application  rewritable filter based imaging system needed  Spectral image can be reconstructed computationally, if needed

ACTIVE TYPE Optimal Light Source Sample CCD camera CCD camera Outdoor Indoor Light Source Optimal Filter Sample + Optimal Light Source Sample + Optimal Filter PASSIVE TYPE Next, computational color filter design and the following spectral imaging systems will be studied

Outline Basics Spectral image Spectral imaging systems Applications Summary

Some applications E-commerce Tele-medicine Image archives Accurate color measurement and representation

Digital Imaging

Display characterictics

Broadband network society Image Processing System Display Image Acquisition Network Database

E-commerce Telemedicine E-commerce, telemedicine in broadband network society Clothes Paints Textile Bags …. Facial color Skin disease Expressions …..

Joensuun yliopisto PL Joensuu puh. (013) fax (013) Jan 02/tj E-commerce Differences between images on a display and real images Online shopping Return of products Figure: Tsumura-san, Chiba Univ.., Japan

Joensuun yliopisto PL Joensuu puh. (013) fax (013) Jan 02/tj Environmental dependence of color Network Patient Medical doctor

Image reproduction multi-primary displays - six components projective display (Natural Vision Research Center, Tokyo) multi-primary printing - inkjet

Multiprimary display The objects are measured as spectral images The display contains 6 primary colors  avoids the problem of metamerism  larger color gamut  natural colors  colors can be seen the same in the measurement place and in the viewing place

RGB-projector

RGB-filters High and low pass filters Multiprimary display 6 filters for

Modified RGB-projector

Multiprimary display

Color gamuts for CRT-display and multiprimary display

Spectral video Sequence of spectral images Shown as movie on the screen Very high memory requirements Efficient compression has been used Shown as RGB or multiprimary (TAO, Japan)

Compression method it has been found that separate spatial and spectral compression give best results in spectral dimension, PCA-based compression give best results combination of PCA in spectral and JPEG in spatial dimension give best PSNR- values for reasonable compression ratios

Outline Basics Spectral image Spectral imaging systems Imaging context Applications Summary

Spectral imaging increasing Novel instruments needed Novel detectors needed Novel image compression methods needed Color research at Joensuu: WWW:

Reconstructed Images Reconstructed Images from Measured Spectral Images

Observation of Spectral Reflectance

Church

Color Change by Cleaning OriginalOnceTwice Three Times Reconstructed images from Measured Spectral Images

Observation of Spectral Reflectance Original Once Twice Three Times

Cleaning Process 1 st Cleaning2 nd Cleaning3 rd Cleaning Each subtract image is normalized by Maximum value