1. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-051 Spectral Imaging at Heriot Watt University Dr Andy R Harvey School of Engineering and Physical Sciences Heriot Watt University Edinurgh, EH14 4AS Tel +0131 451 3356 a.r.harvey@hw.ac.uk

2. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-052 Some Heriot Watt spectral imaging solutions Birefringent 2D Fourier-transform imaging spectrometer (FTIS) Snapshot 2D foveal imaging spectrometer (OFIS) Snapshot 2D imaging spectrometer (IRIS)

3. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-053 Conventional FTIS offers High SNR in low flux MWIR, twilight Very high spectral resolution Wide spectral range But conventional time-sequential interferometry in real-world applications is highly problematic Birefringent Fourier Transform Imaging Spectrometer Fixed mirror Scanning mirror Detector array

4. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-054 Birefringent FTIS Mechanical sensitivity of conventional FTIS makes real-world applications almost impossible Introduce temporal path difference with scanning Wollaston prisms Inherently vibration insensitive since path difference due by birefringence within a single crystal and common path Optical gearing reduces required accuracy of movement by a factor ~200

5. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-055

6. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-056 Movie of spectral image cube Colour image

7. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-057 Foveal hyperspectral imaging in 2D Optical Fibre-coupled Imaging Spectrometer Real-time hyperspectral imaging in 2D would require excessive information throughput GVoxel/sec Bottlenecks include detector – 20 MVoxel/sec Computer processing Biological systems with this problem employ a scanning fovea….

8. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-058 Foveal hyperspectral imager: OFIS Schematic

9. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-059 OFIS: Hardware & raw data The hyperspectral fovea assembly: Custom fibre optic image refromatter 1D dispersive hyperspectral imager CCD camera Spatial extent Wavelength 400 nm 700 nm First fibre Last fibre Raw image at CCD prior to reformatting

10. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0510 OFIS: Movie demonstrating real-time spectral ID with simple recognition Colour image

11. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0511 Snapshot spectral imaging in 2D Image Replication Imaging Spectrometer

12. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0512 Image Replication Imaging Spectrometer: IRIS Single image multiplexed onto 2 N passband images ‘100%’ optical efficiency Snapshot image no temporal misregistration Trade spectral resolution for FoV Low resolution, wide FoV High resolution, small FoV Gas detection High spectral resolution Few Bands Modest FoV Conceptually related to Lyot filter World’s only snapshot, 2D spectral imager (almost !) Large format detector Spectral Demultiplexor

13. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0513 Wollaston prism polarisers replicate images Each Wollaston prism-waveplate pair provides both cos 2 and sin 2 responses All possible products of spectral responses are formed at detector IRIS snapshot spectral imager:

14. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0514 Components & Assembly 8 channel system 3 Quartz retarders 3 Calcite Wollaston prisms

15. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0515 Absolute total transmission Bandpass filter & polariser dominate losses Improved system: T>80% Theoretical throughput is 2 n times higher than for other techniques! Demonstrated 96% transmission for IRIS-only components 0 25 50 Response (%) Absolute response curves in polarised light

16. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0516 An example medical application: Blood oxymetry in the retina

17. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0517 Requirements for a snapshot technique: retinal imaging Improved calibration Patient patience Remove misregistration artefacts; imperfect coregistration arises due to Distortion of eye ball with pulse Variations in imaging distortion between images Similar issues with other in vivo applications Imaging epithelial cancers PC15

18. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0518 Blood oximetry Optimal spectral band for retinal oximetry Vessel thickness ~ optical depth 570-615 nm Eight bands approximately equally spaced 80 40

19. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0519 Spectral Retinal Imaging Difficult imaging conditions render application of traditional HSI techniques problematic IRIS enables real-time and snapshot spectral imaging Canon

20. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0520 Video sequence recorded with low-power, CW tungsten illumination

21. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0521 Retinal image recorded with flash illumination

22. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0522 574581 585 592595 603 607 613 Coregistered and PCA images PC1PC2PC1 & PC2

23. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0523 Application to microscopy: Imaging of multiple fluorophors IRIS fitted to conventional epi-fluorescence microscope Germinating spores of Neurospora crassa stained with GFP – nucleii fluoresce at 510 nm FM4-64 – membranes fluoresce at >580 nm 0 2525 5050 Response (%)

24. This document gives only a general description of the product(s) or services and except where expressly provided otherwise shall not form part of any contract. 24 May 2005Hyperspectral Working Group MWIR IRIS Consists of: n COTS Phoenix MWIR Camera n Specac Polariser n IRIS II Optical Telescope

25. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0525 Conclusions The transfer of spectral imaging from scientific to military and laboratory applications must address the needs of high SNR, accurate coregistration and logistics. No single technique can satisfy all requirements simultaneously ‘Horses for courses’ New techniques such as described here illustrate how these requirements can be satisfied Similar issues occur in both military and civilian (eg medical) applications introducing significant scope for dual use.

26. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0526 Additional information Linked by previous slide buttons

27. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0527 The co-registration problem Co-registration required for time sequential direct and FT imaging Not for snapshot/fully-staring Misregistration of spectral images distorts spectral basis sets Video spectrum frame rates insufficient to freeze motion from most aerial platforms Target

28. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0528 The magnitude of the co-registration problem Co-registration should be better than 1/20 - 1/50 of a pixel Deployment of time sequential DIS and FTIS will normally require ‘step and track’

29. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0529 Bandpass functions Bandpass are overlapping bell shapes Can be optimised by adjusting waveplate thickness and dispersion

30. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0530 Spectral discrimination Bell-shaped IRIS transmission functions tend to smooth spectra Typically 6% reduction in separation in 8D spectral space 8x improvement in SNR Contiguous ‘top-hat’ IRIS

31. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0531 1D image x path difference  Fixed mirror Scanning mirror Detector array NN NxNx N y (t) NN NxNx FT N (t) NxNx NyNy N NxNx N y (t) Direct Imaging Spectrometry(Fourier) Transform Imaging Spectrometry Temporally scanned Snapshot/fully staring N  (t) NxNx NyNy FT NN NxNx NyNy No temporal coregistration problem The traditional technique for 1D remote sensing 2D very immature…. IRIS OFIS Summary and novel HWU techniques in red Very high spectral resolution Highest SNR in low-light conditions The optimum technique for MWIR Unsuitable for poorly controlled environments... FTIS Mature The traditional technique for 2D static spectral imaging Low MPLX efficiency

32. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0532 Ratio of SNRs in 3-5  m band -temporal scan 1500 m nadir path 40 Hz, 10 bands Zero range 1 Hz, 10 bands

33. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0533 Ratio of SNRs in 8-14  m band - temporal scan Zero range 40 Hz, 10 bands 1500 m nadir path

34. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0534 IRIS:FTIS SNR Zero range 40 Hz, 10 bands 1500 m nadir path

35. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0535 Lyot filter: principle of operation Polariser Waveplate

36. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0536 Optical scaling laws Hamamatsu ORCA-ER Inputs: FoV Sub image size on CCD CCD pixel size Primary lens magnification & F# Collimating lens back focal distance, focal length & front element diameter Prism birefringence Outputs: Field stop size Collimating lens rear element diameter Splitting angles, apertures & depths of prisms Apertures of retarders, polarisers and filters Imaging lens focal length & front element diameter Field stop Collimating lens Bandpass filter Imaging lens Camera Polariser, retarders & Wollaston prisms (index matched) Primary lens

37. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0537 Spectral retinal Imaging By 2020 there will be 200 million visually- impaired people world wide Glaucoma, diabetic retinopathy, ARMD 80% of those cases are preventable or treatable Screening and early detection are crucial Spectral imaging provides a non-invasive route to monitoring retinal biochemistry Blood oximetry, lipofuscin accumulation Diabetic Retina Normal Retina

38. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0538 Measured & predicted spectral responses

39. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0539 Imaging Concepts Group Research Group Head Dr Andy Harvey PDRA Dr Colin Fraser Dr Eirini Theofanidou Bertrand Lucotte PhD Students Alistair Goreman Asloob Mudassar Gonzalo Muyo Sonny Ramachandran Ied Abboud Beatrice Graffula External PhD students Ruth Montgomery (NPL) Robert Stead (Thales) Funders/Collaborators AstraZeneca AWE BAE Systems DSTL EPSRC NATO NPL QinetiQ Royal Society Scottish Enterprise South Glos. NHST SAAB Thales

40. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0540 Research areas Imaging Concepts Group Spectral imaging Retinal imaging Wavefront coding Aperture synthesis imaging (optical and mm-wave) Optical encryption for communications mm-wave imaging Biophotonics Insect flight dynamics

41. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0541 Overview Introduction to spectral imaging Spectral imaging techniques at Heriot-Watt University FTIS Inherently robust FT imaging spectrometer IRIS Snapshot, ‘100%’ optical throughput imaging spectrometer OFIS Foveal hyperspectral imaging spectrometer An example application Spectral imaging of the retina Conclusions

42. HSWG Andy Harvey:a.r.harvey@hw.ac.uk25-May-0542 What are the issues High SNR required >100 No spatial or spectral multiplexing desirable Fourier-transform in some conditions Accurate coregistration required (<1/20 pixel) Snapshot spectral imaging preferred Spectral resolving power matched to requirement 100s for data acquisition ~10 for many applications As few as two if clutter allows (eg spectral lines) Detector is ‘information bottleneck’ 20 MVoxel/second per tap