IST – OMNIVIEWS Omnidirectional Visual System Final Review September 27-28, 2001 Lisbon Final Review September 27-28, 2001 Lisbon

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Agenda of the meeting First Day 14:30 Reviewers’ private meeting 15:00 WelcomeJose' Santos-Victor 15:10 Goal of Review Pekka Karp 15:20Omniviews Main Achievements Giulio Sandini 16:00 Coffee Break 16:30 Mirror design principles Branislav Micusik 16:50 Mirror design tools Jose' Santos-Victor, Claudia Decco 17:10 Demos Introduction 17:20 Surveillance Demo Tomas Pajdla, Alex Bernardino 18:00 Transmission Demo Pedro Soares, Giulio Sandini 18:30 End of first Day 20:30 Dinner 14:30 Reviewers’ private meeting 15:00 WelcomeJose' Santos-Victor 15:10 Goal of Review Pekka Karp 15:20Omniviews Main Achievements Giulio Sandini 16:00 Coffee Break 16:30 Mirror design principles Branislav Micusik 16:50 Mirror design tools Jose' Santos-Victor, Claudia Decco 17:10 Demos Introduction 17:20 Surveillance Demo Tomas Pajdla, Alex Bernardino 18:00 Transmission Demo Pedro Soares, Giulio Sandini 18:30 End of first Day 20:30 Dinner

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Agenda of the meeting Second Day 9:00 Navigation DemoJosé Santos-Victor 9:40 Future Outlook and General Discussion Giulio Sandini 10:10 Reviewer’s Private Meeting (with coffee) 11:10Preliminary Evaluation Report 11:40 End of meeting 9:00 Navigation DemoJosé Santos-Victor 9:40 Future Outlook and General Discussion Giulio Sandini 10:10 Reviewer’s Private Meeting (with coffee) 11:10Preliminary Evaluation Report 11:40 End of meeting

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Main Facts Consortium: DIST - University of Genova - Genova CMP - Czech Technical University in Prague VISLAB - Instituto Superior Técnico - Lisbon Consortium: DIST - University of Genova - Genova CMP - Czech Technical University in Prague VISLAB - Instituto Superior Técnico - Lisbon Project Start and Duration: September 1 st 2000 – One year Project Start and Duration: September 1 st 2000 – One year Funding: 100 K€ Open-scheme, Assessment Phase Project

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Project’s Main Objective The main objective of the project is to integrate optical, hardware, and software technology for the realization of a smart visual sensor, and to demonstrate its utility in key application areas. In particular our intention is to design and realize a low- cost, digital camera acquiring panoramic (360°) images and performing a useful low-level processing on the incoming stream of images. The main objective of the project is to integrate optical, hardware, and software technology for the realization of a smart visual sensor, and to demonstrate its utility in key application areas. In particular our intention is to design and realize a low- cost, digital camera acquiring panoramic (360°) images and performing a useful low-level processing on the incoming stream of images.

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Key Technologies Retina-like visual sensor Omnidirectional Mirrors

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Specific Objectives of Assessment Phase n Define the optimal profile of a mirror matching a retina-like visual sensor. Optimal in the sense that direct read-out of panoramic images is obtained. n Demonstrate its utility in key application areas n If successful present a follow-up proposal n Define the optimal profile of a mirror matching a retina-like visual sensor. Optimal in the sense that direct read-out of panoramic images is obtained. n Demonstrate its utility in key application areas n If successful present a follow-up proposal

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project MethodologiesMethodologies 1. Use the currently available SVAVISCA camera for initial experiments 2. Design and simulate mirror using SVAVISCA camera 3. Realize the OMNIVIEWS mirror for the current sensor 4. Demonstrate the mirror in key applications 1. Use the currently available SVAVISCA camera for initial experiments 2. Design and simulate mirror using SVAVISCA camera 3. Realize the OMNIVIEWS mirror for the current sensor 4. Demonstrate the mirror in key applications Simulated image: SVAVISCA camera Hyperbolic mirror Simulated image: SVAVISCA camera Hyperbolic mirror SVAVISCA Pixel layout SVAVISCA Pixel layout

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Mirror’s design principle 1. Uniform Cylindrical Projection 2. Direct read-out through log-polar mapping 1. Uniform Cylindrical Projection 2. Direct read-out through log-polar mapping

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project OMNIVIEWS Mirror Mirror’s Profile Experimental Set-up and test images

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Assessment Criteria 1. Direct read-out of panoramic images 2. Frame rate 3. Resolution and layout of the sensor 4. Mirror profile and size 5. Lens characteristics 6. Camera cost 7. Image quality 1. Direct read-out of panoramic images 2. Frame rate 3. Resolution and layout of the sensor 4. Mirror profile and size 5. Lens characteristics 6. Camera cost 7. Image quality

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC1: Direct read-out of panoramic images Direct read-out from OMNIVIEWS: About 30,000 read-out operations Direct read-out from OMNIVIEWS: About 30,000 read-out operations Image Obtained from a conventional camera: About 1.8 M operations required: 882,000 read-outs (30 times more) 882,000 additions 30,000 divisions Image Obtained from a conventional camera: About 1.8 M operations required: 882,000 read-outs (30 times more) 882,000 additions 30,000 divisions

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC2: Frame rate Currently the maximum read-out frequency is fixed by the camera’s interface (PC Parallel port) limiting the frame rate to about 12 frames/s. More than 25 frames/s is achievable with a faster interface (e.g. USB or PCMCIA) Currently the maximum read-out frequency is fixed by the camera’s interface (PC Parallel port) limiting the frame rate to about 12 frames/s. More than 25 frames/s is achievable with a faster interface (e.g. USB or PCMCIA)

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC3: Resolution and Layout of the sensor

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC4: Mirror Profile and Size Mirror profile and size meets the original plan (6 cm.). Furthermore: 1)New technology for mirror realization 2)Mirror’s design tool of general utility 3)Design and realization of “mixed-mirror” 4)Overall size can be reduced Furthermore: 1)New technology for mirror realization 2)Mirror’s design tool of general utility 3)Design and realization of “mixed-mirror” 4)Overall size can be reduced

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC5: Lens characteristics Standard C-Mount lenses have been used for all experiments and demos No difficulties in principle are envisaged for the design of smaller size lenses (possibly including the mirror).

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC6: Camera Costs Cost of obtaining panoramic images is zero in our case Compared to conventional solutions no extra-cost for the mirror is required. Lower cost is possible with the new “glass-based” technology. The cost of the sensor is equivalent to the cost of conventional sensors realized with the same technology and with the same size. Cost of obtaining panoramic images is zero in our case Compared to conventional solutions no extra-cost for the mirror is required. Lower cost is possible with the new “glass-based” technology. The cost of the sensor is equivalent to the cost of conventional sensors realized with the same technology and with the same size.

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project AC7: Image quality n Topology of images meets the quality criteria n Evaluation of numerical approximations n Three demonstrations: 1)Surveillance (two parts) 2)Navigation 3)Image Transmission n Further processing experiments: 1)Localization using Agam fiducials 2)3D reconstruction n Topology of images meets the quality criteria n Evaluation of numerical approximations n Three demonstrations: 1)Surveillance (two parts) 2)Navigation 3)Image Transmission n Further processing experiments: 1)Localization using Agam fiducials 2)3D reconstruction The project will be successful if we demonstrate that it is possible to create virtual images by simple reading out the pixels from the proposed sensor and to use such images in the aimed applications ….

Project funded by the Future and Emerging Technologies arm of the IST Programme FET-Open scheme - Assessment Project Additional remarks 1. Software mirror-design tools have been developed 2. New kind of mirror have been proposed extending the original plan (i.e. the “mixed mirror”) scientific papers have been published 4. Plans for the future are clearer. 1. Software mirror-design tools have been developed 2. New kind of mirror have been proposed extending the original plan (i.e. the “mixed mirror”) scientific papers have been published 4. Plans for the future are clearer.