1. High-Performance Distributed Multimedia Computing Frank Seinstra, Jan-Mark Geusebroek Intelligent Systems Lab Amsterdam Informatics Institute University of Amsterdam MultimediaN (BSIK Project)

2. MultimediaN and DAS-3

3. Van Essen et al. Science 255, 1999. MultimediaN and high- performance computing

4. A Real Problem, part 1… News Broadcast - September 21, 2005 ( see video1.wmv ) News Broadcast - September 21, 2005 ( see video1.wmv ) Police investigating over 80.000 (!) CCTV recordings Police investigating over 80.000 (!) CCTV recordings First match found no earlier than 2.5 months after July 7 attacks First match found no earlier than 2.5 months after July 7 attacks automaticanalysis?

5. Image/Video Content Analysis Lots of research + benchmark evaluations: Lots of research + benchmark evaluations: –PASCAL-VOC (10,000+ images), TRECVID (200+ hours of video) A Problem of scale: A Problem of scale: –At least 30-50 hours of processing time per hour of video! Beeld&Geluid => 20.000 hours of TV broadcasts per year Beeld&Geluid => 20.000 hours of TV broadcasts per year NASA => over 850 Gb of hyper-spectral image data per day NASA => over 850 Gb of hyper-spectral image data per day London Underground => over 120.000 years of processing … !!! London Underground => over 120.000 years of processing … !!!

6. High Performance Computing Solution: Solution: –Very, very large scale parallel and distributed computing New Problem: New Problem: –Very, very complicated software Solution: tool to make parallel & distributed computing transparent to user - familiar programming - easy execution Wide-Area Grid Systems User Beowulf-type Clusters Beowulf-type Clusters Since 1998: “Parallel-Horus”

7. Parallel-Horus: Features (1) Parallel-Horus Parallelizable Patterns Sequential programming: Sequential programming: Sequential API Seinstra et al., Parallel Computing, 28(7-8):967-993, August 2002 +/- 18 patterns (MPI)

8. Parallel-Horus: Features (2) Lazy Parallelization: Lazy Parallelization: Seinstra et al., IEEE Trans. Par. Dist. Syst., 15(10):865-877, October 2004 Don’t do this: ImageOpImageOpScatterScatterGatherGather Do this: ImageOpScatter Avoid Communication ImageOpGather On the fly!

9. Extensions for Distributed Computing Parallel Horus Client Parallel Horus Client Wide-Area Multimedia Services: Wide-Area Multimedia Services: Parallel Horus Server Parallel Horus Servers Parallel Horus Servers User transparency? User transparency? Abstractions & techniques? Abstractions & techniques? Grid connectivity problems? Grid connectivity problems?

10. A Real Problem, part 2… + LambdaRAM ?? may be time-critical…!

11. Color-Based Object Recognition (1) Our Solution: Our Solution: –Place ‘retina’ over input image –Each of 37 ‘retinal areas’ serves as a ‘receptive field’ –For each receptive field: Obtain set of local histograms, invariant to shading / lighting Obtain set of local histograms, invariant to shading / lighting Estimate Weibull parameters ß and γ for each histogram Estimate Weibull parameters ß and γ for each histogram –Hence: scene description by set of 37x4x3 = 444 parameters += Geusebroek, British Machine Vision Conference, 2006.

12. Color-Based Object Recognition (2) Learning phase: Learning phase: –Set of 444 parameters is stored in database –So: learning from 1 example, under single visual setting Recognition phase: Recognition phase: –Validation by showing objects under at least 50 different conditions: Lighting direction Lighting direction Lighting color Lighting color Viewing position Viewing position “a hedgehog”

13. Amsterdam Library of Object Images (ALOI) In laboratory setting: In laboratory setting: 300 objects correctly recognized under all (!) visual conditions 300 objects correctly recognized under all (!) visual conditions 700 remaining objects ‘missed’ under extreme conditions only 700 remaining objects ‘missed’ under extreme conditions only Geusebroek et al., Int. J. Comput. Vis.. 61(1):103-112, January 2005

14. See also: http://www.science.uva.nl/~fjseins/aibo.html Example: Object Recognition

15. Demonstrated live (a.o.) at ECCV 2006, June 8-11, 2006, Graz, Austria (see video2.wmv)

16. Performance / Speedup on DAS-2 Recognition on single machine: +/- 30 seconds Recognition on single machine: +/- 30 seconds Using multiple clusters: up to 10 frames per second Using multiple clusters: up to 10 frames per second Insightful: even ‘distant’ clusters can be used effectively for close to ‘real-time’ recognition Insightful: even ‘distant’ clusters can be used effectively for close to ‘real-time’ recognition Single cluster, client side speedup Four clusters, client side speedup

17. Results: applicability Beneficial: Beneficial: –Performance gains largely obtained ‘for free’ –With Parallel-Horus we can build similar complex ‘Grid’ applications in a matter of hours… Ok, robot dog is a funny/crazy toy application, but: Ok, robot dog is a funny/crazy toy application, but: –Best performer in TRECVID 2004 & TRECVID 2005 Snoek et al., IEEE Trans. Pattern Anal. Mach. Intell. in press, 2006

18. Current & Future Work Very Large-Scale Distributed Multimedia Computing: Very Large-Scale Distributed Multimedia Computing: –Overcome practical annoyances: Software portability, firewall circumvention, authentication, … Software portability, firewall circumvention, authentication, … –Optimization and efficiency: Tolerant to dynamic Grid circumstances, … Tolerant to dynamic Grid circumstances, … Systematic integration of MM-domain-specific knowledge, … Systematic integration of MM-domain-specific knowledge, … –Deal with non-trivial communication patterns: Heavy intra- & inter-cluster communication, … Heavy intra- & inter-cluster communication, … –Reach the end users: Programming models, execution scenarios, … Programming models, execution scenarios, … Collaboration with VU (Prof. Henri Bal) & GridLab Collaboration with VU (Prof. Henri Bal) & GridLab –Ibis: www.cs.vu.nl/ibis/ –Grid Application Toolkit: www.gridlab.org

19. Conclusions Effective integration of results from two largely distinct research fields Effective integration of results from two largely distinct research fields Ease of programming => quick solutions Ease of programming => quick solutions With DAS-3 / StarPlane we can start to take on much more complicated problems With DAS-3 / StarPlane we can start to take on much more complicated problems But most of all: But most of all: –DAS-3 very significant for future MM research

20. The End (see video3.avi)