Prof.dr. Inald Lagendijk, Coordinator Delft University of Technology Faculty of Electrical Engineering, Mathematics and Computer Science 1 Network of Excellence PetaMedia Peer-to-Peer Tagged Media (42 months, started March 1, 2008)

2 Partner Concept of PetaMedia: EU Network of National Networks

3 Technical Scope of PetaMedia What: Developing new paradigms and technologies for efficient and effective access to multimedia content How: Multimedia content analysis meets P2P/social networks Best of three worlds Peer to peer/social networks (Tribler software) User-based taggingContent analysis

4 Objectives of NoE PetaMedia Four main objectives: 1.European network formation and strengthening in the area of multimedia content access. Consolidation of the network beyond lifetime of NoE as Virtual Centre of Excellence. 2. Progress state of art in (the combination of) user tagging - content analysis - P2P/social networking via joint research. 3.Converge of results to demonstration and experimentation. 4. Dissemination of knowledge (P2P technology, content analysis, information retrieval, …) inside and outside NoE.

5 Example of Scenario and Research (1)

6 Example of Scenario and Research (2) user 1: tree sun dog cloud user 2: tree sun vacation cloud dog clouds score: tree sun dog cloud

7 Example of Scenario and Research (3) Temporal tag integration (learning) Temporal generalization (tag a frame given tagged neighboring frames) Temporal segmentation (into tag-based shots)

8 Example of Scenario and Research (4) Object detection: Spatially connect tags to objects Ask user to collaboratively click on objects, given the temporal tags Use resulting spatio-temporal tags demonstration/field tests Video segment (shot) retrieval Browsing Recommendation

9 Build on Tribler P2P Software (1) Value creation and intelligence goes to the edges: P2P Tribler: Operational P2P system, started Feb 2006 Long term, experimental living lab Largest dataset collection Empirical approach Tribler downloads Open Source & community building Social File sharing Unified search & viewing Bittorrent, Youtube.com, Liveleak.com Distributed sharing, rating, recommendation Research team: 18 full-time people

10

11 Build on Tribler P2P Software (2) Tribler core toolkit and API Tribler Delft: May 27, 2008 More: IP Layer Tribler core toolkit and API P2P peer discovery recommendation download/stream Appl. Specific #1Appl. Specific #2Appl. Specific #3

12 Special Interest Groups Formed during KickOff Content/tag (nonlinear) life cycle and required supporting (distributed) architecture, incl. cooperative content authoring Need for distributed processing and content swapping (in particular scenarios) Automatic object and/or action segmentation and recognition Keep-it-simple user interface supporting tagging and tag-use Incentives to tag: games, type of content, social tagging (friends), contesting and reputation Experience tagging – beyond visual cues: sensor-based tagging (camera, microphone, bio, …) Group intelligence building: trust, reputation, friends, cooperative authority Correctness/quality/originator/evaluation of tags and ways of feedback (create trust) Prepare tags, use tags, link tags: recommendation, personalization, profiling, indexing of video segments

13 Special Interest Groups Formed during KickOff

14 May 27

15 Backup Slides

16 PetaMedia: Network of National Networks Consortium of four national networks in the area of multimedia content analysis and networking Four networks and leading partners in PetaMedia: NIRICT: Netherlands Institute for Research on ICT (Delft University of Technology) IM2: Interactive Multimodal Information Management (Ecole Polytechnique Fédérale de Lausanne) MMKM: Multimedia Knowledge Management network (Queen Mary, University of London) HC3: (Extended) Human-Centric Communication Cluster (Technische Universität Berlin)

17 Strength and Focus of National Networks Four national network encompasses researchers in field of Multimedia (audio and video) processing and networking Personalized multimedia content access Multimodal processing and recognition Methods, algorithms and theoretical foundations for human-centric data search and interpretations User’s perspective, interface and content recommendation in P2P connected virtual communities Distributed semi-automatic metadata extraction, self-adaptation and interconnection (Distributed) database management Common background in COST292: Semantic Multimodal Analysis of Digital Multimedia

18 Dimensions of PetaMedia Joint program of Activities along three Dimension 1.People: Disseminating and outreach Workshops, travel grants, funding junior researchers 2.Organization: Towards a virtual centre of excellence Organization form that extends the national networks 3. Technology: Integrative research and experiments Roadmapping Research for large scale experiments using MCA/Tagging on top of P2P software Tribler User and technological experiments with software

19 Work Packages (42 months) WP Number Work package titleStart month End month Joint Program on Integration and Sustainability (JPIS) JPIS.1Integration of People and Organizations142 JPIS.2Integration of Technologies142 JPIS.3Towards a Virtual Centre of Excellence1942 Joint Program on Research Activities (JPRA) JPRA.1Fundamental Research on MCA and P2P142 JPRA.2Technology Roadmap and Architecture118 JPRA.3Research for Field Trial and Future of P2P/MCA736 JPRA.4Execution of Field Trials using P2P/MCA3142 Joint Program on Spreading Excellence (JPSE) JPSESpreading Excellence742 JPMNetwork Management142

20 Kickoff Presence (in Addition to Leading Partners) Switzerland IDIAP, Uni. Geneva, Uni. Western Switzerland Germany Univ. Aachen, Univ. Hannover UK Imperial, Univ. Sheffield, Goldsmith College, Univ. Glasgow, Open Univ. Netherlands CWI, Univ. Twente, Telematics Inst. Project officer; Advisory board