WP1.4 Index and Search George Kakaletris University of Athens

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search2 Overview WP Structure and Status D1.4.2 Review Issues D1.4.3 Content and Status Index and Search Detailed Design Search Engine Overview

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search3 Index & Search: Timeline & Organisation Index & Search Experimentation Index & Search Services Design Tasks M6M12 D1.4.3: Index & Search Services Detailed Design (M17) Index & Search Services Implementation Tasks M18M24M30M36M0 D1.4.3: Index & Search Services Detailed Design – Revision (M34) D1.4.4a: Index & Search Services Prototype Alpha Version (M24) D1.4.4b: Index & Search Services Prototype Beta Version (M30) D1.4.4c: Index & Search Services Prototype Final Version (M34) M1.4.3a: Index & Search Services Internal Release Version (M20) M1.4.3b: Index & Search Services Internal Release Version (M27) M1.4.3c: Index & Search Services Internal Release Version (M31)

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search4 WP 1.4 Services Content Source Description & Selection Service Filter content sources (incl. collections) to maximise search relevance / cost ratio Automatically generate source description Data Fusion Service Fuse the ranked content Result Sets into single ranked Result Set Feature Extraction Process documents to produce feature descriptors Index Service Build and consume inverted and forward indices on metadata for search performance enhancement Personalisation Service Apply personalised filtering and presentation in search Manage user profiles Search Service Locate content by searching in content metadata and on-the-fly extracted features

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search5 Effort per Partner: M13-30 PartnerEffort (MMs) Tasks UoA33Search, Personalisation FAST30Index, Search, Feature Extraction USG23.2Data Fusion, Content Source Description & Selection, Feature Extraction UMIT9Feature Extraction, Search UNIBAS2Feature Extraction FhG-IPSI6.3Personalisation

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search6 WP1.4 Status Detailed design : in progress D1.4.3 in final stage of submission Implementation in progress: Data exchange mechanism. Full-text indexing. SearchPlanning Basic workflow execution Basic Search Operators:  XPath filtering  Sorting  Joining metadata (on URI) Experimentation in progress: Working on real ARTE scenario data initiated. Continuous experimentation with technologies

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search7 D1.4.2 Review : Objections and actions Provide incremental document (w.r.t. D1.4.1): No action taken: document is intentionally standalone. Revise structure with regards to introductory documents: No action taken: document is as proposed by D1.x.2 template. Present more details on elements, e.g. algorithms etc No action taken : document is built around the logical view of the system. Planned for D Fix writing / language / formatting issues: Majority taken into account. Revise terminology (mostly WP1.3 related) Several revisions yet not entirely fixed. Revision of introductory and conclusion sections: Revised the referenced parts. Comments directly submitted to SC : NOT FIXED!

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search8 D1.4.3 Overview The 1 st (really important) implementation document Standalone document More than 160 pages Services organized in 4 groups Revises information of the D1.4.2PackagesElementsDeployment Presents details on: Classes, methods, attributes LibrariesServicesAlgorithms Reused elements and technologies Use of framework Published information

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search9 D1.4.3 Status Status Due day: 31 st Jan 2006, fail EU Submission deadline: 15 th Mar 2006, fail Authoring: 13 Mar 2006, success Submission for review: 15 Mar 2006, success Review (by FhG – IPSI): 24 Mar 2006, success Revision : 31 Mar 2006, in progress Submission to SC: 3 Apr 2006, planned Submission to EU: ~10 Apr 2006, planned Result Document Content : success Delivery Procedure: failure to commit to deadlines!

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search10 Detailed Design Conclusions Innovation in search: Pre-processing, planning, operators Data exchange optimisation Extremely rich functionality Exploitation of the Grid Dynamic discovery and utilisation of resources StorageExecution Minimisation of requirements for dynamic deployment Use of registries Design with parallel execution in mind Grid-like UI

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search11 Detailed design issues Performance? Experimentation will show. PES dependent Minor gaps at service boundaries: WP-Internal and cross-WP Will be identified and appropriately handled Missing functionality Service and element missing details Planned for subsequent detailed design document updates (D1.4.3)

TCom WP1.4 Session th Mar. ’06, CERN Geneva - Switzerland WP1.4, Index and Search12 Search Engine Overview