ENTERPRISE SEARCH AND ITS VALUE TO THE ENTERPRISE Lee Atkinson or why search and retrieval of ‘relevant’ information is only the start in meeting the business need

Internal Database Internal Unstructured External Licensed External General News feeds and genre specific Introduction A business view of enterprise search and its value to the enterprise. What are the issues? Is there a common problem domain that needs to be solved in many large organisations when trying to put in place a single 'view' of their business and then find what they need, when they need it. The presentation will examine the issues found in three main areas in realising enterprise search and extracting real business value from it; Find it We will focus on a practical example of where simply having the ability to 'search' and be more 'relevant' has limitations in finding the right information at the right time, in the right context and enabling the information to be used. We will then go on to look at how search needs to encompass ALL of the data held in the enterprise and how certainty and completeness of results, more than relevancy, are the key requirements. Core Competencies Information Gathering (structured and unstructured file types) Information Organisation Information Analysis Entity Profiling (single view) Business Applications Organise it Use it

The Problem Domain The ‘reality’ of technology in large organisations and has led us to build a set of pre requisites that any solution has to be able to be deployed in and work within that ‘reality’. Organisation beyond the department level is unusual Chaos is part of a dynamic business, technology’s ability to support it always lags Data silos are a fact of life and wholesale replacement is not going to happen as a big bang. Even organisations that think strategically, deploy tactically to succeed Legacy applications are embedded in the heart of most global business Adapt, not rebuild where the end product achieves the same outcome. Integration and business change are the major problems in delaying the implementation of new technology. The more systems the more overhead in data reconciliation and mapping.

What search based solutions offer as an enabler Search based applications offer a different approach, one that can sit above the chaos and bring organisation to it. Relevancy and semantic search based business solutions offer a means of pulling together information into a common structure and not require changes to existing processes, systems or data storage and classification approach. Search can work with whatever is in place - silos, data quality etc - and create the information structures within the application without the need for any manual intervention. To the end user there is no difference – they get comprehensive information when they need it, how they need it. But most search based solutions are focused on the web and technology

FIND IT

Semantic Web 2.0 Generic Search Harder and harder to differentiate Focus on sources and extraction Relevancy at the margin – so what? Increasing commoditisation Enterprise search Open source competitive delay Completeness of internal information is fundamental Being the most relevant, is not Structured and unstructured is one and the same. Having a relevant results set is not enough. Near real time information gathering becomes a pre-requisite Business process change is not an option if rapid deployment is a need Focused on web and new information sources Search as an information consolidator and organiser but on its own an enabler

Case study as an example Large Business – 14 offices, people, each office different – but teams often from multiple offices, departments work on a single project Problem domain Need to collaborate across offices, no common structure or collaboration capability Images, CAD and unstructured data 90% of their information needs DMS purchased, but business change and information migration overhead meant that never implemented Consolidated and Organised view 32 TB of internal unstructured External 8TB of Database Around an entity - Customer - Project - Supplier

Entity Extraction and Entity Profiling Categorisation Classification and Sources Relevancy based Search Search Optimisation Classification, Dictionaries (manual and automatic) Organisation against predefined categories and sub categories To be able to build inter relationships between categories such as; Type of cavity wall, projects it appeared, suppliers, customers, costs etc May also require combination of data mining with information retreival Detection of new terms and entities from within corpus and expansion into taxonomies and entity lists Relevancy is the start point to information organisation 5/15/2007 Daft V1 Need to be able to organise around entity

Enterprise Issues – once information is organised Needs to be near real time – critical systems, means upto date positions Needs to be complete information set, with audit trail and reconciliation A results set on screen fails the Business Value Proposition test Simply being smarter, quicker and more relevant is hard to build an ROI to business users Focus on business application to unlock value

ORGANISE IT

Piecing it together Consolidated and Organised view 32 TB of internal unstructured External 8TB of Database Integrated single results set Defines results in context of entire data set (what’s not there) Semantic, contextual, file types etc are commodities that defined the scope of the applications possible Core Competencies Information Analysis Entity Profiling (single view) Takes search results as a start point Adds business taxonomies Customer Projects People ‘Entities’ Business Applications Delivers Business Framework Pre determined ‘end state’ All information applications - No difference between structured and unstructured Combines information into Business Process Analysis against rules Exception based event management Business context Business Applications Business Applications Business Applications Business Applications RulesRules Business Application

USING IT AS A CORE BUSINESS RESOURCE

Core Competency Information Gathering (structured and unstructured file types) Information Organisation Information Analysis (Bayesian, Linguistic, Categorisation, Taxonomy, WEKA) Entity Profiling (single view) Internal Database Internal Unstructured External Licensed External General News feeds and genre specific Rules Engine Business Application Real time MIS Business Portal Collaboration Portal SAN Business Processes Business Architecture Business Application Project View Business Application Business View Business Application Compliance Business Application Research Business Application Search based business architecture – unlocking value 1 Information stays where it is 5 min updates 2 Blend DB and documents Its all information 3 Business rules define analysis Information enrichment 4 5 Multiple business applications Single business view Single customer view

ENTERPRISE SEARCH AND ITS VALUE TO THE ENTERPRISE Lee Atkinson or why search and retrieval of ‘relevant’ information is only the start in meeting the business need