11/16/2018 4:57 AM Creating Great End-User Experiences with Fast Search for SharePoint 2010 OSP322 Mark Stone Dan Benson © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

User Experience Search Experience Content Technology

Goals for this session Understand how to improve and enhance: Content User Experience

What we’ll cover… …how we’ll cover it Search Customization Pipeline Extensibility Extending the OOB UX …how we’ll cover it Building an application to detect and display sentiment

… Starting from the top… End Users Search Center Data Sources TechReady12 11/16/2018 Starting from the top… Pipeline Extensibility (FS4SP only) End Users Data Sources Federation OpenSearch Source Content Processor Crawler Indexer Query Processor Search Center User Profiles Index Partition … Format Conversion Language Detection Entity Extraction Lemmatization Mapper Federation Object Model © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Feeding Flow – SharePoint App. Server Request crawl Web crawls FAST Content SSA Admin DB Database Poll request Admin component Content Web Service FAST Content SSA Crawl DB Log request Poll request Master Crawl comp. Crawl data Crawl history Crawl queue additions Distribute work Crawl comp. Crawl comp. Crawl comp. Document batches FAST Search

Feeding Flow – FAST Side TechReady12 11/16/2018 Feeding Flow – FAST Side FAST Content SSA FAST Admin DB FAST Query SSA Admin DB Crawled properties detected Document batch Click-through history query  url Content Distributor Crawled property mappings Content Distributor Document batch Link DB Item Processing Document metadata Web Analysis Item Processing Web Analysis FIXML Links and anchor texts Indexing dispatcher Indexing dispatcher FIXML Local disk Local disk Binary index Indexer Indexer © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Process content with advanced linguistics Automatic and detailed analysis creates a great search experience FAST Content Processing Pipeline Tokenization Apply the language specific rules for identifying words, concepts, idioms and phrases. Also applies custom word breakers found in part numbers or telephone numbers. Language Encoding and Detection Identifies the native written language and locale specific encoding so that the proper dictionaries can be used by the tokenization and lemmatization stages Format Conversion Extracts plain text from multiple file formats, encodings, and applications Lemmatization Finds the root of a word for a given language. For English it maps run, runs, running and ran back to a single lemma. Understands language specific grammar and context. Date and Time Normalization Converts dates and times to a standard representation, to handle locale specific representations. For example, knows that 14-Mar-10 is equivalent March 14, 2010. Map Crawled Properties Maps all of the metadata that was discovered by the various pipeline stages Web Link Analysis Analyzes documents for hyperlinks extracting anchor text which reinforces the authority ranking of a document. Document Vector Creates a unique representation of a document that reflects important terms and frequency of occurrence. Used to find similar documents. Entity Extraction Finds terms in the content and maps them to predefined categories. Out of the box support for People, Companies and Locations, but can be extended to any category. Speaker Notes: This slide builds, so ensure you discuss each build point. Speaker Script: One of the key differentiators between FAST Search Server 2010 for SharePoint and other search products is the ability for you to add logic to the content processing pipeline, which is used at crawl and indexing time. [Build] The first stage in the pipeline is to extract the text from content, regardless of the document type and format. [Build] Then the language and encoding of the text is determined, which helps with later stages in the pipeline (such as defining which dictionaries and language rules to use for word analysis) [Build] The tokenization stage breaks the stream of text down into individual words by using language- specific word breakers. FAST Search Server 2010 for SharePoint includes very powerful word-breaking capabilities based on spaces and punctuation, but also on language-specific rules for dealing with compound words. [Build] After words have been broken down, FAST Search Server 2010 for SharePoint analyzes the words back to their stem form. This process, which is called lemmatization, is language specific, and FAST Search Server 2010 for SharePoint includes very comprehensive rules for applying this stage. [Build] As I discussed earlier, you can create your own entity extractors to enable your users to search in the terms and language of your business. The entity extraction process happens at this stage in pipeline. FAST Search Server 2010 for SharePoint includes some common entity extractors by default, but you can add to those. [Build] The next stage is to normalize dates and times that were found in document contents and metadata, so that features such as sorting, filtering, and refinement will work consistently. [Build] Then a document vector is generated for each document, which represents an overall analysis of the contents and metadata of the item. This vector is used to compare document similarity, which enables the 'Similar Results' feature. [Build] Hyperlinks that point to the document being indexed are then analyzed in terms of their anchor text. This is a good indication of how authoritative the document is when it is returned by a search that contained search terms which match the anchor text. [Build] The final step is to map the metadata and entity values to managed properties, so that the search schema is kept up-to-date. Breaks down content to the smallest addressable chunks to build meaning Understands file encoding, data formats, and written languages Supports 400+ file formats, 80+ languages

Start with FS4SP Pipeline Extensibility TechReady12 11/16/2018 Start with FS4SP Pipeline Extensibility XML Input (file) EXE XML Output (file) PipelineExtensibility.xml Your EXE Bing  PipelineExtensibility.xml © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

demo Build it… TechReady12 11/16/2018 © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Extending the UX

Extending the UX

where the magic happens Core Results WebPart where the magic happens All other Search WebParts depend on this one. Interacts with the Federation OM

How the Core Results WebPart Does It. Inherits from the DataFormWebPart Allows for custom XSLT for various rendering options Uses the DataSource and DataSourceView classes to configure the query Retrieve the results Key Takeaway: Use the DataSource and DataSourceView classes to customize the query

Conceptual View of a Search Query Federation OM DataSourceView Location Configuration DataSource Core Results Web Part Paging, Stats, etc… Refinement Panel Query, WebPart Properties (Defaults from Location) Returns results to WebParts and Refiners to Refinement Panel Location type and location configuration

demo Build it… TechReady12 11/16/2018 © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

where the magic happens Visual Refiners Core Results WebPart where the magic happens All other Search WebParts depend on this one. Interacts with the Federation OM Non-Search Related

In Review: Session Objectives and Takeaways TechReady12 11/16/2018 In Review: Session Objectives and Takeaways Session Objective(s): Identify the customization options and solutions/tools available for FAST Search for SharePoint. Identify the issues and concerns with respect to customizations and try to overcome them with new innovations or strategies. Address customers with advanced document processing requirements using FS4SP's pipeline extensibility mechanism. Key Takeaways Knowledge of Pipeline Extensibility Knowledge of Federation Object Model usage © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

TechReady12 11/16/2018 Wrap Up: Where to look TechNet Script Center: http://gallery.technet.microsoft.com/ScriptCenter SharePoint  Search Management SharePoint  Deployment and Upgrade FAST Search for SharePoint TechNet Site: http://technet.microsoft.com/en-us/library/ee781286(office.14).aspx Mapping crawled to managed properties: http://technet.microsoft.com/en-us/library/ff453913(office.14).aspx Document Processing configuration Optional Processing options: http://technet.microsoft.com/en-us/library/ff395933(office.14).aspx Pipeline Extensibility: http://technet.microsoft.com/en-us/library/ff354941(office.14).aspx Adding addition IFilters: http://technet.microsoft.com/en-us/library/ff354937(office.14).aspx Enabling the Advanced Filter pack: http://technet.microsoft.com/en-us/library/ff383314(office.14).aspx User defined property sets: http://msdn.microsoft.com/en-us/library/aa380374%28VS.85%29.aspx XMLMapper protocol spec: http://download.microsoft.com/download/8/5/8/858F2155-D48D-4C68-9205-29460FD7698F/%5BMS-FSPSCFG%5D.pdf © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Q&A

11/16/2018 4:57 AM © 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.