2. 11/16/2018 4:57 AM
Creating Great End-User Experiences with Fast Search for SharePoint OSP322
Mark Stone Dan Benson
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. User Experience
Search Experience
Content
Technology

6. Goals for this session Understand how to improve and enhance: Content
User Experience

7. What we’ll cover… …how we’ll cover it
Search Customization
Pipeline Extensibility
Extending the OOB UX
…how we’ll cover it
Building an application to detect and display sentiment

8. … Starting from the top… End Users Search Center Data Sources
TechReady12
11/16/2018
Starting from the top…
Pipeline Extensibility (FS4SP only)
End Users
Data Sources
Federation
OpenSearch Source
Content Processor
Crawler
Indexer
Query Processor
Search Center
User Profiles
Index Partition …
Format Conversion
Language Detection
Entity Extraction
Lemmatization
Mapper
Federation Object Model
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9. Feeding Flow – SharePoint App. Server
Request crawl
Web crawls
FAST Content SSA Admin DB
Database
Poll request
Admin component
Content Web Service
FAST Content SSA Crawl DB
Log request
Poll request
Master Crawl comp.
Crawl data
Crawl history
Crawl queue additions
Distribute work
Crawl comp.
Crawl comp.
Crawl comp.
Document batches
FAST Search

10. Feeding Flow – FAST Side
TechReady12
11/16/2018
Feeding Flow – FAST Side
FAST Content SSA
FAST Admin DB
FAST Query SSA Admin DB
Crawled
properties
detected
Document batch
Click-through history
query  url
Content Distributor
Crawled
property
mappings
Content Distributor
Document batch
Link DB
Item Processing
Document metadata
Web Analysis
Item Processing
Web Analysis
FIXML
Links and anchor texts
Indexing dispatcher
Indexing dispatcher
FIXML
Local disk
Local disk
Binary index
Indexer
Indexer
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11. Process content with advanced linguistics
Automatic and detailed analysis creates a great search experience
FAST Content Processing Pipeline
Tokenization
Apply the language specific rules for identifying words, concepts, idioms and phrases. Also applies custom word breakers found in part numbers or telephone numbers.
Language Encoding and Detection
Identifies the native written language and locale specific encoding so that the proper dictionaries can be used by the tokenization and lemmatization stages
Format Conversion
Extracts plain text from multiple file formats, encodings, and applications
Lemmatization
Finds the root of a word for a given language. For English it maps run, runs, running and ran back to a single lemma. Understands language specific grammar and context.
Date and Time Normalization
Converts dates and times to a standard representation, to handle locale specific representations. For example, knows that 14-Mar-10 is equivalent March 14, 2010.
Map Crawled Properties
Maps all of the metadata that was discovered by the various pipeline stages
Web Link Analysis
Analyzes documents for hyperlinks extracting anchor text which reinforces the authority ranking of a document.
Document Vector
Creates a unique representation of a document that reflects important terms and frequency of occurrence. Used to find similar documents.
Entity Extraction
Finds terms in the content and maps them to predefined categories. Out of the box support for People, Companies and Locations, but can be extended to any category.
Speaker Notes: This slide builds, so ensure you discuss each build point.
Speaker Script:
One of the key differentiators between FAST Search Server 2010 for SharePoint and other search products is the ability for you to add logic to the content processing pipeline, which is used at crawl and indexing time.
[Build] The first stage in the pipeline is to extract the text from content, regardless of the document type and format.
[Build] Then the language and encoding of the text is determined, which helps with later stages in the pipeline (such as defining which dictionaries and language rules to use for word analysis)
[Build] The tokenization stage breaks the stream of text down into individual words by using language- specific word breakers. FAST Search Server 2010 for SharePoint includes very powerful word-breaking capabilities based on spaces and punctuation, but also on language-specific rules for dealing with compound words.
[Build] After words have been broken down, FAST Search Server 2010 for SharePoint analyzes the words back to their stem form. This process, which is called lemmatization, is language specific, and FAST Search Server 2010 for SharePoint includes very comprehensive rules for applying this stage.
[Build] As I discussed earlier, you can create your own entity extractors to enable your users to search in the terms and language of your business. The entity extraction process happens at this stage in pipeline. FAST Search Server 2010 for SharePoint includes some common entity extractors by default, but you can add to those.
[Build] The next stage is to normalize dates and times that were found in document contents and metadata, so that features such as sorting, filtering, and refinement will work consistently.
[Build] Then a document vector is generated for each document, which represents an overall analysis of the contents and metadata of the item. This vector is used to compare document similarity, which enables the 'Similar Results' feature.
[Build] Hyperlinks that point to the document being indexed are then analyzed in terms of their anchor text. This is a good indication of how authoritative the document is when it is returned by a search that contained search terms which match the anchor text.
[Build] The final step is to map the metadata and entity values to managed properties, so that the search schema is kept up-to-date.
Breaks down content to the smallest addressable chunks to build meaning
Understands file encoding, data formats, and written languages
Supports 400+ file formats, 80+ languages

12. Start with FS4SP Pipeline Extensibility
TechReady12
11/16/2018
Start with FS4SP Pipeline Extensibility
XML Input (file)
EXE
XML Output (file)
PipelineExtensibility.xml
Your EXE
Bing  PipelineExtensibility.xml
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

13. demo Build it… TechReady12 11/16/2018
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. Extending the UX

15. Extending the UX

16. where the magic happens
Core Results WebPart
where the magic happens
All other Search WebParts depend on this one.
Interacts with the Federation OM

17. How the Core Results WebPart Does It.
Inherits from the DataFormWebPart
Allows for custom XSLT for various rendering options
Uses the DataSource and DataSourceView classes to configure the query
Retrieve the results
Key Takeaway: Use the DataSource and DataSourceView classes to customize the query

18. Conceptual View of a Search Query
Federation OM
DataSourceView
Location Configuration
DataSource
Core Results Web Part
Paging, Stats, etc…
Refinement Panel
Query, WebPart Properties (Defaults from Location)
Returns results to WebParts and Refiners to Refinement Panel
Location type and location configuration

19. demo Build it… TechReady12 11/16/2018
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20. where the magic happens
Visual Refiners
Core Results WebPart
where the magic happens
All other Search WebParts depend on this one.
Interacts with the Federation OM
Non-Search Related

21. In Review: Session Objectives and Takeaways
TechReady12
11/16/2018
In Review: Session Objectives and Takeaways
Session Objective(s):
Identify the customization options and solutions/tools available for FAST Search for SharePoint.
Identify the issues and concerns with respect to customizations and try to overcome them with new innovations or strategies.
Address customers with advanced document processing requirements using FS4SP's pipeline extensibility mechanism.
Key Takeaways
Knowledge of Pipeline Extensibility
Knowledge of Federation Object Model usage
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. TechReady12
11/16/2018
Wrap Up: Where to look
TechNet Script Center:
SharePoint  Search Management
SharePoint  Deployment and Upgrade
FAST Search for SharePoint TechNet Site:
Mapping crawled to managed properties:
Document Processing configuration
Optional Processing options:
Pipeline Extensibility:
Adding addition IFilters:
Enabling the Advanced Filter pack:
User defined property sets:
XMLMapper protocol spec:
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. Q&A

24. 11/16/2018 4:57 AM
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.