1. Toronto SharePoint User Group
Bill Brockbank Navantis Inc.

2. Architecture and Topology
Logical Architecture - Components
Crawl
Content Processing
Analytics Processing
Index
Administration
Query Processing
Search Process
Topologies - Examples

3. Logical Architecture

4. Crawling the Content
The crawl role is responsible for crawling content sources. It delivers crawled items – both the actual content as well as their associated metadata – to the content processing component
Invokes connectors or protocol handlers to content sources to retrieve data
Does not do any document parsing (Content Processing Component does that)
Information about content sources, schedules, etc. are synchronized to the registry on crawl role servers from the search admin database
The Crawl Database is used by the crawl component to store information about crawled items and to track crawl history
Holds information such as the last crawl time, the last crawl ID and the type of update during the last craw.

5. Content Processing Content Processing Component
Processes crawled items and feeds these items to the index component
Document Parsing happens through Format Handlers
iFilter is supported through a Generic iFilter format handler
iFilters are still the extensibility platform for SharePoint 15
Heart of the indexing process: transforms crawled items into artifacts that can be included in the search index by carrying out operations such as document parsing and property mapping
Performs linguistic processing at index time (e.g. language detection and entity extraction)
Writes information about links and URLs to the Link Database directly
Generates phonetic name variations for people search

6. Analytics Processing Component
Search Analytics analyzes crawled items and how users interact with search results.
Usage analytics analyzes usage events e.g. like views from the event store
When an user does an action (e.g. view a page) the event is collected in usage files on the WFE’s and regularly pushed to event store where they are stored until processed
Results are then returned to the Content Processing Component to be included in the search index
You can add write code to add custom events
Analytics Processing Component supports scaling out:
Add more APC roles to have analysis complete faster
Adding more Link databases to increase capacity for links and search clicks, and potentially speed up the DB extraction
Adding more reporting databases to store more reports as well as improving SQL throughtput in retrieveing reports

7. List of Sub Analyses Search Analytics Usage Analytics
Link and Anchor text analysis
Click Distance
Search Clicks
Deep Links
Social Tags
Social Distance
Search Reports
Usage Analytics
Recommendations
Usage Counts
Acivity Ranking

8. Index Component Used in both feeding and query processes:
Feeding: receives processed items from the content processing component and writes those items to index files
Query: receives queries from the query processing component and provides results sets in return
It also physically moves around indexed content when the index architecture is changed by the Search Administration Component

9. Query Processing Component
Performs linguistic processing at query time:
Word breaking, stemming, query spellchecking, thesaurus
Analyzes and processes search queries and results.
When the component receives a query from the search front-end, it analyzes and processes the query to attempt to optimize precision, recall and relevancy. The processed query is then submitted to the index component(s).
As part of this it also decides which query rules are applicable, which index to send the query to, and whether to do any pre- or post- processing of the query
The index component returns a result set based on the processed query back to the query processing component, which in turn processes that result set before sending it back to the search front-end.

10. Search Administration
Search Admin Component
Runs number of system processes required for search
Is responsible for search provisioning and topology changes
Coordinates search components – Content Processing, Query Processing, Analytics, and Indexing.
Search Admin DB
Stores search configuration data:
Topology
Crawl rules
Query rules
Managed property mappings
Content sources
Crawl schedules
Stores Analytics settings
Does not store ACLs anymore

11. Search Processes MSSearch.exe Host Controller NodeRunner.exe
A Windows Service that supervises NodeRunner process(es) on a given box
It restarts failed NodeRunner processes: if a NodeRunner fails, the HostController will detect this and restart it
NodeRunner.exe
Is the process that hosts the search components
There might be several instances of this process on a single box
MSSearch.exe
Is the Windows Service who hosts the Crawl Component

12. Search Host Process
NodeRunner.exe
Crawl Component
NodeRunner.exe
Content Processing Component
Multiple NodeRunner instances can run on the same server
Each NodeRunner instance hosts one search component
E.g. If you have two index replicas on one server you will have two NodeRunner instances – one for each replica
On a default single server install there will be 5 instances of the NodeRunner.exe process
NodeRunner.exe
Query Processing Component
NodeRunner.exe
Index Component
NodeRunner.exe
Analytic Processing Component

13. Enterprise Search – Query Configuration

14. Query Configuration
Ranking
Query Spell Correction
Query Rules

15. Ranking
Ranking model describes which criteria are included in sorting, how much they contribute to the rank score and how they relate to one another
Custom ranking models are managed through PowerShell and the public OM using XML files as it was in SharePoint 2010
Ranking model for a specific query can be selected at query time by setting the RankingModelId of the query; otherwise default is used
NOTE: This property is currently not available in the OOB search results web part

16. Working with Ranking Models
A search service administrator can do the following basic operations on the rank models:
List ranking models
Specify a default ranking model
Change an existing custom ranking model
Delete an existing custom ranking model
Create a new ranking model (either based on an existing one or from scratch)
Import and export a ranking model to XML
Specify how much weight the various properties should have in the model
These operations are all administered through PowerShell
There may be a CodePlex GUI-based solution after RTM

17. Query Spelling Correction
Customizations to Query Spelling Correction is now managed in the term store – both inclusions and exclusions
We also still have a dynamic dictionary that is created based on content in the index, or you can switch to the static OOB dictionary
Here’s an example:
Provides customizable
“did you mean” functionality:

18. Query Rules
Query Rules are a new feature in SharePoint 15 that help act upon the “intent” of a query
Query Rules are composed of three top level elements:
Query Conditions (i.e. matching rules)
Query Actions (i.e. what do you do when you find a match)
Publishing Options (i.e. when should this rule be active)
Rules are created at the site collection level, but you will probably also be able to create them at the SSA level through PowerShell (TBD by RTM)
Several Query Rules are provided out of the box and can be found in the Site Settings for each site collection

19. Query Rules Implementation
Query Rules allow you to have search requests from a user trigger multiple queries and multiple result sets.
This differs from SharePoint 2010 where there were only simple queries – one query has one set of results.
All matching Query Rules can generate results; the query orchestration engine retrieves all the different sets of search results and organizes them for rendering to the user
The results themselves are then rendered using a new feature called Query Results display templates, which is discussed later

20. Query Rules - Conditions
Conditions allow you to define the circumstances under which your query rule will apply
You can choose to match on these use cases:
Query contains a specific word or words
Query contains a word in a specific dictionary
Query contains an action word that matches a specific phrase or term set
Query is common in a different source (like Videos Result Source)
Results include a common result type (like file type)
Advanced rule – match across a set of terms, dictionary, regular expression, etc.
You can also match if a query is against a specific Result Source or category (like a Topic Page category)
Each Query Rule can have multiple conditions

21. Query Rule Actions There are three types of actions:
Assign a “promoted result” to the query
Promoted result is similar to Best Bets with SharePoint 2010 or Visual Best Bets with FAST Search 2010, but much more flexible
Create and display a Result Block
Result blocks specify an additional query to run and how to display results
The feature includes a full query designer so you can build and test queries before finalizing them
You can include the results above those returned by core results, or interleaved by ranking
You can also choose custom display templates instead of the default for the result
Change the query that returns the core results
Change the original query by changing the query terms, adding additional terms, apply an XRANK formula to the query, etc.
You can add multiple best bets or results blocks for each query rule

22. Query Rules Publishing Options
Publishing options for a Query Rule determine when it will be used
It works just like rules for publishing a page in a publishing site template
You decide whether the rule is active
You decide when to start and when to finish using the rule, or don’t set dates at all and it’s always used
You can configure a review date, which is basically an reminder for you to look at the rule again
Publishing a rule is particularly useful in commerce scenarios. For example, queries that should return items you want to percolate to the top and sell during a particular period of time

23. Enterprise Search – New Search Results Display Options

24. Search UI Configuration
Result Types
Display Templates
Search Navigation
Search Refinement
Query Suggestions
Thumbnail Previews
Site Level Search Admin Summary

25. Result Types
SharePoint 15 includes a new framework for presenting search results to end users called result types
Result types have several elements that enable each type to be rendered differently:

26. Result Type Rules
You can create a variety of different rules on which to match:
Equality(= or !=), comparison (< or >), or logical (AND or OR or NOT)
You can have multiple rules as a condition
You can use managed properties in a rule
In B1R there will be a new UI for creating these rules, so what you see in the B1 bits will change quite a bit
Example
Name
Rule
Matches…
Word
FileExtension=docx OR FileExtension =doc
All Word documents
Spec
ContentType=Spec AND
(FileExtension=docx OR FileExtension =doc)
Specs which are also Word documents

27. Result Type Management
Result types are managed as follows:
Result Types can apply to an entire search service application or to a specific site or web
OOB Result Types are immutable but can be overridden with a copy
There is an inheritance hierarchy to result types in a site: the current web, plus the site collection root, plus the Search Service Application.
So if you add a new result type to some random web, the sub webs under it won’t inherit that type unless you are working in the site collection root web

28. Display Templates
Display templates contain the following characteristics:
They define the visual layout of a Result Type
A template is really just HTML
You can edit it with ANY HTML designer you want – DreamWeaver, SharePoint Designer, Visual Studio, Notepad – doesn’t matter
You add placeholders to your HTML file where managed properties should be emitted
It gives you a real WYSIWIG experience when designing templates
A set of display templates is included OOB; you can find them in the Master Pages/Display Templates/Search folder
Site collection admins can upload new display templates.

29. Result Type with Display Template Example
Here’s an example from one of the Hands On Labs that demonstrates building a custom result type and display template for .TXT file search results:

30. Steve Peschka Sr. Principal Consultant Microsoft Corporation
Result Types
demo
Steve Peschka
Sr. Principal Consultant
Microsoft Corporation

31. Search Refinement
There are two different modes for the refiner web part: search results and faceted navigation
With search results the refinement data works essentially the same as SharePoint 2010
With faceted navigation it uses a term from the term store to filter what kind of data should be displayed (explained on next slide)
Refinement is different with SharePoint 15 in that you can define display templates to use for rendering different kinds of refinements
In SharePoint 2010 you had to write your own custom refiner

32. Faceted Navigation with Search Refiners
With Faceted Navigation, it is used in conjunction with term sets that are used for navigation
With each term you select which managed properties should be used as refiners with that term
Within the managed property you need to configure it as “Refinable”
Example:
You have term store terms Camera and Laptop
You have managed properties Megapixel Count, Color and Manufacturer
For Camera term, you add refiners for Megapixel Count and Manufacturer
For Laptop term you add refiners for Color and Manufacturer
Another Example:
Department is the term store term, Customers and Projects could be refiners

33. Query Suggestions
Query suggestions take a big leap forward in usability. It improves on the experience in SharePoint 2010 as follows:
Your personal SharePoint activity factors into the query suggestions, i.e. you have a personal query log
It includes weighting based on sites that you have previously visited
It uses the most frequent queries across all users that “match” the search terms
The behavior of the query suggestions turns into more of a “browse and find” kind of experience
You can also add inclusion and exclusion lists for suggestions via the SSA admin pages

34. Query Suggestion Types
There are two types of query suggestions – what you see when you are entering a query, and what you see when you get results
When entering a query you will see two types of suggestions:
A list of items you have clicked on before from your personal query log
A list of items that others are typing for their queries
When you get query results back, you will get another set of suggestions
They are a list of links that you have clicked through at least twice before and match your search criteria

35. Query Suggestion Examples
Here are some examples of how query suggestions might look:
Before the Query Suggestion
Above are items based on what other users have queried and found
Bottom 3 items are ones you clicked before
After the Query Suggestion
Top 3 items are ones you clicked before

36. Thumbnail Previews
The new Office Web Apps is the engine for thumbnail previews in SharePoint 15
The BIG WIN HERE – you can now browse through the entire document in the preview
See all pages, see animations, zoom in, scroll through the entire document
The point of this is to allow users to find the exact item they’re looking for right in search results – no more clicking a result, hitting the back button, and on and on until they find the one they’re looking for
It also addresses the two major shortcomings of thumbnails in SharePoint 2010:
It could only be used with FAST Search
It did NOT work with claims authentication 
Since there’s only one search engine in SharePoint 15 you get document previews out of the box
In a different twist, previews only work with claims authentication – it will not work with classic Windows authentication

37. Summary: New Site Search Admin Tasks
Just to summarize, here are things site admins can do to manage search themselves:
Task
Site Collection Admin
Site Admin
Create and/or override Query Rules, including Promoted Results
Create and/or override Result types and display templates
Create and/or override Result Sources, either for remote locations or as a custom search “vertical”
Create managed properties
Create refiners
Start a local crawl – can even be done down to the list level

38. Online Content links:
IT Pro training for SharePoint
SharePoint 2013 enterprise search Module: