1. Architecting Search in SharePoint 2016
Ajay Iyer
Sr. Consultant (MCS)

2. Thank You Sponsors for participating in SPS St. Louis 2017!
You can use the hashtag #SPSSTL & follow Gold Sponsors Silver Sponsors

3. Ajay Iyer SharePoint Architect @shankarajay1
Solution Architecture, Capacity Planning, Search, Migrations, Enterprise Content Management, Document Imaging and….. InfoPath

4. Outline What’s new in SP2016 & Search?
Review Search Components & Terminologies
Gather Requirements for Search (What specifics should you ask?)
Search Farm Architecture & Design
Incremental Crawls or Continuous Crawls?

5. What’s new in SP2016

6. What’s New in SP2016

7. MinRole Auto-Provisioning of SP Services/Service App Endpoints
Provisioning is based on Server Roles
Ensures each server in farm is running services it needs
Farm Admins can manage services at farm-level, not server-level
Improves farm reliability
Simplified Capacity Planning
Choose “Custom” role to manage services on server-level

8. What’s New in SP2016 Search
Each Search Service Application now supports indexing up to 500 million items (SP2013 supported 250 million items)
Searching for sensitive documents. E.g. documents with SSNs, credit cards, passport numbers etc. in them, using Data Loss Prevention (DLP)
Hybrid Search is available in SharePoint 2016 and SharePoint September 2015 CU
Can crawl Office 365 content as well as On-Prem content and get unified results across both.

9. Other Notable Changes Supports upload & download of files > 2GB
Filenames can now include special characters
&, ~, {, } are allowed
# and % are still NOT allowed
Search now supports indexing up to 500 million items (SP2013 supported 250 million items)
STSADM is deprecated! PowerShell is your best friend!
SharePoint Foundation NOT available for SP2016
Easy User Profile Synch – Microsoft Identity Manager!
Excel Services are now part of Excel Online
10 million per index partition and up to 25 index partitions per search service application

10. Upgrade
Farm Requirements
SharePoint Server March 2013 PU (v ) or higher
All content databases that need to be upgraded should at least also be on v
Use Database Attach Upgrade
Make sure no site collections/subsites are in 2010 Compatibility Mode
Custom code from SP2013 “should” work with SP2016
Source:
General purpose slide

11. Review of Search Components
Search Roles
Search Databases
Search Administration
Crawl
Content Processing
Analytics Processing
Index
Query Processing
Search Administration DB Crawl DB Analytics Reporting DB Link DB
Search Roles
Crawl component crawls the data and discovers all the properties. It then goes to the content processing component (CPC) that parses the documents, extracts metadata and creates properties mappings. It then sends this all to the Indexing component. Index component performs link extraction of all the documents and stores them in the links db. These links are then processed by the Analytics Processing component (APC). APC also processes information like user clicks, referrer, etc. The Query Processing Component (QPC) performs word breaking, linguistics/stemming and parsing operations. It runs under the noderunner.exe service. It no longer queries the search db for security trimming info. It gets that from the index.
Search Administration: The Search Administration database hosts the Search service application configuration and access control list (ACL) for the crawl component.
Analytics Reporting: The Analytics Reporting database stores the results for usage analysis reports and extracts information from the Link database when needed.
Crawl: The Crawl database stores the state of the crawled data and the crawl history.
Link: The Link database stores the information that is extracted by the content processing component and the click through information

12. Other Search-related Terminologies
No. of items in Index
Search Topology
Index Partitions
Content Sources
Incremental Crawls/Continuous Crawls
Full Crawls
Crawled Properties
Managed Properties
General purpose slide

13. Review of Search Components
WFEs
Crawl
Content Processing
Index
Crawl DB
Query Processing
Analytics Processing
Links DB
Crawl component crawls the data and discovers all the properties. It then goes to the content processing component (CPC) that parses the documents, extracts metadata and creates properties mappings. It then sends this all to the Indexing component. Index component performs link extraction of all the documents and stores them in the links db. These links are then processed by the Analytics Processing component (APC). APC also processes information like user clicks, referrer, etc. The Query Processing Component (QPC) performs word breaking, linguistics/stemming and parsing operations. It runs under the noderunner.exe service. It no longer queries the search db for security trimming info. It gets that from the index.
Crawl DB
Analytics DB
Search Index

14. Gathering Requirements for Search

15. Gathering Requirements for Search
Questions
How many web applications & site collections in current environment?
How many total documents?
What are the different document file types? PDFs, TIFFs, DOCX, XLSX, etc. ?
Are the PDF’s, image-only or searchable?
What’s the frequency of content additions and changes?
How quickly do the users expect their documents to be searchable, after uploading to SharePoint?
Are there any Line-Of-Business external applications that interface with SharePoint?

16. Gathering Requirements for Search
Sample Response
Questions
Response
No. of Web Apps 2
No. of Site Collections 25
Total No. of Documents
3,980,272
Document File Types
PDFs (mostly), approx. 25,000 office documents, approx TIFFs
PDF Types
Mostly Searchable PDFs, and approx Image-Only PDFs
Frequency of content change
Approx. 200 documents added per hour
How soon should docs be retrievable from search
10 minutes or less

17. Search Farm Architecture & Design

18. Search Farm Architecture & Design
Sample Search Farm Design
WFE 1
APP 1
Office Online Server
DB Server
APP 2
Workflow Server
WFE 2

19. Search Farm Architecture & Design
Sample Search Farm Design
WFE 1
Crawl/Search Admin components
Query component
APP 1
CRWLR 1
Office Web Apps
DB Server
APP 2
CRWLR 1
WAWS
Crawl/Search Admin components
WFE 2
Query component

20. Search Servers Specifications
Search Farm Architecture & Design
Search Servers Specifications
Crawl & Search Admin Servers:
8 vCPUs
32GB RAM
80GB - System Drive
120GB - Data Drive (SP Binaries, Data, Logs)
40GB – Drive for Search Index

21. Search Farm Architecture & Design
Search Servers Specifications
Analytics & Content Processing Servers:
4 vCPUs
16GB RAM
80GB - System Drive
120GB - Data Drive (SP Binaries, Data, Logs)

22. Search Farm Architecture & Design
Search Servers Specifications
Query Servers:
4 vCPUs
16GB RAM
80GB - System Drive
120GB - Data Drive (SP Binaries, Data, Logs)
40GB – Drive for Search Index

23. Incremental Crawls or Continuous Crawls??
Search Farm Architecture & Design
Incremental Crawls or Continuous Crawls??

24. Search Farm Architecture & Design
Possible Resolutions:
Check disk I/O on the SQL server data drive
Move the Search Databases to a different LUN (if virtual) or DISK (if physical)
Change the Search Performance level from the default “Maximum” to “Partly Reduced” or “Reduced”
Here’s how you fix it using PowerShell
Check memory utilization on the crawl & query servers
Look for RAM utilization on the msssearch.exe and noderunner.exe services

25. Additional Questions?
Questions slide

26. Thank You
Ajay Iyer Sr. SharePoint Consultant

27. Thank You Sponsors for participating in SPS St. Louis 2017!
You can use the hashtag #SPSSTL & follow Gold Sponsors Silver Sponsors