Effective Search Deployment and Operations in SharePoint 2013

Slides:



Advertisements
Similar presentations
Module 13: Performance Tuning. Overview Performance tuning methodologies Instance level Database level Application level Overview of tools and techniques.
Advertisements

Gold Sponsors Bronze Sponsors Silver Sponsors Taking SharePoint to the Cloud Aaron Saikovski Readify – Software Solution Specialist.
Intro to SharePoint 2013 Architecture Liam Cleary.
A Java Architecture for the Internet of Things Noel Poore, Architect Pete St. Pierre, Product Manager Java Platform Group, Internet of Things September.
   HTTP File shares SharePoint User profiles Lotus Notes Documentum Exchange folders Custom - BCS SharePoint SP Apps Devices Non-SP UX Search.
Design Solutions for a SharePoint in Flux. Agenda © 2014 Jamie McAllister.
SharePoint 2010 Search Deep Dive
SharePoint de Contact Search and Find With SharePoint 2010 Thierry Gasser Technical Specialist Collaboration Platform
Implementation Considerations for FAST Search For SharePoint (FS4SP) Presenter : Shyam Narayan MOSSIG – February 2011 Meeting b:
Enterprise Search With SharePoint Portal Server V2 Steve Tullis, Program Manager, Business Portal Group 3/5/2003.
You can do it! Quick, easy, powerful search (for free!) Complete intranet search High-end search delivered through SharePoint Basic search Intranet-wide.

Microsoft ® Official Course Interacting with the Search Service Microsoft SharePoint 2013 SharePoint Practice.
Microsoft SharePoint 2013 SharePoint 2013 as a Developer Platform
Thank you SPSKC15 sponsors!. SharePoint 2013 Search Service Application (SSA) Ambar Nirgudkar Software Engineer
Yes, yes it does! 1.Guest Clustering is supported with SQL Server when running a guest operating system of Windows Server 2008 SP2 or newer.
SharePoint Server 2013 Architecture and Identity
Presented by Jacob Wilson SharePoint Practice Lead Bross Group 1.
Capacity Planning in SharePoint Capacity Planning Process of evaluating a technology … Deciding … Hardware … Variety of Ways Different Services.
Cost Effort Complexity Benefit Cloud Hosted Low Cost Agile Integrated Fully Supported.
Title Information First Lastname, Title August 2012 Software Assurance Planning Services.
Russ Houberg Senior Technical Architect, MCM KnowledgeLake, Inc.
Search Admin Content UX Crawl Content Processing Index Query ProcessingWFE Analytics Processing FAST Search Index SharePoint SP Apps Devices Non-SP.
©2012 Microsoft Corporation. All rights reserved..
4/20/2017 © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks.
Manage & Configure SQL Database on the Cloud Haishi Bai Technical Evangelist Microsoft.
SPC095 Search Admin Content UX Crawl Content Processing Index Query ProcessingWFE Analytics Processing FAST Search Index SharePoint SP Apps Devices.
SharePoint Farm On Azure IAAS Prepared By : Prakhar Rastogi Premier Field engineer Microsoft India.
©2012 Microsoft Corporation. All rights reserved. Content based on SharePoint 15 Technical Preview and published July 2012.
Search Topology and Optimization April 12, 2013 Mike Maadarani SharePoint Architect.
First Look Clinic: What’s New for IT Professionals in Microsoft® SharePoint® Server 2013 Sayed Ali (MCTS, MCITP, MCT, MCSA, MCSE )
Maintaining a Microsoft SQL Server 2008 Database SQLServer-Training.com.
Module 8 Configuring and Securing SharePoint Services and Service Applications.
TechEd /22/2017 5:40 AM © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks.
Building Search Portals With SP2013 Search. 2 SharePoint 2013 Search  Introduction  Changes in the Architecture  Result Sources  Query Rules/Result.
©2012 Microsoft Corporation. All rights reserved. Content based on SharePoint 15 Technical Preview and published July 2012.
Search 2013 Thierry Gasser Technical Solution Professional (TSP)
Module 10: Monitoring ISA Server Overview Monitoring Overview Configuring Alerts Configuring Session Monitoring Configuring Logging Configuring.
HTTP File shares SharePoint User profiles Lotus Notes Documentum Exchange folders Custom - BCS SharePoint SP Apps Devices Non-SP UX Search Admin Content.
SharePoint 2010 Search Architecture The Connector Framework Enhancing the Search User Interface Creating Custom Ranking Models.
Search Admin Content UX Crawl Content Processing Index Query ProcessingWFE API Analytics Processing Crawl Search Admin Link Analytics Reporting FAST.
ProLiant Gen 8: What’s New Next generation ProLiant with Active Insight architecture Operational efficiency Application acceleration & availability Always.
Module 13 Implementing Business Continuity. Module Overview Protecting and Recovering Content Working with Backup and Restore for Disaster Recovery Implementing.
Module 10 Administering and Configuring SharePoint Search.
Azure IAAS Overview & What’s New Steve Letford & Stu Fox M318.
Module 14 Monitoring and Optimizing SharePoint Performance.
0 SharePoint Search 2013 Rafael de la Cruz SharePoint Developer Seneca Resources twitter.com/delacruz_rafael
Búsqueda en SharePoint 2010: una introducción. Quick, easy, powerful search (for free!) Complete intranet search High-end search delivered through SharePoint.
1 | SharePoint Saturday Calgary – 31 MAY 2014 About Me.
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Architecting Search in 2013/2016 On-Prem Ajay Iyer.
The best of WF 4.0 and AppFabric Damir Dobric MVP-Connected System Developer Microsoft Connected System Division Advisor Visual Studio Inner Circle member.
Configuring SQL Server for a successful SharePoint Server Deployment Haaron Gonzalez Solution Architect & Consultant Microsoft MVP SharePoint Server
Hybrid Identity Deep dive Ross Adams 2016 Redmond Summit | Identity Without Boundaries May 25 th 2016 Azure AD
Productivity Architect Meet Chris Bortlik Author, Blogger, Speaker.
Managing Microsoft SQL 2000 with MOM MOM Overview Why Monitor SMS 2003 with MOM 2005 The SMS 2003 Management Pack Inside The Management Pack Best.
Blog.eardley.org.uk SharePoint Databases What you need to know Alan Eardley SQL Saturday Exeter 25 April 2015.
Hybrid SharePoint Search
Architecting Search in SharePoint 2016
SharePoint 2010 Performance and Capacity Planning Best Practices
Lead SQL BankofAmerica Blog: SQLHarry.com
Migration to SharePoint 2013
Toronto SharePoint User Group
Multi-Farm, Cross-Continent SharePoint Architecture
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
SharePoint Online Hybrid – Configure Outbound Search
SharePoint 2013 Enterprise Search Topology
Technical Capabilities
AI Discovery Template IBM Cloud Architecture Center
SharePoint 2013 Best Practices
Presentation transcript:

Effective Search Deployment and Operations in SharePoint 2013 Microsoft Ignite 2015 4/16/2017 4:11 PM BRK3176 Effective Search Deployment and Operations in SharePoint 2013 Kristopher Loranger Sr PFE - Search Brian Pendergrass Sr PFE - Search © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Architectural Assessments Microsoft Ignite 2015 4/16/2017 4:11 PM Migration Planning Search Engineers Scalability Arun Selvaraj Brent Groom Brian Pendergrass Chris Stone Dan Harrington Eric Dixon Filipe Moreira Humberto Giuntoli Igor Veytskin Jon Waite Kristopher Loranger Nathan Brackett Nick Meader Russ Maxwell Proof of Concepts Best Practices Relevancy Tuning Architectural Assessments © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Is this normal? Planning the trip: Architecture, Ops, and Admin SPC2012 – IT Pro 4/16/2017 Planning the trip: Architecture, Ops, and Admin Automate monitoring of SSA Analyze performance reports Review usage reports for trends Is this normal? Result Sources / Query Rules / Refiners Managing performance Backup / DR Frequent Infrequent Security updates + Cumulative updates Sizing / Scaling (and Migrations) Relevancy tuning © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Search Architecture Microsoft Ignite 2015 4/16/2017 4:11 PM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SharePoint 2013 Search Architecture Overview Search Admin Admin Search SearchCore * noderunner.exe Content Crawl Component Crawl Components Content Processing Content Processing Components Index Components Query Processing (IMS) Components Web App Index Components FAST Search Core Index Interaction Engine Content Engine WFE Search Core CFE Crawler Mars Engine Index Interaction Management Services Content Transformation Services Gatherer FastServer Connectors * * * mssearch.exe noderunner.exe noderunner.exe noderunner.exe w3wp.exe events usage Analysis Engine (APC) Analytics Processing Components noderunner.exe Analytics Processing Engine *Juno Components (noderunner.exe) *

Architecture: Deployment Best Practices 4/16/2017 4:11 PM Architecture: Deployment Best Practices Operating System Disk Layout Disable Antivirus scanning for all Search Data volumes (and for all Search processes) At install, set “DataDirectory” to non-system drive Separate I/O intensive paths to dedicated drives Enable “High Performance” power plan (and in BIOS, enable OS control of CPU power throttling) SharePoint: OS, ULS/logging, Search Data/Index SQL Server: OS, DBs, transaction logs, tempDB Use a time zone without daylight savings time For virtual disks, use fixed size VHDs only Service Instances Data Volume Only start Search Query & Site Settings ( SQ&SS) on servers with a Query Processing Component Ensure IOPS requirements Avoid disk compression and adaptive read-ahead Stop all non-search Service Instances (e.g. WFE) Format Data disk volumes as NTFS with 64kB blocks Network Disable OS from “indexing” contents/properties 1 Gbit/s is minimum, 10 Gbit/s is better Multiple network adapters are supported 2 1 4 3 © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Classic Search Farm (SP2013) 4/16/2017 Architecture: The Options in Broad Strokes Corporate Network Boundary Cloud SSA (SP2013 + SP2016) One single search across cloud and on-premises content, with capabilities like Delve for both Search Admin Crawler Content Farm(s) (SPO) Search Index Classic Search Farm (SP2013) Search Index Implementing Next Generation SharePoint Hybrid Search with the Cloud Search Service Application Wednesday, May 6th 05:00PM - 06:15PM © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Architecture: Azure IaaS and DR 4/16/2017 Architecture: Azure IaaS and DR Microsoft Azure SP Search “On Prem” Warm standby or Cold Standby CPC APC Adm Crawl CPC APC Adm Crawl Index0 QPC Index0 QPC Cloud Service VPN Gateway Restore SSA July 2014 CU Cloud Service Virtual Network © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Monitoring Search Microsoft Ignite 2015 4/16/2017 4:11 PM © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SharePoint 2013 Search Architecture Overview Monitoring with SCOM: Crawl Components Crawl Component Crawl Components Query Processing (IMS) Components Content Processing Content Processing Components UI Analysis Engine (APC) Analytics Processing Index Components FAST Search Core Index SearchCore Content noderunner.exe Alert: Disk Full Crawler Threshold: Search Gatherer Projects - SharePointServerSearch\Transactions Waiting > N CC

SharePoint 2013 Search Architecture Overview Monitoring with SCOM: Content Processing Crawl Component Crawl Components Query Processing (IMS) Components Content Processing Content Processing Components UI Analysis Engine (APC) Analytics Processing Index Components FAST Search Core Index SearchCore Content noderunner.exe Alert: Gatherer Content Processing connector Threshold: Search Flow Statistics\Input Queue Full Time > 1000 CPC

SharePoint 2013 Search Architecture Overview Monitoring with SCOM: Index Components Crawl Component Crawl Components Query Processing (IMS) Components Content Processing Content Processing Components UI Analysis Engine (APC) Analytics Processing Index Components FAST Search Core Index SearchCore Content noderunner.exe Alert: Missing partition Threshold: Search Index\DocsPerIndexCell = 9-9.5Mill Index

SharePoint 2013 Search Architecture Overview Monitoring with SCOM: Query Processing Crawl Component Crawl Components Query Processing (IMS) Components Content Processing Content Processing Components UI Analysis Engine (APC) Analytics Processing Index Components FAST Search Core Index SearchCore Content noderunner.exe Alert: Service availability query processing Threshold: Search Query Processing\Failed QPS > 3 QPC

SharePoint 2013 Search Architecture Overview Monitoring with SCOM: Analytics Processing Crawl Component Crawl Components Query Processing (IMS) Components Content Processing Content Processing Components UI Analysis Engine (APC) Analytics Processing Index Components FAST Search Core Index SearchCore Content noderunner.exe Alert: Analytics analysis: failed to start Threshold: Search Analysis Engine Analytics Processing Components\LongRunningTasks > 6 hours APC

Architecture: Search Topology … Search Admin

Synchronize Constellation Tag: xmnv Product: SharePoint Foundation Level: Medium Message: Name=Timer Job job-application-server-admin-service 3ec1589c-6cc5-9096-7fea-7345dcae1238 Tag: dkd5 Product: SharePoint Server Search Level: High Message: synchronizing search service instance 3ec1589c-6cc5-9096-7fea-7345dcae1238 Tag: drx4 Category: GatherSvc Level: High Message: initializing gatherer application –SSA-ID-crawl-0 Tag: agwve Product: SharePoint Server Search Level: Medium Message: Synchronizing legacy admin 3ec1589c-6cc5-9096-7fea-7345dcae1238 Tag: aie8r Product: SharePoint Server Search Level: Medium Message: MasterMergeScheduleManager: Start trigger checks 3ec1589c-6cc5-9096-7fea-7345dcae1238 Tag: aih3f Category: Search Component Level: Verbose Message: Retrieving configuration from database - AdminComponent, 1904, Constellation 3ec1589c-6cc5-9096-7fea-7345dcae1238 Tag: ajncw Product: SharePoint Server Search Level: Medium Message: [ Constellation ] Nodes: QPC1, QPC2, CPC1, CPC2, APC2, APC1, Index2, Index1, AdminComponent2, AdminComponent1 3ec1589c-6cc5-9096-7fea-7345dcae1238 owstimer “Unseen” Admin Search Server (Host) Application Server Administration Service Timer Job owstimer.exe Search Admin mssearch “Application Server Administration Service” Timer Job should run once a minute on ALL Search Servers in the Farm… SharePoint Server Search Service Instance owstimer SharePoint Config Cache Crawl Component Crawl Component SearchAdmin.svc Search Admin (Primary) Master Merge? Synchronize Constellation

Step 1… From Central Admin: PowerShell: How to view the status of the SharePoint 2013 search service Step 1… Script uses detailed Component “Health Reports” from Get-SPEnterpriseSearchStatus to provide a more complete picture

Demo Using the “topology” script for troubleshooting (modified to show Application Server Administration Services Timer Job status)

Availability and Performance Microsoft Ignite 2015 4/16/2017 4:11 PM Availability and Performance © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Medium Topology: 40M Items 4CPU/8GB/100GB 4CPU/16GB/300GB 4CPU/16GB/500GB 4CPU/16GB/500GB 4CPU/16GB/500GB 4CPU/16GB/500GB Adm Crawl CPC APC Index0 QPC Index1 Index2 Index3 Be careful here…. 4CPU/8GB/100GB 4CPU/16GB/300GB 4CPU/16GB/500GB 4CPU/16GB/500GB 4CPU/16GB/500GB 4CPU/16GB/500GB Adm Crawl CPC APC Index0 Index1 QPC Index2 Index3

Estimating Load: The QPC may not be “free” 4 CPU Cores Search Server (Host) Query Processing Component Search Server (Host) Guidance (per CPU Core) 1 query per second (10 million items in index) Index Component 1 million items in the index * Guidance (per CPU Core) 4 queries per second Query Rule Query Rule 1 million items in the index Query Rule Query Rule Query Rule 1 million items in the index Query Rule Guidance (per CPU Core) 2 queries per second (5 million items in index) Query Rule 1 million items in the index Rate assumes simple queries Complex or high number of transformations can decrease this rate and add to CPU load 1 million items in the index Query Rule Query 1 Query 2 Query 3 Query 4 Query 5 1 million items in the index 1 million items in the index Guidance (per CPU Core) 5 queries per second (1 million items in index) 1 million items in the index 1 million items in the index 1 million items in the index (0.8 cores * 4 QPS) = 3.2 QPS (1.3 cores * 4 QPS) = 5.2 QPS (2.2 cores * 4 QPS) = 8.8 QPS (4 cores * 5 QPS) = 20 QPS (1.8 cores * 5 QPS) = 9 QPS (2.7 cores * 2 QPS) = 5.4 QPS (3.2 cores * 1 QPS) = 3.2 QPS …approximate net: ~3 QPS ~5 QPS 20 QPS ~9 QPS = 1 CPU Core (e.g. “NumberOfCores” from Win32_Processor Class)

Properly Scale and Deploy Enterprise Search Components: Performance Profile Cheat Sheet Components: Performance Profile Cheat Sheet Component CPU Network Disk RAM Crawl Component MEDIUM1,2 HIGH2,3 MEDIUM3 MEDIUM2 Content processing (CPC) HIGH4 MEDIUM HIGH Analytics processing (APC) MEDIUM4 HIGH5 MEDIUM5 Index Component HIGH6 Query processing (QPC) MEDIUM7 MEDIUM8 Search Admin Component LOW LOW9 Properly Scale and Deploy Enterprise Search Friday, May 8th 12:30PM - 01:45PM Crawler: With PerformanceLevel "maximum” (the default in SP2013), threads are scheduled with "normal” priority (with max 256 threads-per-component) APC: With Map/Reduce distribution, network load grows quadratically with the number of APC nodes, but dilutes the disk storage requirements per APC Crawler: Impact increases with the number of distinct hosts being crawled Index: Ensure the storage meets the IOPS requirements Crawler: Gathered items are temporarily stored to disk and shared for CPC QPC: CPU load increases with queries-per-minute and/or many query rules CPC and APC threads scheduled with "below normal” priority QPC: Network IOPS increases linearly with number of index partitions and QPS Admin: Memory footprint increases with topology size

Availability: Patching with ”Upgrade Domains” Partition #0 Partition #1 Partition #2 Partition #3 …assumes idle crawls 4CPU/8GB/100GB Adm Crawl 4CPU/16GB/300GB CPC APC 8CPU/24GB/500GB Index1 QPC 4CPU/16GB/500GB Index2 Index3 Index4 Adm 1 2 3 4CPU/8GB/100GB Adm Crawl 4CPU/16GB/300GB CPC APC 4CPU/16GB/500GB Index1 8CPU/24GB/500GB Index2 Index3 Index4 QPC Adm 1 2 3 December 2014 CU =Primary Admin =Primary Index Replica =Secondary Replica

Relevancy and Rank: An Opportunity Identify Trends Targeted Queries Refiners, Result Sources/Blocks, Promoted Results Content Enrichment Manipulate Content Before It Gets Indexed Thesaurus and Entities Synonyms and Custom Entity Extraction Query Transformations Re-write the Query, Action Words, Dynamic Rank Deploy Diagnose Problems Try to Fix

Popular, Abandoned, and No Result Queries SPC2012 – IT Pro 4/16/2017 Identify Trends: Popular, Abandoned, and No Result Queries © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Administration: Query Rules Clean Up SPC2012 – IT Pro 4/16/2017 Administration: Query Rules Clean Up Test-BestBetQueryRules.ps1 .\Test-BestBetQueryRules.ps1 -Delete -Mode InvalidRules \ -PathToCSV .\InvalidBestBetQueryRules-SSA.csv Found 25 query rules and 1 Best Bets at SSA level Deleted 1 Best Bets and 0 Query Rules Done. © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

The Life of a Query

Search Query & Site Settings Search Query & Site Settings Follow the Query: Start at WFE, follow to SQ&SS Search Query & Site Settings (SearchService.svc) Application or Farm Boundary Web App Server-side UI (Web Parts) w3wp.exe SSA Proxy Query OM Search Query & Site Settings (SearchService.svc) Query CSOM w3wp.exe w3wp.exe w3wp - wfe Tag: dka1 Area: SharePoint Server Search Category: Query Level: High Message: SearchServiceApplicationProxy::Execute--Proxy Name: Search Service Application Proxy EndPoint: http://querySrv-1:32843/-SSA-/SearchService.svc

Follow the Query: …to a QPC with FewestQueries Processing Component (IMS) SQ&SS (SearchService.svc) Application or Farm Boundary Web App IMS Query Proxy Server-side UI (Web Parts) w3wp.exe SSA Proxy Query OM Query Processing Component (IMS) SQ&SS (SearchService.svc) IMS Query Proxy Query CSOM w3wp.exe w3wp.exe w3wp Tag: ac3iu Area: SharePoint Server Search Category: Query Level: High Message: Ims::EndPoints: Candidate FewestQueries: net.tcp://querySrv-1/E9862A/QueryProcessingComponent1/ImsQueryInternal, status: Succeeded, queries-in-progress: 0

Follow the Query: …to a QPC with FewestQueries Processing Component (IMS) SQ&SS (SearchService.svc) Application or Farm Boundary Web App IMS Query Proxy Server-side UI (Web Parts) w3wp.exe SSA Proxy Query OM Query Processing Component (IMS) SQ&SS (SearchService.svc) IMS Query Proxy Query CSOM w3wp.exe w3wp.exe noderunner Tag: aizc0 Area: Search Category: Query Processing Level: High Message: 'Microsoft.Office.Server.Search.Query.Ims.ImsQueryInternal : New request: Query text 'ninja cat riding fire breathing unicorn', Query template '{searchboxquery}'; HiddenConstraints: ; SiteSubscriptionId: 00000000-0000-0000-0000-000000000000

Concepts: Inside the Productivity Search Flow Query Processing Component ProductivitySearchFlowExecutor OAuth token created from service context (received from the SSA Proxy) QueryRuleConditionMatching People Search Flow Remote Sharepoint Flow BestBet Provider Flow ExchangeSearchProvider Flow SharepointSearchProvider Flow PersonalFavoritesProvider Flow Thread [n] QueryTransformer Remote Sharepoint Flow QueryRouter ResultsMixer SharepointSearchProvider Flow LayoutsSelector Thread [A]

Follow the Query: Response from each Cell Partition #0 Partition #1 Query Processing Component (IMS) SQ&SS (SearchService.svc) Application or Farm Boundary Web App IMS Query Proxy Cell 0 Cell 0 Server-side UI (Web Parts) w3wp.exe SSA Proxy Cell 1 Cell 1 Query OM Query Processing Component (IMS) SQ&SS (SearchService.svc) IMS Query Proxy Query CSOM Cell 2 Cell 2 w3wp.exe w3wp.exe noderunner Tag: ajkph Area: Search Category: Search Component Level: High Message: Microsoft.Ceres.SearchCore.Query.MarsLookupComponent.LookupService. QueryClient.QueryExecutor: ExecuteQuery timings for correlation: <correlationID>, task dispatch 0 ms, blocked waiting 180 ms, total hits: 14998, with dupes: 16361, 2 tasks: ( cell: I.0.0 at IndexComponent2, total task time: 179 ms, …etc… cell: I.1.1 at IndexComponent4, total task time: 162 ms, …etc… )

Follow the Query: Overall timings at the WFE Partition #0 Partition #1 Query Processing Component (IMS) SQ&SS (SearchService.svc) Application or Farm Boundary Web App IMS Query Proxy Cell 0 Cell 0 Server-side UI (Web Parts) w3wp.exe SSA Proxy Cell 1 Cell 1 Query OM Query Processing Component (IMS) SQ&SS (SearchService.svc) IMS Query Proxy Query CSOM Cell 2 Cell 2 w3wp.exe w3wp.exe w3wp - wfe Tag: dk91 Area: SharePoint Server Search Category: Query Level: High Message: SearchServiceApplicationProxy::Execute—Id: Elapsed Time: 458 IMSProxyTime: 399 QP Time: 372 Client Type AllResultsQuery

Monitoring: ”Overall" timings at the WFE “Backend” (QP Time) “Object Model”

Monitoring: Breaking Out of the Backend Time These activities should remind you of the “Productivity Search Flow” slide covered earlier

Monitoring: Breaking Out of the Backend Time cell: I.0.0 at IndexComponent2, total task time: 179 ms cell: I.1.1 at IndexComponent4, total task time: 162 ms …etc…

Query and Index Availability: Partial Results Processing Component (IMS) $ssa=Get-SPEnterpriseSearchServiceApplication $ssa.AllowPartialResults = $true $ssa.Update() Cell 0 Partition #0 Cell 1 Cell 0 Partition #1 Cell 1 Cell 0 Partition #1 Cell 1 ! November 2014 CU

Demo Partial results and High Availability in action Microsoft Ignite 2015 4/16/2017 4:11 PM Demo Partial results and High Availability in action © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Clearly too many items pooling up in “CTS” Crawl Health Report: Crawl Load How many transactions are running at this time… Clearly too many items pooling up in “CTS”

…and pooling because “CTS” is the longest stage Crawl Health Report: Crawl Latency How fast items move through each stage… …and pooling because “CTS” is the longest stage

…and time “Indexing” is clearly the main issue Crawl Health Report: Content Processing Activity How fast items move through each sub-stage …and time “Indexing” is clearly the main issue

Concepts: Submitting to the Index Partition 0 Content Processing Components Index Component Index Component Cell I.0.0 Cell I.1.0 103 102 101 103 102 101 Active Passive Replica 0 “Primary” Replica 1 noderunner.exe noderunner.exe Search Admin Component noderunner Tag: ajj8t Category: Search Component Level: Verbose Message: GenerationDispatcher[SP953b8ca4ed25]: CommitGeneration GID[103] in 377 ms

“Indexing” Activity: The Impact on Resources Antivirus Exclusions? Isolated Data Paths? CPUs Parked? IOPS Requirements? Network Saturated?

Demo Build SP Search 2013 Index health reports for monitoring and troubleshooting (A Peek Inside the Index System Using PowerShell)

Re-Living the Memories… SPC2012 – IT Pro 4/16/2017 Re-Living the Memories… Automate monitoring of SSA Analyze performance reports Review usage reports for trends Result Sources / Query Rules / Refiners Managing performance Backup / DR Frequent Infrequent Security updates + Cumulative updates Sizing / Scaling (and Migrations) Relevancy tuning © 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Any More Questions? SearchEngineers@microsoft.com Microsoft Ignite 2015 4/16/2017 4:11 PM Any More Questions? SearchEngineers@microsoft.com http://aka.ms/SearchEngineers © 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Please evaluate this session 4/16/2017 4:11 PM Please evaluate this session Your feedback is important to us! Visit Myignite at http://myignite.microsoft.com or download and use the Ignite Mobile App with the QR code above. © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4/16/2017 4:11 PM © 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.