Search and Find With SharePoint 2010 Thierry Gasser Technical Specialist Collaboration Platform Thierryg@microsoft.com

Agenda SharePoint 6 pillars Overview Search versions and Product comparison Demos Search Architecture and customization info

SharePoint 2010 Pillars Sites Communities Content Search Insights Business Connectivity Services InfoPath Form Services External Lists Workflow SharePoint Designer Visual Studio API Enhancements REST/ATOM/RSS Ribbon UI SharePoint Workspace SharePoint Mobile Office Client and Office Web App Integration Standards Support Sites Communities Content Search Insights Composites Tagging, Tag Cloud, Ratings Social Bookmarking Blogs and Wikis My Sites Activity Feeds Profiles and Expertise Org Browser PerformancePoint Services Excel Services Chart Web Part Visio Services Web Analytics SQL Server Integration PowerPivot Enterprise Content Types Metadata and Navigation Document Sets Multi-stage Disposition Audio and Video Content Types Remote Blob Storage List Enhancements Social Relevance Phonetic Search Navigation FAST Integration Enhanced Pipeline

[Screenshot] [Screenshot] Enterprise Search is Transforming Business Connecting people to information, driving better outcomes On a company website On a corporate network [Screenshot] [Screenshot] Search part !!! search helps your customers get what they want search helps your employees get their jobs done increasing revenue cutting costs

The Search 2010 Wave SharePoint SharePoint FAST Search SharePoint 11/13/2018 2:50 PM Internet Business Productivity FAST Search for Internal Apps (ESP) FAST Search for Internet Sites (ESP) Stand Alone Search Server 2010 Search Express2010 SharePoint for Internet Sites, Enterprise (can be FS4SP or SP2010 search) SharePoint + Enterprise CAL + FS4SP FIS Enterprise (FIS-E) includes the rights to both the SharePoint FIS bits and the FAST Search for SharePoint bits (for use outside the firewall only). Customers who buy a single server of FIS-E can deploy it as SharePoint FIS server or FAST Search for SharePoint server, but not both concurrently. FIS Enterprise includes all the features in the SharePoint Enterprise CAL. FIS Standard (FIS-S) does not include the features in the SharePoint Enterprise CAL, does not include FAST Search capabilities, and is limited to use on a single domain SharePoint Integrated SharePoint for Internet Sites, Standard SharePoint Search + Standard CAL © 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Knowledge Amplification Enterprise Deployment SharePoint Search Improved relevance based on usage and history Rich results-navigators for shorter search time Support for 400+ content types and 85 languages (FAST) Better Answers, Faster Phonetic and multi-lingual for global relevance Social connections driven by interests and expertise Secure access to content, whether internal or external Knowledge Amplification Improved scale to meet any demand volume Deployment-specific tuning for best results Search-driven applications to enrich platform Enterprise Deployment

FAST Search for SharePoint and ESP Eliminate compromise IT Professional Do more with search Developer   Expertise Finding People & expertise search Presence Ranking via Social distance Configuring different groups search behavior Search Profiles User Context Pipeline Customization Editing XML Config files Writing python code Spy Stages GUI configuration PowerShell Managed Code Add-ons Separate + specialized Featured Content Unity GeoSearch Integrated + extensible Visual Best Bets SharePoint Federator Not OOB, done via sortformula() Text analytics Classification Relationship Extraction Today: partners or services + more tomorrow: FS for SharePoint is the platform for future innovation

Capabilities from F4SP not in ESP (1/2) In FS4SP not in ESP FAST ESP FS4SP Taxonomy management Only available via partners (such as Teragram/SAS) taxonomy and taxonomy management tools + Partners (Teragram/SAS, Factiva, Concept Searching, SchemaLogic, …) OOB GUI ESP does not ship with a production GUI typically GUIs are custom designed or developed with a toolkit OOB GUI built from production web parts. no-code customization via web parts. People Search Only Index information about people not integrated people and expertise search, relevancy tuned to people, phonetic search OOB, nicknames, etc. people and expertise search (source from SP) including phonetic search capabilities. Closely integrated with Active Directory and SharePoint social computing capabilities. Easy Administration ESP has specialized tool for search administration with specific in-class training required Integrated admin UI Scriptable via PowerShell Relevance improves with use Not available with ESP relevance based on clickthrough, provide related queries based on popularity, etc.

Capabilities from F4SP not in ESP (2/2) In FS4SP not in ESP FAST ESP FS4SP Business Connectivity Services and BDC ESP has a connector toolkit Special programming skills to create new connectors Use BCS as a broad framework Custom search analytics Possible via custom projects which export logs, combine with business and clickstream data, and do BI Built-in query and click log access extensible and customizable Claims Authorization security Not available with ESP security mechanism which can be used with federated ID, forms authorization, ADFS, SAML, etc.). 64 bit native app FS4SP and all of SharePoint is a native 64 Development Tools with built-in tooling SPD and VS2010 - tooling specifically for SharePoint applications including FS4SP Query Suggestions based on social behavior search logs based (popular queries on closely related subjects out-of-the-box.)

Demo Search Live MS search http://msw/searchcenter/Pages/default.aspx -> Fast for SharePoint 2010 -> People search, skills.

SharePoint Search Architecture MS Confidential : SharePoint 2010 Developer Workshop (Beta1) SharePoint Search Architecture . . . SharePoint Sites SSA Database Metadata Database Crawl Database Index Repositories Remote File Shares Get metadata info from SQL Send Query to full text index Send Query WFEs Query Processor Query servers Index Servers Custom Repositories Index servers reads config information from the crawl DB Index server crawls the content and it goes through the indexing pipeline Send index + metadata to Query sever + metadata database WFE sends query to QP QP sends to QS + MD QP gets results back and sends to WFE Get Results Get Results from full text index Propagate index + metadata to query server & DB Web Sites . . .

FAST Search for SharePoint Integration Architecture MS Confidential : SharePoint 2010 Developer Workshop (Beta1) FAST Search for SharePoint Integration Architecture . SSA Database Metadata Database Crawl Database . . SharePoint Sites Index Repositories using SharePoint gatherer Query request to FAST location WFEs Query Processor Query servers Index Servers Remote File Shares Show results in SharePoint Send Query to FAST backend Send content to FAST for indexing Final search results Custom Repositories This slide shows the integration architecture of FS2010 as part of sharepoint There are two ways in which data can be indexed in FS2010 1. By using the index servers of FS2010 2. By using crawler of SS2010, which then uses the shared sharepoint conector to send data to fast using content API. The data does not go through any processing on SS2010 side.. All processsing is done at the FS2010 side Both sharepoint and fast shares the APIs to access the query capabilities Both sharepoint and fast uses the same interface to communicate between the WFE and QP. Make a point that new verticals can go against sharepoint and fast and OOB we use people to go against sharepoint Query Processing Query & Results Server Modified Query Search Engine Processing servers Content Interface Search Results Results Processing Web Sites` Propagate index to search engine Index Repositories using FAST system Index . . .

Item Processing Pipeline FS4SP – Data Flow … Conversion Format Language Detection Property Extraction Stemming Mapper Item Processing Pipeline Content OpenSearch Query Processing Query Matching Item Processing Crawler Federation Indexing This shows the overall architecture and data flow Highlight most important component People Search Web Link Analysis Search Center Crawl DB Index Column User Profiles FAST Server(s) SharePoint Server(s) Other Server(s)

FS4SP – Minimum Deployment Single-tier setup Two-tier setup Single tier: 1 server with all SP roles 1 server with all FAST roles Two tier: 1 DB server Virtualize for redundancy Note: None of these environments are redundant. For a production environment, you should at a minimum consider running the SharePoint and FAST servers in a virtual environment and with a mirrored or clustered SQL server.

FS4SP – Large Deployment Scope: Query rate: < 10 QPS Content: < 100 M items Standard configuration SQL server needs to be scaled for increased IO # of Web Analyzers depends on amount of clicks, 2 or 3 needed for 100M index

FS4SP – Server Calculation Matrix Max item count (in Millions) Adm + WA Indexers (1 row) SharePoint Crawlers Crawl DB Server Redundancy Total 1 2 10 6 40 3 11 100 19 150 5 31 200 14 42 500 34 16 96 Scope: Query rate: < 10 QPS Intranet search with 70% items and 30% documents, source doc between 10-100Kb. Standard deployment and configuration Admin: 1 per installation SP crawler + DB: ~25M docs per node Web Analyzer: ~30M per node Indexer: ~15M docs per node Question: So, i got a 64 bit OS, a huge disk and FAST Search...cant i do 50M or even 100M on one node????“ Answer: Maybe ;-) Currently we recommend 15M per node based on the dogfood testing and the fact that there is a physical limit in the configuration file. But, we know that R&D is now running tests, and it will be very interesting to hear what results they come up with. They are doing tests with both 50M and 100M. Disclaimer: This table is based on early testing and results from an internal dogfood project. The numbers might not be representative for the customer environment and data. Please use caution when using these numbers for sizing.

Customization Search center based on Web parts XSLT can be used to change the Result Queries can be developed with XML and/or .NET >Eg: www.swiss.ch

© 2009 Microsoft Corporation. All rights reserved © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.