Russ Houberg Senior Technical Architect, MCM KnowledgeLake, Inc.

RBS Background Implementation Guidance Framework Components Process Flow Required Resources Install & Configuration Migrate() Demo Maintenance Impact on Scalable Architecture

Remote BLOB Storage (RBS) By default SharePoint stores Binary Large Objects (BLOBs) in the content database When enabled… Intercepts binary content (documents) and sends them to a BLOB store SharePoint assemblies function as a client that implements a base provider in the Microsoft.Data.SqlRemote assembly 3 rd Party providers use the Microsoft.Data.SqlRemote assembly to construct RBS providers that are “loosely” invoked by SharePoint in order to store binary data in a BLOB store Microsoft provides the “local” FILESTREAM provider to allow for usage of the SQL Server local NTFS file system as a BLOB store.

SharePoint 2003 SharePoint 2007 SharePoint 2010

Enabling RBS allows BLOB data to be stored on less expensive storage Weigh the value of the storage against the administrative and performance overhead of RBS. Enable RBS when: Content Database is expected to be larger than 500 (GB) BLOB data files are larger than 256KB on average BLOB data files are at least 80KB and the DB server I/O is a bottleneck Many small BLOBs can decrease performance Content Database size is likely to be more of a decision factor than minor increases in retrieval latency RBS is most beneficial with very large files such as many page PDF or TIF files or digital media. RBS is typically implemented in which large files are infrequently accessed, such as an archive.

Enabling RBS allows a system that is specifically designed to work with BLOB data to manage storage Write Once Read Many (WORM) device can raise an exception in the RBS Provider that prevents the BLOB related SharePoint metadata row from being deleted High end SANs are able to replicate BLOB content to another (mirrored) device over a WAN using sophisticated bit mapping (delta) technology. May be possible to combine with log shipping or database mirroring to provide DR solution. Maintainer is key to identify orphans in the database or the BLOB store after failover

Consider the following when selecting an RBS Provider Backup and restore capability Tested disaster recovery Deployment and data migration Tested performance impact Long-term administrative costs V.Next SharePoint Upgrade Support

SQL RBS Add-in “Framework” (rbs_x64.msi) Provides “BlobStore” Abstract Class RBS Provider Library Managed library that implements the BlobStore abstract class in order to store and retrieve content in the BLOB store. RBS Client Library Invokes the BlobStore class in the Enabled/Active Provider in order to send to and request data from the BLOB store.

SQL Database RBS Objects (tables, stored procs, etc) Stores BlobId and BlobReference values and provides resources for managing BLOB pointers Application (SharePoint 2010) Invokes an RBS Client in order to send data to and request data from the BLOB store

2. Enforce Business Logic 1. Save Request 3. Save Blob 4. Write Blob 5. Return BLOB ID 6. Save Metadata & BLOB ID 7. Back to User

SharePoint Foundation 2010 (minimum) SQL Server 2008 R2 Any Version SQL Express R2 (Supports 10GB Database Size) NOTE: Microsoft SharePoint Foundation installs with SQL Server 2008 by default (not SQL Server 2008 R2) FILESTREAM RBS Provider / Framework

The FILESTREAM provider is supported by SharePoint Server 2010 when it is used with SQL Server 2008 R2 or SQL Server 2008 R2 Express. Only “local commodity storage” (hard drive) is supported. Direct Attached Storage (DAS), Network Attached Storage (NAS), and Storage Area Network (SAN) are all considered to be “remote commodity storage” and are not supported by SharePoint Any other 3 rd Party RBS Provider is considered to be a “remote server” provider and SharePoint 2010 licensing requires that SQL Server 2008 R2 Enterprise Edition be implemented.

NAS as BLOB Store storage target 20ms maximum latency to first byte on retrieval The FILESTREAM provider… Is limited to SQL Server local storage only Does not support document encryption Does not support document compression Does not support SQL Server 2008 Pool sizes can only be configured by editing Machine.Config

Create master key in content database Used by RBS security model. Master Key should be BACKED UP and stored in secure, off- site location! Install RBS_x64.msi on SQL Server and all WFEs Special command line parameters for FILESTREAM RBS provider will execute RBS stored proc on content database

Prepare Content Database (other providers) C:\Program Files\Microsoft SQL Remote Blob Storage 10.5\Server\RemoteBlobEnable.sql Creates rbs_sp_enable_rbs stored proc Execute rbs_sp_enable_rbs stored proc (builds out all RBS resources in content database) If not using FILESTREAM provider Execute RBS Provider installer

Use PowerShell to activate provider $site = get-spsite [ $rbss = $site.ContentDatabase.RemoteBlobStorageSettings $rbss.Installed() True $rbss.Enable() $rbss.SetActiveProviderName($rbss.GetProviderNames()[0]) $rbss Enabled ActiveProviderName MinimumBlobStorageSize UpgradePersistedProperties True FileStoreProvider_1 0 { } $site = get-spsite [ $rbss = $site.ContentDatabase.RemoteBlobStorageSettings $rbss.Installed() True $rbss.Enable() $rbss.SetActiveProviderName($rbss.GetProviderNames()[0]) $rbss Enabled ActiveProviderName MinimumBlobStorageSize UpgradePersistedProperties True FileStoreProvider_1 0 { }

Use PowerShell to EXTERNALIZE all content $site = get-spsite [ $rbss = $site.ContentDatabase.RemoteBlobStorageSettings $rbss.Migrate() [command completes when finished] [can be stopped and restarted any time] $site = get-spsite [ $rbss = $site.ContentDatabase.RemoteBlobStorageSettings $rbss.Migrate() [command completes when finished] [can be stopped and restarted any time]

Use PowerShell to INTERNALIZE all content $site = get-spsite [ $rbss = $site.ContentDatabase.RemoteBlobStorageSettings $rbss.SetActiveProviderName(“”) $rbss.Migrate() [command completes when finished] [can be stopped and restarted any time] $rbss.Disable() $rbss Enabled ActiveProviderName MinimumBlobStorageSize UpgradePersistedProperties False 0 { } $site = get-spsite [ $rbss = $site.ContentDatabase.RemoteBlobStorageSettings $rbss.SetActiveProviderName(“”) $rbss.Migrate() [command completes when finished] [can be stopped and restarted any time] $rbss.Disable() $rbss Enabled ActiveProviderName MinimumBlobStorageSize UpgradePersistedProperties False 0 { }

RBS Maintainer (Garbage Collection Tool) Provider Agnostic Reference Scan Compare BLOBIDs in application tables with BLOBIDs in RBS tables Finds BLOBs that are no longer referenced Delete Propagation Mark BLOBs for deletion Delete the BLOBs from the BLOB store Orphan Cleanup Enumerate BLOBs in the BLOB store Remove any BLOBs that are not known to RBS

RBS Maintainer (continued) Can be interrupted if necessary Defaults to 30 day delete window Can override for immediate delete with stored procedure call Sample maintainer command: Microsoft.Data.SqlRemoteBlobs.Maintainer.exe -ConnectionStringName RBSMaintainerConnection -Operation GarbageCollection ConsistencyCheck ConsistencyCheckForStores -GarbageCollectionPhases rdo -ConsistencyCheckMode r -TimeLimit 120

RBS Installation and Configuration Demo

Backup and Restore Order Backup Restore SQL BLOB Store Both Backups CompleteBackup Start BLOB Store SQL Both Restores are CompleteRestore Start

Library Size vs Content Database Size 10 Million to 30 Million Documents Item metadata ~5 – 7KB For archive scenarios, break up content by year or, if necessary, month Leverage Content Organizer Rules to route content to the right site collection 1.5 Million10 Million20 Million30 Million DB 5KB7GB48GB95GB143GB DB 6KB9GB57GB114GB172GB DB 7KB10GB76GB153GB229GB

Microsoft has a solid long term solution to BLOBs in the database RBS Providers guarantee persistence We can migrate existing content that is either in-line or externalized with EBS. Reduces Content DB size and facilitates large repositories.