Download presentation
Presentation is loading. Please wait.
1
FILESTREAM Storage Attribute
Klaus Aschenbrenner
2
About me Software Architect @ EDS, an HP company Speaker, Author
„Pro SQL Server 2008 Service Broker“
3
Agenda Challenges for using unstructured data
The Filestream Feature - An Overview Filestream and T-SQL OpenSqlFilestream System.Data.SqlTypes.SqlFileStream Filestream access patterns Filestream transactional semantics
4
Motivation Storing large binary objects in databases is suboptimal
Large objects take buffers in database memory Updating large objects cause database fragmentation In file system however, "update" is delete and insert "Before image" in an update is not deleted immediately Storing all related data in a database adds Transactional consistency Integrated, point-in-time backup and restore Single storage and query vehicle
5
Challenges using Unstructured Data
Cost of storing your unstructured data, including not only hardware costs, but also the management overhead costs Managing retention and archival and policies Integrating unstructured data files with associated relation data Indexing and searching unstructured data
6
Business Scenarios 1/2 An insurance company needs to store policy and retrieve them for claims processing An online retailer needs to store video footage of its products for display in an e-commerce site A radio station needs to make searchable library of podcasts available for download from its website
7
Business Scenarios 2/2 A telephony system needs to store voic messages as audio streams so that they can be retrieved remotely A library needs to convert and archive large volumes of existing paper and analog content for indexing and use in digital research tool An online retailer needs to store video footage of its products for display in an e-commerce site
8
FILESTREAM Storage FILESTREAM is an alternative storage format
For VARBINARY(MAX) columns Specified by FILESTREAM keyword on column in CREATE/ALTER TABLE Metadata still stored in database Data stored in file system One file per non-null row In database optimal up to 256 KB In file system optimal over 1 MB
9
FILESTREAM Instance Setup
OS portion of FILESTREAM setup SQL Server Configuration Manager Three setup alternatives Enable FILESTREAM, local access only, remote access SQL Server configuration sp_configure filestream_access_level Disabled, SQL only, SQL and Win32 streaming
10
FILESTREAM Setup Database
Add filegroup for filestream storage ALTER DATABASE ADD FILEGROUP The filegroup is mapped to a directory Directory must not exist Parent directory must exist Must be NTFS file system NAS volumes and remote shares not supported Caution: Files deletable from file system if you have appropriate permissions
11
FILESTREAM Table Setup
Columns must be VARBINARY(MAX) (N)VARCHAR(MAX), XML type not supported Table must contain UNIQUEIDENTIFIER Marked with ROWGUID property FILESTREAM columns can be larger than 2 GB Limited by file size
12
FILESTREAM Programming
FILESTREAM is a normal SQL column Can be accessed using Win32 I/O APIs and special file handle Note: file must not be accessed with an ordinary Win32 file handle or database corruption can result DACL'd to Administrators and SQL Server Service Account only
13
FILESTREAM Programming - TSQL
Using T-SQL directly means SQL Server's buffer pool is used WRITE method is unavailable Insert and updating NULL to non-NULL must use SQL Insert or update to empty content creates the file Use streaming APIs to populate Delete deletes underlying file
14
SQL Functions Functions that work against the VARBINARY(MAX) column work with FILESTREAM SUBSTRING - reads part of column CHARINDEX, LEN, REPLACE CAST, CONVERT Full text indexed FILESTREAM columns can use CONTAINS, FREETEXT Encryption is NOT supported on FILESTREAM data
15
FILESTREAM Programming - TSQL
Demo FILESTREAM Programming - TSQL
16
FILESTREAM - Streaming
Using FILESTREAM with streaming API is like using multiple SQL statements BEGIN TRANSACTION Obtain UNC name and transaction context Read, Write,Update file Open is "beginning of statement" Close is "end of statement" COMMIT or ROLLBACK transaction
17
Streaming Requirements
To use streaming IO, you must have Windows login (not SQL login) Access to the table with FILESTREAM column SMB access to the server if access is remote
18
OpenSqlFilestream OpenSqlFilestream is a method defined in SQL Native Client driver Useable in ODBC, OLE DB directly Useable in .NET through DLLImport
19
OpenSqlFilestream OpenSqlFilestream returns a file handle Requires
File name Required Access (Read, Write, ReadWrite) AccessOptions FILESTREAM transaction context
20
Getting Parameters Pathname exposed through PathName method on FILESTREAM instance PathName is case sensitive Returns a symbolic UNC name Returns null if FILESTREAM column is NULL GET_FILESTREAM_TRANSACTION_CONTENT Gets the transaction context Transaction must already be started
21
Opening Options ASYNC NO_BUFFERING WRITE_THROUGH SEQUENTIAL_SCAN
WRITE_THROUGH is default for native APIs SEQUENTIAL_SCAN Fastest for reading entire file RANDOM_ACCESS
22
Win32 APIs Supported OpenSqlFilestream – retrieves a handle
CloseHandle – frees the handle ReadFile, WriteFile, TransmitFile, SetFilePointer, SetEndOfFile, FlushFileBuffers supported Any other APIs return permission denied Creating memory-mapped views unsupported
23
SqlFileStream Class SqlFileStream is a .NET class that encapsulates OpenSqlFilestream In System.Data.SqlTypes Inherits from System.IO.Stream Supports asynchronous reads and writes using .NET AsyncCallback pattern Write-through is not default, uses client buffering
24
Demo SqlFilestream
25
Access Pattern - SELECT
SELECT requires BEGIN TRANSACTION SELECT to retrieve other columns, transaction context, file name Open SqlFileStream for Read COMMIT TRANSACTION
26
Access Pattern - INSERT
Insert consists of Begin transaction SQL INSERT w/empty (not null) FILESTREAM column OUTPUT clause to obtain transaction context, filename SqlFilestream, access Write Fill filestream content Using 60k multiples for buffer size optimal Commit Transaction
27
Access Pattern - UPDATE
UPDATE consists of BEGIN TRANSACTION SQL UPDATE statement, must update NULL FILESTREAM column to empty OUTPUT to obtain transaction content, filename UPDATE (or fill) FILESTREAM column COMMIT TRANSACTION
28
Partial Updates Handle returned by SqlFilestream can get existing data
ReadFile on this handle returns end of file Use fsctl_sql_filestream_fetch_old_content to cause copy of existing file to be returned Using this with overlapped IO can perform a partial update "in place"
29
Transactional Semantics
Transactions and Isolation Levels Streaming – read committed only SQL – read committed, repeatable read, serializable, snapshot SQL – read uncommitted supported but will not see in-flight changes Do not attempt to commit/rollback transaction when file handle is open
30
SQL and Streaming Interactions
When FILESTREAM is open for read SQL DDL and DML statements are allowed SQL SELECT is allowed SQL commit and save points not allowed When FILESTREAM is open for write SQL DDL and DML statements not allowed SQL SELECT is allowed (gets old data)
31
FILESTREAM & Triggers Triggers fire when
INSERT/UPDATE statement completes File handle is closed
32
FILESTREAM Performance
SQL access uses SQL buffers File access use less SQL buffers But using file buffers could decrease storage allocated to SQL Server Except for small data, use file access For incremental updates, all data must be read in buffers
33
Load Balancing Partitioning can be used to create a table schema that allows crude of load balancing of file stream between multiple volumes Desirable to avoid I/O hardware limitations problems Enable easy hardware scalability without N simultaneous users accessing videos data files might saturate I/O subsystem
34
Demo Load Balancing
35
LINQ to SQL & Entity Framework
These APIs support FILESTREAM column as VARBINARY(MAX) No special support for SqlFileStream and related APIs
36
Remote BLOB Storage RBS is part of SQL Server 2008 Feature Pack
Provides blob management with transactional consistency Blob files can reside on remote server Content Addressable Store (CAS) Service Storage Solution (MSN XStore) Consists of Stored procedures for transactional blob management SQL Agent jobs for maintenance and cleanup API for integrating with SQL Server and TransactionScope
37
Summary FILESTREAM is a storage choice for VARBINARY(MAX)
Set up at OS, instance, and database level Defined as a column property You can access FILESTREAM in “Normal" T-SQL Streaming APIs through file system OpenSqlFilestream or SqlFileStream data type
38
Complete the Evaluation Form & Win!
You could win a Dell Mini Netbook – every day – just for handing in your completed form! Each session form is another chance to win! Pick up your Evaluation Form: Within each presentation room At the PASS Booth near registration area Drop off your completed Form: Near the exit of each presentation room Sponsored by Dell
39
for attending this session and the 2009 PASS Summit in Seattle
Thank you for attending this session and the PASS Summit in Seattle
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.