Filter Manager Overview © 2004 Microsoft Corporation. All rights reserved.

Problems with today’s Filter Model Poor control over filter load order Non-deterministic Increases test matrix Causes interop problems No unload support Requires reboot Must process all operations ~5000 lines of code to start © 2004 Microsoft Corporation. All rights reserved.

Problems with today’s Filter Model (cont) Kernel stack overflow Call-through model Filters generate recursive IO Inefficiencies due to redundant work in filters Each filter does its own name lookup, normalization, and name caching Implement their own context support Don’t always get it correct © 2004 Microsoft Corporation. All rights reserved.

Problems with today’s Filter Model (cont) Complex interfaces Difficult to add new file system operations (IRP/FastIO) Existing filters must be modified New IRP and FastIO operations were added to W2K Broke all existing filters TxF (new Longhorn feature) Multiple OS version support Filter must be developed to “least common denominator” API set © 2004 Microsoft Corporation. All rights reserved.

The Filter Manager addresses all of these Issues © 2004 Microsoft Corporation. All rights reserved.

What is the Filter Manager Legacy file system filter Reduces the complexity of IO system through new interfaces and library routines Has kernel and user-mode interfaces © 2004 Microsoft Corporation. All rights reserved.

Filter Manager User Mode IO Operations I/O Manager FAT NTFS RDR Filter Manager Frame 1 (1000-9999) IRP + FastIO + FsFilter Interfaces Legacy Filter Driver Filter Manager Frame 0 (0-1000) Minifilter © 2004 Microsoft Corporation. All rights reserved.

Definition of Terms Legacy Filter Minifilter A file system filter developed to the current Filter model Minifilter A file system filter developed to the Filter Manager model © 2004 Microsoft Corporation. All rights reserved.

Definition of Terms Instance Altitude A filters attachment to a volume at a particular altitude Multiple filter instances per volume is supported See MiniSpy Altitude A unique identifier which defines the relation of this filter to other filters An “infinite precision” string with a decimal point Example: 4501.345 Can always insert a new filter between two existing filters All minifilters must have a unique altitude Altitudes are managed by Microsoft Developing a web site for requesting altitudes. Altitude determines relative stack position © 2004 Microsoft Corporation. All rights reserved.

Instances and Altitudes Instance: A filters attachment to a volume at a particular altitude Support multiple instances of a minifilter on a volume Altitude determines relative stack position Volume c: AntiVirus Filter (Altitude: “300”) Encryption Filter (Altitude: “100”) “LanmanRedirector” Conceptual IO Flow MiniSpy Filter (Altitude: “200”) (Altitude: “400”) © 2004 Microsoft Corporation. All rights reserved.

Definition of Terms Frame Multiple filter manager attachments to a file system stack to support proper interop with legacy filters Each frame defines an altitude range © 2004 Microsoft Corporation. All rights reserved.

Filter Manager with Multiple Frames User Mode IO Operations I/O Manager FAT NTFS RDR Filter Manager Frame 1 (1000-9999) IRP + FastIO + FsFilter Interfaces Legacy Filter Driver Filter Manager Frame 0 (0-1000) Minifilter © 2004 Microsoft Corporation. All rights reserved.

Definition of Terms CallbackData Iopb Filter manager's equivalent of an IRP All operations come to minifilters via this structure Iopb Io Parameter Block Points to current stack location © 2004 Microsoft Corporation. All rights reserved.

Problems with today’s Filter Model (Review) Poor control over filter load order No unload support Must process all operations Kernel stack overflow Inefficiencies due to redundant work in filters Complex interfaces Difficult to add new operations Cross version support © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed Poor control over filter load order Minifilter can be loaded at any time Minifilters can be inserted into the middle of the attachment chain Altitude determines relative stack position Deterministic load order © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) No unload support Minifilters can be unloaded at any time Filter Manager synchronizes the safe removal of all minifilter attachments through notifications Filter Manager handles operations which complete after the minifilter unloads © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) Must process all operations Minifilter registers only for operations in which it is interested Can uniquely register for pre- or post-Operation callbacks Can ignore certain classes of Operations Paging IO Cached IO © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) Kernel stack overflows Filter Manager uses a “callback” model instead of a “call-through” model Supports non re-entrant filter initiated i/o IO will only be seen by filters below your filter Filter Manager is optimized to reduce the amount of stack it consumes © 2004 Microsoft Corporation. All rights reserved.

Filter Manager’s Callback model User Mode IO Operations I/O Manager FAT NTFS RDR Filter Manager Frame 1 IRP + FastIO + FsFilter Interfaces Legacy Filter Driver Filter Manager Frame 0 Minifilter © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) Inefficiencies due to redundant work in filters Filter Manager provided infrastructure Name generation is time consuming Filter Manager caches file names for use by multiple filters Filter Manager is optimized to support multi-processor systems © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) Complex interfaces Filter Manager simplifies complexity by providing support routines for common functionality Naming Support Context Support User/Kernel mode communication Masking differences between file systems Filter Manager eliminates complexity by automatically handling certain functionality Enumerating and attaching to file system stacks IoMarkIrpPending() © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) Difficult to add new operations Only filter manager needs to be updated to support new operations Minifilters only register for those operations they are interested in © 2004 Microsoft Corporation. All rights reserved.

How these problems are addressed (cont) Multiple OS version support Filter Manager supports running of minifilters on down level OS’s Can register for operations that don’t exist Can programmatically determine which APIs exist FltGetRoutineAddress() Structures are designed to be extensible Registration structure has a version number Structures contain a size field © 2004 Microsoft Corporation. All rights reserved.

Additional Filter Manager Features © 2004 Microsoft Corporation. All rights reserved.

Context Management Allows minifilters to attach private information to a system object Volume Context Instance Context File Context (not implemented yet) Associated with all opens across all data streams for a given file Stream Context Associated with all opens for a given data stream of a file Stream Handle Context Associated with a single open of a single data stream © 2004 Microsoft Corporation. All rights reserved.

Name Management Copy-free retrieval of file names Retrieves names in one of three formats: Normalized, opened, short When possible, name is cached Filter manager manages name lifetime Handles renames, hardlinks, etc. © 2004 Microsoft Corporation. All rights reserved.

Filter Initiated IO IO is targeted to lower filters Io is not recursive Properly integrated with minifilter unload © 2004 Microsoft Corporation. All rights reserved.

User/Kernel Communication Minifilter creates a named port Minifilter controls who can access the port via ACLs Bi-directional communication channel © 2004 Microsoft Corporation. All rights reserved.

fltmc.exe Command line utility for common filter management operations Load and unload minifilters Attach/detach minifilters to/from volumes Enumerate minifilters, instances, volumes “fltmc help” Displays help information © 2004 Microsoft Corporation. All rights reserved.

Debugging Help fltkd debugger extension Part of debugger package .load fltkd !fltkd.help List all available commands For more specific help on a single command, issue that command with no parameters Important commands !cbd Filter Manager equivalent to !irp !frames List all Frames and their filters © 2004 Microsoft Corporation. All rights reserved.

Debugging Help fltkd debugger extension (cont) !filters List all Frames and their filters !volumes List all volumes and their instances !volume, !filter, !instance Give detailed information on the specific object Run with debug version of fltmgr.sys Lots of ASSERT to catch common errors © 2004 Microsoft Corporation. All rights reserved.

Filter Verifier Enable via Driver Verifier (verifier.exe) Select minifilter driver name Enable “I/O Verification“ option Recommend verifier value: 0x7b Verification starts when a filter registers with the Filter Manager Validates all Filter Manager APIs Validates parameters and calling context Verifies return values from minifilter’s pre/post callback routines Ensures minifilter changed the parameters in the callback data in a coherent/consistent manner Lots more to come Always develop minifilters with driver verifier and filter verifier enabled! © 2004 Microsoft Corporation. All rights reserved.

Thoughts about the Filter Manager It is much simpler to start developing a filter. Eliminates ~5000 lines of infrastructure code It is still just as hard to complete a production quality filter due to the complexity of the IO system © 2004 Microsoft Corporation. All rights reserved.

Filter Manager Changes/Enhancements © 2004 Microsoft Corporation. All rights reserved.

Filter Manager Changes (May 2004) Modified APIs User/Kernel Communication Removed: FltClosePort() Added: PFLT_PORT for port handles Added: FltCloseClientPort() Added: FltCloseCommuniciationPort() FltCancelFileOpen() IoCancelFileOpen() bug has been fixed FltPerformAsynchronousIo() Can now be used on FSCTL operations © 2004 Microsoft Corporation. All rights reserved.

Filter Manager Changes (May 2004) Following fields were modified to be a CONST FLT_CALLBACK_DATA.Thread FLT_CALLBACK_DATA.Iopb FLT_RELATED_OBJECTS.* © 2004 Microsoft Corporation. All rights reserved.

New APIs (May 2004) Added lock APIs for EResources FltAcquireResourceExclusive() FltAcquireResourceShared() FltReleaseResource() Use existing Ex routines to init and delete Acquire/release wrapped by KeEnter/LeaveCriticalRegion Added lock APIs for PushLocks FltInitializePushLock() FltDeletePushLock() FltAcquirePushLockExclusive() FltAcquirePushLockShared() FltReleasePushLock() © 2004 Microsoft Corporation. All rights reserved.

New APIs (May 2004) FsRtl Byte Range Lock package support Added new routines FltAllocateFileLock() FltCheckLockForReadAccess() FltCheckLockForWriteAccess() FltFreeFileLock() FltInitializeFileLock() FltProcessFileLock() FltUninitializeFileLock() Use in conjunction with existing routines Allows minifilter to process byte range lock operations © 2004 Microsoft Corporation. All rights reserved.

New APIs (May 2004) New APIs FltIsOperationSynchronous() FltRequestOperationStatusCallback() Used to request a callback which receives the return value from IoCallDriver() Necessary if you need to know if an oplock was granted or not FltIsOperationSynchronous() FltSetSecurityObject() © 2004 Microsoft Corporation. All rights reserved.

New APIs (May 2004) New APIs FltIs32bitProcess() Always returns TRUE on 32bit processors FltCreateSystemVolumeInformationFolder() Will create the “System Volume Information” folder if it does not already exist. Created with correct ACLs © 2004 Microsoft Corporation. All rights reserved.

New APIs (Nov 2003) FltReferenceContext() Add reference to context object Call FltReleaseContext() to remove reference FltReferenceFileNameInformation() Add reference to name information object Call FltReleaseFileNameInformation() to remove reference © 2004 Microsoft Corporation. All rights reserved.

New APIs (Nov 2003) FltGetRoutineAddress() FltIsIoCanceled() Retrieves address of filter manager routines by name Used for dynamic detection of new filter manager APIs FltIsIoCanceled() Returns if the given operation has been canceled or not FltNotifyFilterChangeDirectory() See FsRtlNotifyFilterChangeDirectory() Allows minifilter to process directory change notification operations © 2004 Microsoft Corporation. All rights reserved.

New APIs (Nov 2003) Additional APIs which support non-recursive operations: FltQueryInformationFile() FltSetInformationFile() FltQueryVolumeInformationFile() FltQuerySecurityObject() FltFlushBuffers() FltFsControlFile() FltDeviceIoControlFile() See ZW equivalents © 2004 Microsoft Corporation. All rights reserved.

New APIs (Apr 2003) FltCancelFileOpen() FltGetFileNameInformationUnsafe() Allows you to retrieve a file name given a file object and an instance Should only use when you know query is safe from deadlocking system FltCancelFileOpen() No longer return special status from post-Create callback Equivalent of IoCancelFileOpen() IoCancelFileOpen() bug has been fixed © 2004 Microsoft Corporation. All rights reserved.

New APIs (Apr 2003) FltIsVolumeWritable() FltEnumerateVolumeInformation() FltGetRequestorProcess() and FltGetRequestorProcessId() Work the same as their Io equivalents, but take a FLT_CALLBACK_DATA structure © 2004 Microsoft Corporation. All rights reserved.

Three New Minifilter Samples (May 2004) MetadataManager Shows what a minifilter needs to do to manage a private metadata file Handles volume lock/unlock Handles volume mount/dismount including surprise removal Ctx Shows how to use Instance, Stream, and StreamHandle contexts CDO Shows how to write a minifilter which needs to have a control device object Shows how to handle filter load/unload requests © 2004 Microsoft Corporation. All rights reserved.

Release Plan Already in XP SP2 Already in Srv03 SP1 Already in Longhorn Running prototype for W2K Working on final approval for a QFE which runs on W2K SP4 © 2004 Microsoft Corporation. All rights reserved.