© 2004 Microsoft Corporation. All rights reserved. 1 Processing IO Operations.

Slides:



Advertisements
Similar presentations
Filter Manager Overview
Advertisements

Generalized Requests. The current definition They are defined in MPI 2 under the hood of the chapter 8 (External Interfaces) Page 166 line 16 The objective.
WDM 드라이버의 기본 구조 What is WDM?
Chapter 3 Process Description and Control
CSCC69: Operating Systems
A Pipeline for Lockless Processing of Sound Data David Thall Insomniac Games.
Loading and Unloading Minifilters
© 2004 Microsoft Corporation. All rights reserved. 1 Context Management.
© 2004 Microsoft Corporation. All rights reserved. 1 File Name Management.
Precept 3 COS 461. Concurrency is Useful Multi Processor/Core Multiple Inputs Don’t wait on slow devices.
IO Request Flow in WDF Kernel-Mode Drivers
Introduction to Operating Systems – Windows process and thread management In this lecture we will cover Threads and processes in Windows Thread priority.
Concurrency: Mutual Exclusion, Synchronization, Deadlock, and Starvation in Representative Operating Systems.
3.5 Interprocess Communication
CS 206 Introduction to Computer Science II 10 / 26 / 2009 Instructor: Michael Eckmann.
I/O Request Flaw in WDF Kernel-Mode Driver
© 2004 Microsoft Corporation. All rights reserved. 1 Accessing User Buffers.
1 I/O Management in Representative Operating Systems.
1 Threads Chapter 4 Reading: 4.1,4.4, Process Characteristics l Unit of resource ownership - process is allocated: n a virtual address space to.
Driver Verifier Advancements In Windows 7 Daniel Mihai Principal Software Design Engineer Windows Engineering Tools.
Chapter 4: Threads Adapted to COP4610 by Robert van Engelen.
I/O Systems ◦ Operating Systems ◦ CS550. Note:  Based on Operating Systems Concepts by Silberschatz, Galvin, and Gagne  Strongly recommended to read.
Kumar Rajeev SDET Microsoft Corporation. KMDF does not support HID minidrivers natively due to conflicting KMDF and HID architecture requirements HID.
File System and File System Filter Ecosystem Update Neal Christiansen Development Lead File System Filter Team.
CSE451 Introduction to Operating Systems Spring 2008 Tour of the Windows NT File Systems Gary Kimura & Mark Zbikowski.
AHCI: ATAport Miniport Example. Outline AHCI Features Goals Basics AHCI ATA Miniport Design Philosophy Memory Structures and Resources Enumeration IO.
Experience with Processes and Monitors in Mesa
1 Lecture 4: Threads Operating System Fall Contents Overview: Processes & Threads Benefits of Threads Thread State and Operations User Thread.
© Microsoft Corporation1 Windows Kernel Internals I/O Architecture *David B. Probert, Ph.D. Windows Kernel Development Microsoft Corporation.
FINAL MPX DELIVERABLE Due when you schedule your interview and presentation.
Object Oriented Analysis & Design SDL Threads. Contents 2  Processes  Thread Concepts  Creating threads  Critical sections  Synchronizing threads.
1 CSE Department MAITSandeep Tayal Computer-System Structures Computer System Operation I/O Structure Storage Structure Storage Hierarchy Hardware Protection.
CHAPTER 3 TOP LEVEL VIEW OF COMPUTER FUNCTION AND INTERCONNECTION
© 2004 Microsoft Corporation. All rights reserved. 1 Modifying Parameters.
1-1 Embedded Network Interface (ENI) API Concepts Shared RAM vs. FIFO modes ENI API’s.
Recall: Three I/O Methods Synchronous: Wait for I/O operation to complete. Asynchronous: Post I/O request and switch to other work. DMA (Direct Memory.
The Functions of Operating Systems Interrupts. Learning Objectives Explain how interrupts are used to obtain processor time. Explain how processing of.
Lecture 3 Process Concepts. What is a Process? A process is the dynamic execution context of an executing program. Several processes may run concurrently,
CE Operating Systems Lecture 11 Windows – Object manager and process management.
May 8, USB Client Driver Etiquette Jay Senior Windows Base Developer Microsoft.
Device Drivers CPU I/O Interface Device Driver DEVICECONTROL OPERATIONSDATA TRANSFER OPERATIONS Disk Seek to Sector, Track, Cyl. Seek Home Position.
NT Kernel CS Spring Overview Interrupts and Exceptions: Trap Handler Interrupt Request Levels and IRT DPC’s, and APC’s System Service Dispatching.
WHDC PowerPoint Template Notes & Handouts
© 2004 Microsoft Corporation. All rights reserved. 1 Minifilter Generated IO’s.
CE Operating Systems Lecture 2 Low level hardware support for operating systems.
Operating Systems 1 K. Salah Module 1.2: Fundamental Concepts Interrupts System Calls.
1 CS.217 Operating System By Ajarn..Sutapart Sappajak,METC,MSIT Chapter 2 Computer-System Structures Slide 1 Chapter 2 Computer-System Structures.
How to write a MSGQ Transport (MQT) Overview Nov 29, 2005 Todd Mullanix.
CE Operating Systems Lecture 2 Low level hardware support for operating systems.
Lecture 4 Mechanisms & Kernel for NOSs. Mechanisms for Network Operating Systems  Network operating systems provide three basic mechanisms that support.
4P13 Week 12 Talking Points Device Drivers 1.Auto-configuration and initialization routines 2.Routines for servicing I/O requests (the top half)
CSC 360, Instructor: Kui Wu Thread & PThread. CSC 360, Instructor: Kui Wu Agenda 1.What is thread? 2.User vs kernel threads 3.Thread models 4.Thread issues.
© 2004 Microsoft Corporation. All rights reserved. 1 Rules for Filters (both Legacy and Mini)
Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze Unit OS3: Concurrency 3.3. Advanced Windows Synchronization.
Embedded Computer - Definition When a microcomputer is part of a larger product, it is said to be an embedded computer. The embedded computer retrieves.
Daniel Mihai Software Design Engineer Microsoft Corporation.
Processes and threads.
GPU Computing CIS-543 Lecture 10: Streams and Events
Topics Covered What is Real Time Operating System (RTOS)
Background on the need for Synchronization
CSE451 I/O Systems and the Full I/O Path Autumn 2002
I/O Manager, 64-bit Porting, and New Driver Models
Module 2: Computer-System Structures
Processor Fundamentals
Top Half / Bottom Half Processing
Chapter 4: Threads.
CSE 153 Design of Operating Systems Winter 2019
Software Engineering and Architecture
More concurrency issues
Chapter 13: I/O Systems “The two main jobs of a computer are I/O and [CPU] processing. In many cases, the main job is I/O, and the [CPU] processing is.
Presentation transcript:

© 2004 Microsoft Corporation. All rights reserved. 1 Processing IO Operations

© 2004 Microsoft Corporation. All rights reserved. 2 In the IRP world… In your dispatch (pre-operation) routine you can: Pass through the operation with no completion Do not care about this operation Do not need to see the final status of this operation Pend an operation Will call IoCallDriver() / IoCompleteRequest() from a worker thread sometime later Pass through the operation, setting a completion Need to see the final status of the operation Synchronize the operation Allows post-operation processing to be done in the context of the same thread that the dispatch routine was called – safe IRQL, maintain your local variable state Complete the operation

© 2004 Microsoft Corporation. All rights reserved. 3 In the IRP world… In your completion (post-operation) routine you can: Do completion work directly in completion routine All the post-operation work can be done at DPC-level Do completion work at safe IRQL Queue to a worker thread to allow processing at safe IRQL, call IoCompleteRequest() to continue completion processing Synchronize back to dispatch by signaling event Dispatch routine is waiting and will pick up doing the post- operation work in the context of the dispatch routine thread, call IoCompleteRequest() when done Cancel a successful CREATE operation

© 2004 Microsoft Corporation. All rights reserved. 4 In the Filter Manager world… Still have same flexibility but is mainly controlled by the value returned in preOperation completion No longer need code that does IRP maintenance Copying parameters to next stack location Propagating pending flag

© 2004 Microsoft Corporation. All rights reserved. 5 PreOperation Callback Return Values FLT_PREOP_SUCCESS_NO_CALLBACK PostOperation callback will not be called for this operation FLT_PREOP_SUCCESS_WITH_CALLBACK PostOperation callback will be called for this operation Not valid to return if no postOperation callback was registered for this operation FLT_PREOP_PENDING Operation will be pended Filter must complete or continue processing this IO sometime in the future

© 2004 Microsoft Corporation. All rights reserved. 6 PreOperation Callback Return Values (cont) FLT_PREOP_SYNCHRONIZE PostOperation callback will be called in the context of the current thread at safe IRQL IRP_MJ_CREATE operations are always synchronized Do not synchronize asynchronous read/write operations Will destroy the performance of the system FLT_PREOP_COMPLETE Filter completed this operation FLT_CALLBACK_DATA.IoStatus will be returned as the final status for operation FLT_PREOP_DISALLOW_FASTIO FAST_IO will be failed – IO Manager may reissue request via IRP path Can only be returned for operations that map to FAST_IO calls ( FLTFL_CALLBACK_DATA_FAST_IO_OPERATION will be set in FLT_CALLBACK_DATA.Flags)

© 2004 Microsoft Corporation. All rights reserved. 7 Retrieving operation status For some operations it is important to retrieve the operation status returned by IoCallDriver() Needed to know if an oplock was granted or not FltRequestOperationStatusCallback() Specifies a callback routine to be called when the operation status is returned (return value of IoCallDriver()) Properly synchronized with unload Handles operation being completed by a lower minifilter/legacy filter As in the legacy model, may be called after the operation has completed

© 2004 Microsoft Corporation. All rights reserved. 8 PostOperation Callback Return Values FLT_POSTOP_FINISHED_PROCESSING Finished with post-operation processing Can continue with higher filter/IO Manager post-operation processing FLT_POSTOP_MORE_PROCESSING_REQUIRED Have queued completion work to a worker thread Filter must continue completion processing on this IO at a later time Note: PostOperation callbacks may be called at DPC level like completion routines To cancel a CREATE operation: Call FltCancelFileOpen() IoCancelFileOpen() bug has been fixed Return failure status through callback data

© 2004 Microsoft Corporation. All rights reserved. 9 Pending IO in PreOperation Callback FltCompletePendedPreOperation() Must call this to restart processing of IO pended in the preOperation callback Filter may: Complete operation Request postOperation callback through the FLT_PREOP_CALLBACK_STATUS parameter Typically called from a worker thread

© 2004 Microsoft Corporation. All rights reserved. 10 Pending IO in PreOperation Callback (cont) FltQueueDeferredIoWorkItem() Queue operation to worker thread for continued processing Can fail if this operation is not safe to post – i.e., this is paging IO or top-level IRP is already set on this thread Can fail if this Instance is in the process of being torndown

© 2004 Microsoft Corporation. All rights reserved. 11 Pending IO in PostOperation Callback FltDoCompletionProcessingWhenSafe() Can fail if it is not safe to post Not safe to post Paging IO’s Synchronize operation if you cannot handle this failure Tells filter the appropriate postOperation callback status to return Will only post operation if currently at DPC IRQL Will continue completion processing when WorkerRoutine has completed

© 2004 Microsoft Corporation. All rights reserved. 12 Pending IO in PostOperation Callback (cont) FltQueueDeferredIoWorkItem() Works just like in preOperation Will always queue work to worker thread if it is safe to do so If successfully queues work, return FLT_POSTOP_MORE_PROCESSING_REQUIRED FltCompletePendedPostOperation() Must call this to restart processing of IO pended in the postOperation callback

© 2004 Microsoft Corporation. All rights reserved. 13 Queuing Mechanisms Generic queuing support Deferred IO queuing support Cancel safe queuing support

© 2004 Microsoft Corporation. All rights reserved. 14 Generic Queuing Support FltAllocateGenericWorkItem() Allocates a generic work item FltQueueGenericWorkItem() Queues a generic work item Associated with an Instance or Filter object FltFreeGenericWorkItem() Frees a generic work item

© 2004 Microsoft Corporation. All rights reserved. 15 Generic Queuing Support (cont) Work just like ExXxxWorkItem() routines Can reuse work items Filter is responsible for allocation and freeing The Instance/Filter will not finish unloading while outstanding work items are still being processed

© 2004 Microsoft Corporation. All rights reserved. 16 Deferred IO Queue Support FltAllocateDeferredIoWorkItem() Allocates deferred IO work item FltQueueDeferredIoWorkItem() Queues deferred IO work item This can fail if operation cannot be pended FltFreeDeferredIoWorkItem() Frees deferred IO work item

© 2004 Microsoft Corporation. All rights reserved. 17 Deferred IO Queue Support (cont) Filter is responsible for allocation and freeing of deferred IO work item Deferred IO work item can be reused Instance cannot go away while there are outstanding deferred IO work items being processed

© 2004 Microsoft Corporation. All rights reserved. 18 Cancel Safe Queuing Support FltCbdqInitialize() Initializes the cancel safe queue FltCbdqInsertIo() Inserts a FLT_CALLBACK_DATA into the queue FltCbdqRemoveIo() Removes a given FLT_CALLBACK_DATA from the queue FltCbdqRemoveNextIo() Removes the next FLT_CALLBACK_DATA that meets the peek criteria from the queue FltCbdqDisable() Disables insertion of new IOs to the queue Used to allow the queue to be drained when an Instance is being torn down

© 2004 Microsoft Corporation. All rights reserved. 19 Cancel Safe Queuing Support FltCbdqEnable() Enables new IOs to be added to the queue Once the queue is initialized, it is enabled by default Used only when pending preOperation callbacks Provides cancel logic for filters that need to maintain a queue of outstanding IOs to service Equivalent functionality to IoCsqXxx() provided for IRPs Filter is responsible for disabling and draining queue on InstanceTeardownStart()

© 2004 Microsoft Corporation. All rights reserved. 20 Cancel Support FltSetCancelCompletion() Allows setting of cancel routine for a given FLT_CALLBACK_DATA FltClearCancelCompletion() Allows clearing of the cancel routine for a given FLT_CALLBACK_DATA FltIsIoCanceled() Returns TRUE if the current operation has been canceled

© 2004 Microsoft Corporation. All rights reserved. 21 Cancel Support (cont) Equivalent to IoSetCancelRoutine() for IRPs (read documentation) Filter is responsible for locking to make sure set/clear/cancel is properly synchronized Can be used to implement your own cancellation logic It is recommended that you use cancel safe queues

© 2004 Microsoft Corporation. All rights reserved. 22 Oplock Support FltInitializeOplock() FltUninitializeOplock() FltOplockFsctrl() FltCheckOplock() FltOplockIsFastIoPossible() FltCurrentBatchOplock() Internally extracts IRP from Callback Data and calls equivalent FSRTL routines

© 2004 Microsoft Corporation. All rights reserved. 23 Byte Range Lock Support FltInitializeFileLock() FltUninitializeFileLock() FltAllocateFileLock() FltFreeFileLock() FltProcessFileLock() FltCheckLockForReadAccess() FltCheckLockForWriteAccess() Use in conjunction with existing FsRtl routines Internally extracts IRP from Callback Data and calls equivalent FSRTL routines