1. 11/17/2018 9:30 PM
Session Code: ARC 420
Transaction Support Under The Covers: Transactional NTFS, KTM, and WinLS
Rajeev Nagar
Surendra Verma
Dana Groff
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

2. 11/17/2018 9:30 PM
Tools
Client Application Model
Web & Service Application Model
Data Systems Application Model
Mobile PC & Devices Application Model
Command Line
Compact Framework
Avalon
Windows Forms
ASP.NET / Indigo
Win FS
Yukon
System.Console
Mobile PC Optimized
System.Web
System.Storage
System.Data.SqlServer
System.Windows.Forms
System.Windows
NT Service
System.Windows
System.Windows.Forms
System.ServiceProcess
Presentation
Data
Communication
System.Windows
System.Search
System.Data
System.Messaging
System. Discovery
System.Collaboration
UI Element
Explorer
Media
Annotations
SqlClient
DataSet
System.DirectoryServices
RealTimeEndpoint
Documents
Controls
Animation
SqlTypes
Mapping
Active Directory
TransientDataSession
Monitoring
System.Remoting
Text Element
Dialogs
Controls
SqlXML
ObjectSpaces
SignalingSession
Logging
System.Runtime.Remoting
Uddi
Shapes
SideBar
Control
OdbcClient
ObjectSpace
Media
Shape
Notification
Panel
Relevance
OleDbClient
Query
Activities
Ink
Navigation
Design
OracleClient
Schema
System.Web.Services
System.MessageBus
Web.Service
Transport
Queue
System.Storage
System.Windows.Forms
System.Web.UI
Port
PubSub
Item
Core
Description
Forms
Page
WebControls
Discovery
Channel
Router
Relationship
Contact
Control
Control
Adaptors
Service
Policy
Media
Location
Protocols
Print Dialog
HtmlControls
Design
Peer Group
Audio
Message
Design
MobileControls
Video
Document
System.Net
Images
Event
System.Web
System.Help
System.Speech
HttpWebRequest
NetworkInformation
System.Xml
Personalization
FtpWebListener
Sockets
System.Drawing
Recognition
Synthesis
Caching
Schema
Xpath
SslClientStream
Cache
System.NaturalLanguageServices
Serialization
Query
SessionState
WebClient
Fundamentals
Base & Application Services
Security
Configuration
Deployment/Management
System.Timers
System.Text
System.Collections
System.Windows. TrustManagement
System.Web.Configuration
System.Globalization
System.Design
System.Security
System.Web
Generic
System.MessageBus.Configuration
Administration
System.Serialization
System.IO
System.Web. Security
Authorization
Permissions
System.ComponentModel
System.Configuration
Management
System.Threading
Ports
AccessControl
Policy
System.CodeDom
System.Resources
System.Message Bus.Security
Credentials
Principal
System.Management
System.Runtime
System.Reflection
Cryptography
Token
System.Deployment
Serialization
InteropServices
System.EnterpriseServices
System.Diagnostics
CompilerServices
System.Transactions
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

3. Internet Connection Firewall
11/17/2018 9:30 PM
Presentation
Data
Communication
Communication
Avalon
Avalon
Windows Forms
Windows Forms
ASP.NET
WinFS
WinFS
ADO.NET
Indigo
Indigo
Collaboration
Document UI
Media
Object
T/SQL
XML
Service
Models
Services
Schemas
Desktop Window
Manager
Controls Interop
Engine
Connectivity
Messaging
Services
Desktop
Services
Adaptive UI
Engine
People
Group
ObjectSpaces
People and
Groups
Synchronization (WinFS, Win32..)
Channels (Datagram, Reliable, Peer, …)
Policy
Engine
Page/Site Composition
Calendar
Media
Queuing
Eventing
Routing
Presentation Object Manager
Desktop Composition Engine
Application
Services
Document …
DataSet
Channel
Security
Collaboration
History
Media Services
InfoAgent
(PreferenceRules..)
Data Model
Transport Channels
(IPC, HTTP, TCP…)
Animation and Composition
Media Processing
Capture and
Sourcing
Designer
Services
Personalization and
Profiling Services
Items
SQL XML
Message
Encoder
Real-Time
Activities
System Services
FileSystem Services
(MetaDataHandlers..)
Relationships
Providers
Communications Manager (Port)
Transaction
Signaling
Framework
Hardware
Rendering
Software Rendering
and Sinks
Controls
Membership and
Security Services
Extensions
Federation
Base Operating System Services
CLR
Base Class Libraries
Network Class Library
Application Deployment
Engine
(Click-Once)
Memory Manager
Code Execution
Loader
Security
Serialization
Hosting Layer
Network Services
Demand Activation and Protocol Health
GDI/GDI+
Window
Manager
Global
Audio
Engine
Direct 3D Graphics
Transactions
Identity &
Security
System
Storage
PNRP
Native
WiFi
SIP
Management
Services
(Event Logs,
Tracing,
Probes,
Auto Update,
Admin)
TCP
Listener
UDP
Listener
IPC
Listener
Graphics drivers
Lightweight
Transactions
Transaction Coordinator
Virtual Shadow
Copy Service
File Replication
Service
Virtual Disk
Service
Internet Connection Firewall
DDI
Input
Manager
Audio
Drivers
DirectX
Graphics
Mini port
Kernel Transaction Manager
Logging Service
Redirectors
Distributed
File System
Filter
Manager
Cache
Manager
Protocols
Filter
Engine
TCP, UDP
IPV4, IPV6
IPSEC
QOS
HTTP
Listener
Transactional
NTFS
Plug and Play
Memory Manager
Power Manager
Config Manager
Process
Manager
Security Reference Monitor
LPC Facility
Universal
Data
Format
FAT 16/32
IO Manager
Device Drivers
Kernel
SCSI/FC
802.3
802.11 ..
Kernel Mode
Hardware Abstraction Layer
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

4. Kernel Infrastructure for Tx
11/17/2018 9:30 PM
Kernel Infrastructure for Tx
TxF
KTM
System.IO
System.Transactions
WinLS
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

5. Transactional NTFS (TxF)
11/17/2018 9:30 PM
Transactional NTFS (TxF)
Surendra Verma
Software Design Engineer (Lead)
Core File Services
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

6. Transactional NTFS (TxF)
11/17/2018 9:30 PM
Transactional NTFS (TxF)
Web site publishing
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

7. Functionality Transaction Support for all NTFS file operations
11/17/2018 9:30 PM
Functionality
Transaction Support for all NTFS file operations
Full Atomicity, Consistency, Isolation, Durability
Allow an arbitrary number of file system operations as an atomic unit
Reads, writes, file creations, deletions, renames, etc.
Memory mapping
Network file access via SMB
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

8. Usage Scenarios For TxF
11/17/2018 9:30 PM
Usage Scenarios For TxF
Safe and efficient in-place update of documents
Transactional multi-file update and network transfer
Files in NTFS, related data in another transactional store such as SQL
WinFS Filestream (File Backed)
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

9. Example Usage – Compliance, Archive & Retrieval
11/17/2018 9:30 PM
Example Usage – Compliance, Archive & Retrieval
Metadata index (SQL)
Dequeue message
Add metadata to index
Store message in NTFS Tx
Exchange
Server
Browser
MSMQ
Client
NTFS
Query index
Add “read” record to database
Add “read” record to message in NTFS
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

10. Transactional NTFS (TxF)
11/17/2018 9:30 PM
Transactional NTFS (TxF)
Transactional File I/O
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

11. 11/17/2018 9:30 PM
Isolation And Locking
Transactions don’t see changes made by other transactions
Isolation for non-transacted readers
File is the unit of locking
Readers don’t block
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

12. Logging Modes Simple Mode Full Mode
11/17/2018 9:30 PM
Logging Modes
Simple Mode
Only before-images of changes logged
Minimal logging for common cases
Create a new file and write to it
Delete or rename a file
Extend a file
Fully truncate or overwrite a file
Full Mode
Full change history over time in log
Very Useful, comes at a cost
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

13. Build Powerful Solutions
11/17/2018 9:30 PM
Build Powerful Solutions
Disaster recovery
Specify and backup a set of files
Set Logging Mode to Full
Continuously Archive log
Restore the set of files and archived log from backup
Roll-forward using the log to a chosen point in time
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

14. Resource Units (RU) Subset of a volume: System-managed Resource Unit
11/17/2018 9:30 PM
Resource Units (RU)
Subset of a volume:
You create it for a directory tree
Keep data and transactional meta-data in one unit
Unit moves as a whole
You control start, stop and policies
System-managed Resource Unit
Includes files not in any other RU
Self managed – logging mode, …
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

15. How To Program To It System.IO, System.Transaction, Win32
11/17/2018 9:30 PM
How To Program To It
System.IO, System.Transaction, Win32
Start Transaction
Associate with current context
Do multiple file operations
Commit/Rollback Transaction
Try Win32 APIs in Hands On Labs
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

16. Kernel (mode) Transaction Manager
11/17/2018 9:30 PM
Kernel (mode) Transaction Manager
Rajeev Nagar
Lead Program Manager
Core File Services
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

17. Technology Overview Kernel infrastructure for transaction support
11/17/2018 9:30 PM
Technology Overview
Kernel infrastructure for transaction support
Services for both kernel and user mode components
Superior performance and scale
Well integrated with Distributed Transaction Services – supplies the state machine core for distributed transactions
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

18. Distributed Transaction Flow
11/17/2018 9:30 PM
Distributed Transaction Flow 4
Distribute Tx 4a
App
CRM
CRM
WinFX
App
App 5
SQL Update
WinFX
SQL Server
Create Tx
File I/O (Tx) 2 1
TxF (RM)
KTM 3
Enlist
KTM
TxF
Machine 1
Machine 2
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

19. KTM Features Include Full (fast) 2-phase commit on single machine
11/17/2018 9:30 PM
KTM Features Include
Full (fast) 2-phase commit on single machine
Virtual clocks & save-points
Roll-forward recovery
Multiple log support
Highly available (log failover)
Integrated with protocol layers
Kernel, Win32 and System.Transaction Interfaces
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

20. Windows Logging Service
11/17/2018 9:30 PM
Windows Logging Service
Dana Groff Program Manager
Core File Services
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

21. Usage Scenarios When you need Examples Sequential Record Oriented Data
11/17/2018 9:30 PM
Usage Scenarios
When you need
Sequential Record Oriented Data
Grouped Records
Persistence
High performance
Examples
Transactional Stores
Change Records for Replication
Event Logging
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

22. 1 Kernel Transition per flush
11/17/2018 9:30 PM
Data Flow
Log File
Application
Application
Marshal Records
Marshal Records
Encoded  Unique Identifier (LSN)  Batched
Single I/O
Flush Queue
1 Kernel Transition per flush
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

23. Windows Logging Service (WinLS)
11/17/2018 9:30 PM
Windows Logging Service (WinLS)
Create And Write To A Log
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

24. Log:/full/path/name/to/baseLog::StreamName
11/17/2018 9:30 PM
Log Records in Streams
Log:/full/path/name/to/baseLog::StreamName
One or many streams in a log No need to change to code, just change the logname!
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

25. Windows Logging Service (WinLS)
11/17/2018 9:30 PM
Windows Logging Service (WinLS)
Easy Streams
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

26. 11/17/2018 9:30 PM
Recycling Space
Restart Area A R C H I V E
Archiving without interrupting reading and writing to the log
Restart Area : dedicated area to write restart / recovery data.
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

27. Windows Logging Service (WinLS)
11/17/2018 9:30 PM
Windows Logging Service (WinLS)
Recycling Log Space
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

28. Key Features To Remember
11/17/2018 9:30 PM
Key Features To Remember
Fast, dependable, and optimized I/O
Multiple streams without added coding
Services for transactional logging
APIs:
Managed Code (not available for PDC)
Win32
Kernel
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

29. Community Resources Get Your Questions Answered!
11/17/2018 9:30 PM
Community Resources Get Your Questions Answered!
HOL
HOL-404 Transactional File System
Meet the team Thursday 9 am to 11 am
Send Us
309 Foyer
connect with PDC 2003 Architecture and Infrastructure Speakers and their product team members
Newsgroups:
microsoft.public.windows  developer.winfx.fundamentals
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

30. © 2003-2004 Microsoft Corporation. All rights reserved.
11/17/2018 9:30 PM
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

31. 11/17/2018 9:30 PM
© Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.