1. PCI Express Update for Windows Longhorn
Tony Pierce
Technical Evangelist Windows Hardware Evangelism microsoft.com
Microsoft Corporation

2. Session Outline PCI Express Update PCI Express 1.1
PCI Express features supported in Windows codenamed “Longhorn”
PCI Express features not supported in Longhorn
Resource considerations during design
PCI Express 1.1
Overview of Delta from PCI Express1.0a
Functionality that should be considered in Windows Platform design

3. Session Goals Attendees should leave this session with the following:
Know which PCI Express features are supported by Longhorn:
How feature is supported by Longhorn
Any hardware, firmware, or driver considerations to work well with Longhorn
Know which PCI Express features are not supported by Longhorn:
Why features are not supported
Any work to address feature for future support
Which of the new functionality in PCI Express 1.1 is important when designing for Longhorn

4. Enhanced Configuration Space: Support
Memory mapped access to enhanced configuration space
MCFG table indicates memory base to OS
No new interfaces
Existing APIs updated to support 4K enhanced config space:
BUS_INTERFACE_STANDARD
IRP_MN_READ_CONFIG, IRP_MN_WRITE_CONFIG
_OSC indicates support by OS
ASL config access PCIConfig operation region extended to 4K
Segments support
SAL support for Itanium

5. Enhanced Configuration Space: Considerations
Firmware considerations
Use MCFG table to communicate memory mapped config base to Longhorn
_OSC method indicates segment and ASL support by OS
See PCI Firmware Specification V3.0 at
Driver considerations:
Drivers must NOT use CFC/CF8
Driver must use documented interfaces for configuration space access

6. PCI Express Registers PCI Express Capability registers
Longhorn will comprehend and program registers
Devices with capability version not understood by OS (1 is the only currently defined value) will be treated like a PCI device
Save/Restore of PCI Express Configuration Settings
Save and restore PCIe capability settings and registers across power events
BIOS Configurations
BIOS configurations should be maintained on Longhorn
Not always possible

7. Device Serial Numbers Recommended for removable devices
Must be unique per device, same for functions okay
Allow device setting migration from port to port
Identifying new device in same port with a different serial number needs a remove/reinstall
Required for combo PCIe/USB ExpressCards

8. PCI Express Hierarchy
Hierarchies must follow PCIe specification – device type tells what it is
Violation will assert and not have PCIe features enabled
Violations involve root complex and switches
No switches or endpoints in root complex
Virtual PCI-to-PCI bridges must be implemented in switches
No integrated devices appearing on internal bus of switches

9. Implementation Considerations
Firmware considerations:
Some base features, such as Max Payload Size/Max Read Request Size, should be initially set by firmware
OS may override firmware settings
Driver considerations:
Drivers will not be allowed to modify base feature registers
Hardware considerations:
Pay attention to switch implementation and PCIe hierarchy
Be sure to properly support TransactionsPending
Serial numbers required for certain classes of devices

10. Message Signaled Interrupt (MSI)
Kernel updates to improve interrupt handling and support
Both MSI/MSI-X fully supported:
Reduces problems associated with interrupt sharing
MSI/MSI-X preferred over INTx emulation for PCIe
Class driver support:
VideoPort
StorPort
NDIS
Others in future
Windows Driver Foundation fully supports MSI/MSI-X

11. MSI: Considerations Driver considerations: Hardware considerations:
Driver changes required to use MSI/MSI-X
Whitepaper:
Hardware considerations:
Single vs. multiple messages:
Most devices support single message
Still requires a read to interrupt status register
Multiple messages
Different interrupt status use different messages
MSI-X does not require contiguous IDT entries or identical target processor sets across messages (very flexible)

12. Hot Plug System interrupt signals Hot Plug event
Root/switch ports must implement MSI as the system interrupt
Legacy INTx support is allowed as well
State machine supports all combinations of hardware elements
Supports orderly or surprise removal
ExpressCard can be surprise removed
Edge-style cards will issue OS request for removal

13. Hot Plug: Considerations
Firmware considerations:
Firmware implements _OSC for OS to take native control
See PCI Firmware Specification V3.0
Driver considerations:
No additional driver requirements but drivers must be able to start and stop properly – test thoroughly!
Hardware considerations:
Special HW design considerations for compatibility with legacy OSs

14. PME System interrupt signals PME event PME event handling
Root/switch ports must implement MSI as the system interrupt
Legacy INTx support is allowed as well
PME event handling
OS interrupt handler reads Requester ID
Notifies driver of waking device of the wakeup event
Driver interface remains the same – Wait/Wake DDK rules don’t change
Fully supported by Windows Driver Foundation
ACPI PME notification highly discouraged
Increases firmware support and development costs
Firmware-directed PME doesn’t work in many cases and causes crashes

15. PME: Considerations Firmware considerations: Hardware considerations:
Firmware implements _OSC for OS to take native control
See PCI Firmware Specification V3.0
Hardware considerations:
Don’t share PME between PCI Express and PCI/PCI-X
Special HW design considerations for PME# routing for PCI Express to PCI/PCI-X bridges

16. Active State Power Management (ASPM)
Enables ASPM based on user power scheme and performance preference
Plans are still under consideration
Policy
System wide on/off
Per device on/off
Fixed per OS or user defined power scheme
Supports setting of ASPM on hot plugged devices

17. ASPM: Considerations Firmware considerations: Driver considerations:
Firmware can enable at POST if desired
OS may override firmware settings
Driver considerations:
Drivers will not be allowed to modify ASPM
Hardware considerations:
Devices must correctly indicate L0s/L1 exit and acceptable latencies

18. Advanced Error Logging and Reporting
Reporting and logging of correct and uncorrectable errors
I/O errors are big issues for server systems
Error logging and reporting a big part of diagnosis
Hardware vendors strongly encouraged to support this feature
Both chipsets and endpoint devices
Windows Longhorn Server will support AER via the Windows Hardware Error Architecture (WHEA)
Devices will be expected to implement support for AER
Windows expects to be able to “own” AER
PCI Express (via the PCI bus driver) will be a WHEA error source

19. Resource Considerations
The BIOS provides resource information for each PCI root bus.
Windows attempts to respect BIOS configuration of devices from these resources. When not possible, Windows chooses an acceptable location for these devices.
There is a growing trend towards 64-bit systems with large memory configurations and large number of PCI devices on every system.
This puts constraints on the lower region of the physical address space.
Configure PCI Devices with memory resources above 4 GB

20. Memory Resource Assignment
Windows Server 2003
A PCI device with BIOS configuration above 4GB, is always assigned resources from below the 4GB region
If no range below 4GB region is available, then the device is assigned a range above the 4GB boundary
This holds good even if the Windows OS cannot physically access the address range above 4GB
Longhorn
A memory address range above 4GB is available for PCI devices only if that range is physically accessible by the OS
Within this constraint, Windows will always attempt to respect the BIOS configuration on a PCI device.
A PCI device with 64-bit BARs and no BIOS configuration is still assigned a memory address range above 4GB as available on the parent

21. Hardware Design Recommendations
For Longhorn to assign memory resources to a PCI device above 4GB:
Entire device path from the PCI root bus to the device must support 64-bit memory BARs
A PCI device behind a PCI-PCI bridge can only have a Prefetchable memory window above the 4GB boundary
This constraint is imposed by the PCI-PCI bridge specification
PCI devices on root bus can have any memory type resources above the 4GB boundary
Include Prefetchable BARs on PCI devices behind a bridge

22. BIOS Design Recommendations
The ACPI BIOS should describe the memory range above 4GB in the _CRS and/or _PRS of the PCI root bus
This is described using the QWord Address Space descriptor as defined in the ACPI spec (Section ).
Windows will evaluate the _SRS method with a buffer in the same format as the _CRS/_PRS
The memory range for the PCI root bus should not overlap with the physical RAM or some other range
The memory range is required to be physically accessible by the processor/chipset
The ACPI BIOS should configure the resources on the PCI devices after evaluating the _OSI method to account for the Server 2003 behavior
The ACPI BIOS should return an appropriate buffer in the evaluation of the _CRS/_PRS on a PCI root bus to account for the Server 2003 behavior

23. PCI Express Features Not Supported in Longhorn
Extended VC/Isochronous
Not supported and also a potential logo failure issue
Slot Power Budgeting

24. Extended VC/Isochronous Is Not Supported: Potential Logo Failure!
Optional feature of PCI Express
Chipsets may or may not implement it
Devices cannot rely on chipset Isochronous support, and must be capable of using VC0!
No usage rules and latency guarantees
VCs are a shared resource
Different device assumptions may be incompatible
Designed For Windows Logo requirements in place:
WLP 2.2 and 3.0 requirements disallow usage of Extended VC/Isochronous

25. Slot Power Budgeting Why Not Supported: Future Direction:
No standard mechanism to convey overall system power budget to OS
Firmware can set slot power budget before booting OS
Future Direction:
Currently being evaluated

26. PCI Express 1.1 PCI Express 1.1 contains
All PCI Express ECNs for 1.0a
All errata against 1.0a
Learnings of Design community for past 2 years
PCI Express 1.1 scheduled to be released 1H05

27. PCI Express 1.0a ECNs—Base
Root Complex Topology Discovery
Optional: enables software discovery of Root Complex internal components to program extended capabilities
Integrated Devices/Event Collector
Optional: allows integration of devices in a Root Complex; adds Root Complex Event Collector endpoint association capability
Reset Limit Adjustment
Reduces time limit from end of Fundamental Reset to the entry to LTSSM Detect from 80ms to 20ms; clarifies PERST#. Must validate components against new limit.
Assorted Error Clarifications
Clarifies and removes inconsistencies in the description of error handling
Multi-Function Virtual Channel
Optional: defines new MFVC capability structure to enable QoS management such as mapping of traffic classes to VCs, etc in Multi-Function Devices
MMConfig/Enhanced Configuration
Describes multiple enhancements to the way in which the memory-mapped configuration space is accessed in an improved and flexible manner
Training Error Removal
Removes references to Training Error (chapters 4, 6-7) from the specification. Remove or disable Training Error if implemented.
Bridge Updates
Incorporates VDM from the PCI Express Bridge specification and shows usage of the Bridge Configuration Retry Enable bit
MSI-X Addition
Requires devices that support interrupts to implement MSI or MSI-X or both
Request Dependencies
Clarifies the rules for forwarding packets by Root Complexes, Switches and End Points between different Virtual Channels
CRS Software Visibility
Describes Root Complex behavior for configuration requests that return the Configuration Request Retry Status (CRS)
Software
Transaction
Data Link
Physical
Mechanical

28. PCI Express 1.0a ECNs—Base
Platform Reference Clock Power Management
Provides for platform clock power management reporting mechanism
VSEC
Defines Vendor-Specific Extended Capability for Extended Configuration Space in a standard and architected manner
Return Loss
Relaxes transmitter differential RL budget from 12dB to 10dB and receiver differential RL budget from 15dB to 10dB. Phy signal validation must comprehend these changes.
PME Turn Off
All components are potentially required to participate in PME_Turn_Off protocol and must comprehend these changes even if they have no special power management functionality.
Surprise Down Error
Introduces optional error logging mechanism to report unexpected link failure (transition from Active to Inactive)
Flow Control Initialization
Updates the FC Init protocol with clarifications to the permitted intervals. All components must verify compliance with these changes.
Hot Plug
Includes significant revisions to all hot plug specifications. System software impacted. Improves ability to validate hardware for correct hot plug functionality. Hardware simplifications may be enabled for some components.
Posted Request Acceptance
Imposes an acceptance limit on posted requests (memory writes) to avoid or reduce PCIe fabric congestion
Jitter
Incorporates many changes including Tx & Rx eye measurements done with ‘clean’ clock. Phy signal validation must comprehend the changes.
Error Reporting
Modifies the required behavior for all devices when enabled to report detection of non-fatal errors.
Software
Transaction
Data Link
Physical
Mechanical

29. ECNs that are Important to Windows
Integrated Devices - Event Collector
Allows chipset integrated devices that are Express-like but sit on an internal bus to take advantage of Express features
Reset Limit Adjustment
Clarifications impact hot plug
MMCONFIG ECN
Critical for correct system operation
Windows will rely on systems and firmware being built to this spec

30. ECNs that are Important to Windows
MSI-X ECN for PCI Express
MSI-X offers great flexibility and benefits for many devices
Longhorn is planned to have full support
CRS Software Visibility
Windows my use this capability. More relevant to high end systems
Longhorn plan under investigation.
PR Acceptance Limit
Not software visible but very important clarifications
PCI Express Hot Plug
Longhorn hot plug implementation will be built to these updates

31. ECNs that are Important to Windows
PCI Express Base PCI Bus Power Management
Longhorn will utilize the no device reset
Platform reference clock power management
Not software visible but very important for ExpressCard implementations
Surprise Down Error
Critical piece of hot plug
All hot plug ports need to implement
Error Reporting
Important to systems that implement Advanced Error Reporting capability

32. Call to Action
Ensure you are up to date on the PCI Express 1.1 and PCI Firmware 3.0 specifications
Implement _OSC in your firmware for proper handoff of PCI Express Features to Windows Longhorn
Ensure your devices meet Designed for Windows Logo requirements
Ensure you are signed up for the Windows Longhorn Beta program
Regularly test
Provide Feedback
Send your hardware to our test labs

33. Community Resources Windows Hardware & Driver Central (WHDC)
Technical Communities
Non-Microsoft Community Sites
Microsoft Public Newsgroups
Technical Chats and Webcasts
Microsoft Blogs

34. Additional Resources Community Sites List of Newsgroups
List of Newsgroups
Attend a free chat or webcast
Locate a local user group(s)
Non-Microsoft Community Sites

35. Additional Resources Email Specs Whitepapers Related Sessions
microsoft.com
Specs
PCI Express Base Specification v1.0a
PCI Firmware Specification v3.0
Whitepapers
PCI Express Overview
PCI Express Legacy Transition
PCI Express Longhorn Implementation
PCI-PCI Bridge Resource Issues
PCI Hot-Plug
PCI I/O Reduction
Related Sessions
Windows Hardware Error Architecture
Error Management Solutions Synergy with WHEA
PCI Express Trusted Configuration Space

37. © 2005 Microsoft Corporation. All rights reserved.
This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.