Hardware and OS Design and Layout

Computers Desktops, PCs, Laptops…

Hardware Commodity PCs are built from similar, replaceable components Motherboard Random Access Memory (RAM) Hard Drive Peripherals (Video Card, Keyboard, Mouse)

Even Laptops? Yes, even laptops. The components may be integrated together and less replaceable, but they still adhere to standardized designs and interfaces.

Diagram Motherboard Memory (RAM) CPU NorthBridge AGP or PCI Express FrontSide Bus NorthBridge AGP or PCI Express PCI Slots IO Serial/Parallel Keyboard Mouse SouthBridge Hard Drive USB IDE/SATA

Why use Memory? Hard Drives are much larger… Memory is FAST A read can take around 0.00000001 seconds Hard drives are SLOW A read can take around 0.01 seconds It comes down to electrical (Memory) vs. mechanical (Hard Drives)

Windows Concepts The Windows OS is highly structured and has many concepts: Kernel and User modes Drivers Processes Threads Services Registry

Architecture Diagram Executive User Applications Windows API Services Base Kernel Device Drivers Environment Subsystems USER and GUI support System Support Processes HAL Kernel mode User mode

CPU Privilege Level Ring 0: Unrestricted Ring 1, 2: Not used in Windows Ring 3: Restricted Ring 0 Ring 1 Ring 2 Ring 3

Windows Kernel Windows Exectuive, which handles memory, process, thread, security, object, IO, and networking management Hardware Abstraction Layer (HAL) USER and GUI functionality Device Drivers, to provide extendable user and hardware IO

Windows Kernel Kernel components have unrestricted access to the entire system Dangerous! The kernel is Ring 0

What are Device Drivers? Dynamic, Loadable modules that run in kernel mode and can provide hardware IO support, and/or user IO translation. Again, as with all kernel components, unrestricted access to the system (Dangerous)!

Windows User mode Contains user applications Contains Support processes (logon) Contains Service processes Contains Environment subsystems Restricted access to the system (Ring 3) Must access system resources through the Windows API

What are Processes? Processes are containers for executing a program Private Virtual Memory space Unique Identifier called a Process ID (PID) At least one Thread of execution Security context

What is a Thread? A Thread is a container for execution CPU registers and state Stacks Private storage called TLS Unique Identifier called a Thread ID (TID or client ID) Security context

Services User mode programs that provide functionality independent of the current user For example: Task Scheduler Print Spooler Windows Update

Services.exe Svchost.exe Others (see VMWareService.exe)

Registry A system database that contains important information For example: Startup settings Hardware configurations Application configurations Current user data

Physical Memory vs. Virtual Memory Physical Memory refers to the hardware view of memory There is only one Virtual Memory refers to “virtualized” OS views of memory There can be many different virtual memory spaces

Memory Physical Memory Virtual Memory(s) Memory (RAM) Operating System

Why have Virtual Memory Can provide process memory isolation (security) Allows more “logical” memory by increasing the addressable space (Each process gets its own 4GB of virtual memory) When combined with paging, can increase the total available memory (more on this later)

Total Logical Memory Sum of all Virtual Memory Physical Memory 2 GB Memory (RAM) OS 4GB 4GB 4GB 4GB 4GB 4GB Virtual Memory 6 x 4GB = 24 GB of Logical Memory

How does 2GB become 24GB (or more)? The OS utilizes CPU features to create page directories and page tables which can be used to divide physical memory among multiple virtual memory spaces

Physical <-> Virtual 2 GB Virtual Memory for Process A Page Directories and Page Tables 0 GB 4 GB Physical Memory Virtual Memory for Process B 0 GB 4 GB Virtual Memory for Process C 0 GB 0 GB 4 GB

What happens when all of Physical Memory is used? Paging to the Hard Drive (SLOW!) Pagefile.sys

Virtual Memory for Process A 2 GB Virtual Memory for Process A Page Directories and Page Tables 0 GB 4 GB Physical Memory Virtual Memory for Process B 0 GB 4 GB 0 GB Virtual Memory for Process C 0 GB 4 GB Hard Drive

Paging to Disk When Physical Memory is getting full, the least used pages of memory are written to disk When those pages are needed again, they are read back into Physical Memory and some other pages are written to disk. This is called Swapping. Reduces system performance.

Memory Dump To get a complete collection of memory you need to collect two pieces: Physical Memory The on disk pagefile

But is it really complete? There is another feature of Windows Memory Management that may leave empty sections in a memory dump Unreferenced Memory

Unreferenced Memory When loading a binary from disk, the Windows Memory Manager may decide to only read portions of the binary into memory The unread portions of the binary are tracked We call them “unreferenced pages”

Why not read everything? Speed Reduction of actual memory usage Some binaries are very large but only a small section may be commonly used

Virtual Memory for Process A Page Directories and Page Tables 2 GB Virtual Memory for Process A Page Directories and Page Tables 0 GB 4 GB Physical Memory Virtual Memory for Process B 0 GB 4 GB 0 GB Hard Drive pagefile ANYFILE

Virtual Memory Allocation Programs can allocate virtual memory dynamically The size can range from a single byte to several GBs (or 8192 GBs in x64 OS versions)

How is this tracked? The Windows kernel uses a data structure known as Virtual Address Descriptors (VADs) to track virtual memory allocations Responder combines this information with Page Table data for each process and displays it in the Memory Map detail panel

Virtual Memory for Process A 2 GB Virtual Memory for Process A Page Directories and Page Tables 0 GB 4 GB Physical Memory VAD Tree 0x00C00000 – 0x00E00000 0x00CD0000 – 0x00CDF000 PTE 0010 - 0015 0x00CE0000 – 0x00E00000 0 GB 0x00CE0000 – 0x00CF0000 0x00D10000 – 0x00D20000 Hard Drive

Memory Block Block Length Individual Pages for this Block Some Unreferenced Pages

Virtual Memory Layout 4 GB The upper 2GB* of every Virtual Memory space is reserved for the Windows Kernel to use. It is not accessible to user mode processes. * Note: except with the rarely used /3GB switch Kernel Memory 2 GB User Memory 0 GB

User Virtual Memory 2 GB Process specific Windows system structures Windows System DLLs Windows and Application DLLs or Allocated Memory DLLs or Allocated Memory Application Binary Stack Heap or Allocated Memory 0 GB

Might be Heap Stack Application DLLs System DLLs

User mode How do user mode programs (which are restricted) access system resources? The Windows API

What is the Windows API? The core set of Application Programming Interfaces for the Windows Operating System Provides all the functionality required to create software on the Windows platform. http://msdn.microsoft.com/en-us/library/aa383749(VS.85).aspx

API Example

How does the API work? From a user mode program: A call is made to an API function The API function eventually calls an exported function of NTDLL that provides the needed ability The NTDLL function issues an interrupt (or SYSENTER instruction) to pass control to kernel mode

API - Kernel Mode The interrupt/SYSENTER handler, KiSystemService, is called by the CPU KiSystemService looks up and calls the requested service in the System Service Descriptor Table (SSDT) Note there is also a Service Descriptor Table Shadow that handles USER and GDI services

Windows API Application Calls WriteFile Kernel32.DLL WriteFile Calls NtWriteFile NTDLL.DLL NtWriteFile Issues a SYSENTER instruction User mode Kernel mode NtosKrnl.exe KiSystemService Looks up requested service in the System Service Descriptor Table (SSDT) SSDT The SSDT is a table of function pointers 0x84c0780 Perform a write Return to user mode caller NtosKrnl.exe NtWriteFile