Windows Operating System Internals - by David A. Solomon and Mark E. Russinovich with Andreas Polze Unit OS12: Scripting The Registry

2 Copyright Notice © David A. Solomon and Mark Russinovich These materials are part of the Windows Operating System Internals Curriculum Development Kit, developed by David A. Solomon and Mark E. Russinovich with Andreas Polze Microsoft has licensed these materials from David Solomon Expert Seminars, Inc. for distribution to academic organizations solely for use in academic environments (and not for commercial use)

3 Roadmap for Section Registry Fundamentals Registry Structure Registry Limits Monitoring Registry Activity with Regmon

4 Registry Fundamentals Repository for system/user configuration information Contains information NT requires to boot & configure System-wide application settings (e.g. install directory) Per-user settings (e.g. fonts, window placement, etc.) Called Configuration Manager (CM) in NTOSKRNL Network accessible (via Remote Registry service) Provides limited remote configuration, but not management Changing some registry entries causes action, but only if something watching for changes WMI provides remote management (described in next section) Also is a window into in-memory (volatile) system information Hardware description (built during boot process) Performance data

5 Registry Interface to Performance Counters Custom application A Custom application B Performance tool Windows Management Instrumentation High-performance provider interface Registry DLL provider Pdh.dll Advapi32.dll RegQueryValueEx PerfLib System performance DLL Performance extension DLL High-performance data provider object Performance- monitoring applications Programming Interfaces

6 Viewing and Changing the Registry Should never have to do this! Most common Registry parameters can be adjusted using a graphical NT utility But, some advanced tuning and configuration options require modifying and/or adding registry data Registry Editor tools Windows NT4/2000 had 2 tools (Regedt32.exe and Regedit.exe) REGEDT32.EXE Interprets NT-specific data types, NT permission lists REGEDIT.EXE Doesn’t understand NT-specific data types or security But has more thorough find capability XP/2003: single combined Regedit.exe does everything No shortcuts for these tools

7 Command Line Registry Tools Reg.exe (built into XP/2003; also in 2000 Support Tools) Can compare keys, load/unload hives, add/remove/query keys, and backup/restore (useful since NT Backup can only backup local registry hives) Regini.exe (built in to XP/2003; also in 2000 Resource Kit) - make registry changes via a script (.INI file) Regdmp (2000 Resource Kit) – dump registry in format that Regini reads Scanreg.exe (2000 Resource Kit) - flexible registry search tool

8 Registry Usage When is the Registry read? During boot (to determine which drivers, configure the system) During login (to set user preferences, such as drive letter mappings, screen saver, wallpaper, etc.) When you start applications (to read app-specific settings and per- user settings) When is the Registry modified? When you install applications or drivers When you change system, user or application settings It should not be polled for changes Interesting to check your systems for “quietness” of Registry Should use Registry change notification Windows functions Thread sleeps until a change is made to the part of the Registry they are interested in

9 Registry Structure Registry editors display Registry as an Explorer-like tree view Lab: run Regedit and examine The Registry is structured like a file system: Keys are like directories Values are like files Values have data types – like a file type Data is like the contents of a file The most common value types (there are 12 total): REG_DWORD - 32-bit integer REG_SZ - string REG_MULTISZ – array of strings REG_BINARY – array of bytes to represent arbitrary data Registry keys have full NT security Allows fine grained control as to who has what access

Registry Documentation Latest Registry documentation is in the Windows Server 2003 Deployment Kit t.mspx If not there, search Microsoft.com for articles on specific registry keys or values

11 Logical Registry Structure Five visible “root” keys organize the data Really only two “roots”: HKLM and HKU 1. HKEY_CLASSES_ROOT (HKCR) Link to HKLM\Software\Classes - file extension associations, COM class registrations HKEY_LOCAL_MACHINE (HKLM) System-global static and dynamic configuration HKEY_USERS (HKU) Root for the loaded (logged on) user profiles HKEY_CURRENT_USER (HKCU) Link to profile of the user accessing this key HKEY_CURRENT_CONFIG (HKCC) Link to subset of HKEY_LOCAL_MACHINE - has hardware profile information Hidden virtual key for performance data HKEY_PERFORMANCE_DATA

12 HKEY_LOCAL_MACHINE Five main subkeys for local machine information HKEY_LOCAL_MACHINE\Hardware Hardware configuration data, resource usage, etc. Completely volatile (not saved across boots) HKEY_LOCAL_MACHINE\SAM Local account & groups database (for NT4 domain controllers, served as the domain account database) HKEY_LOCAL_MACHINE\Security System-wide security policies (for NT4 domain controllers, served as the domain security policy database) HKEY_LOCAL_MACHINE\Software Per-machine software data not critical for booting HKEY_LOCAL_MACHINE\System Controls booting and running the system (config.sys functionality) Available during bootstrap

13 HKEY_USERS HKEY_USERS is root for loaded user profile Registry data This is stored in \Documents and Settings\%USERNAME%\ntuser.dat In NT 4 it was stored in \Winnt\Profiles\%USERNAME% Note this is just a small part of your user profile, which includes your desktop, start menu, My Documents, etc. A user profile is “loaded”: When a user logs on interactively When a service process configured to run under a specific user account starts RUNAS (unless /NOPROFILE specified)

14 User Profiles HKEY_USERS\.Default is NOT the default Registry profile It is the profile for the System account E.g. Change logon desktop settings are here because Winlogon runs under System The default user Registry profile is in: \Documents and Settings\Default User\Ntuser.dat The starting user profile for users logging without a profile This does not show up in the Registry because it is not loaded Note: roaming profiles are left on a system even after you logoff! Anyone with local administrative privileges can view your documents and settings and Registry information Can override with group policies

15 Registry Physical Structure Registry consists of a number of files called “hives” A hive is a piece of registry database stored in its own file System-wide configuration information is in %systemroot%\system32\config User profile (HKEY_CURRENT_USER\...) in \Documents and Settings\…\ntuser.dat User profile is more than just the registry settings Entire directory under \Documents and Settings (e.g. start menu, My documents, temporary files, etc) Profile for new users is in \Documents and Settings\Default User NTLDR loads System hive, which points to other hives Registry points to loaded hives at: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\hivelist Can defrag registry hives with Pagedfrg from Sysinternals

16 Backup Hives.sav hive files in \Windows\System32\Config are the template hive files stored by Windows Setup \Windows\Repair has copies of the hives that were made after Setup completed Windows Backup updates them when it backs up “System State” System Restore restore points (described in the startup/shutdown section) also store copies of registry hives

17 Registry Limits SYSTEM hive was limited to 12MB in Windows 2000 Now limited to 200 MB or # of RAM, whichever is lower Limit is due to fact that entire SYSTEM hive is loaded into memory by NTLDR (discussed in Startup and Shutdown section) Total loaded registry hive data was limited to 376MB in Windows 2000 This was because registry hives were read into paged pool when loaded Explains why there was a system registry quota (see Control Panel- >System->Performance Tab - click on Virtual Memory “Change” button) Limited number of terminal server users XP/2003: No limit to loaded registry hive data Hives are accessed as memory mapped files Views are mapped as necessary Registry no longer in paged pool System registry quota is gone

18 Registry Reliability & Recovery Registry mechanism guarantees atomicity Uses transaction logging to ensure structural integrity One.LOG file per registry hive Automatically used to recover corrupt registry hives other than SYSTEM SYSTEM hive corruption recovery: Prior to Windows XP the system keeps a backup version of System called System.Alt NtLdr uses System.Alt of System is corrupt On Windows XP, Ntldr knows how to apply the registry log files to the System hive in order to repair it in cases where the system crashed during updates On Windows Server 2003 the configuration manager “self-heals” by amputating parts of the registry that are corrupt

19 Vista Transactions The Vista Registry supports transactions Allows application file system and Registry changes to be undone Changes committed only when associated transaction commits Relies on Distributed Transaction Coordinator (DTC) on Vista Server to enable multi-node transactions Transaction API is with NTFS (TxF) and the Registry Documented in Vista Beta 1 SDK CreateTransaction, SetCurrentTransaction, CommitTransaction, RollbackTransaction

20 Configuration Problems Missing, corrupted or overly-secure Registry settings often lead to application crashes and errors Some applications don’t completely remove registry data at uninstall Regmon may yield the answer…

21 How Regmon Works Regmon uses a driver to intercept Registry operations Up until now Regmon has relied on system call “hooking” to intercept Registry accesses Hooking isn’t supported by the kernel As of XP the system call table is write- protected by default if a system has <256 MB, requiring a trick Server 2003 introduces a Registry callback mechanism Driver can see and modify Registry behavior Latest version of Regmon comes with two drivers: one for Server 2003 and one for previous versions Defined in a DDK header file and used by antivirus products Application RegmonDriver RegmonGUI RegistrySubsystem

22 How Regmon Works Regmon catches all Registry calls that pass through the system service dispatcher Does not see direct calls to the native Registry API from kernel-mode code Regmon has to dynamically obtain system call numbers to index into the system service dispatch table System call numbers are assigned by scripts in the build process and can change between service packs All system call functions start with code like this: MOVE SYSCALL # -> REGISTER EXECUTE SYSTEM CALL Regmon reads the system call numbers out of the functions

23 Regmon Output Request: OpenKey, CreateKey, SetValue, QueryValue, CloseKey Path HKCU=HKEY_CURRENT_USER (per-user settings) HKLM=HKEY_LOCAL_MACHINE (system wide settings) Result – return code from Registry operation Other – extended information or results

24 Controlling Regmon Start/stop logging (Control/E) Clear display (Control/X) Regedit jump (opens Registry Editor and jumps directly to key) Double clicking on a line does this Filtering/HighlightingFind Save to log file

25 Regmon Filtering Filtering options: Process name or registry path (or partial name) Success/failure, read/write

26 Troubleshooting with Regmon Two basic techniques: Go to end of log and look backwards to where problem occurred or is evident and focused on the last things done Compare a good log with a bad log Often comparing the Registry activity of a failing process with one that works may point to the problem Have to first massage log file to remove data that differs run to run Delete first 3 columns (they are always different: line #, time, process id) Easy to do with Excel by deleting columns Then compare with FC (built in tool) or Windiff (Resource Kit)

27 Troubleshooting with Regmon Sometimes examining the values read and/or written (in Other column) may show up corrupt or wrong configuration data Sometimes queries to what is not there is more interesting than what is there Identify missing Registry keys Search for status “NOTFOUND” For permission problems, Search for status “ACCESS DENIED ” Note: BUFROVRFLOW is normal Applications query a value’s size by passing a 0-length buffer They are returned the length of the data and the overflow error and then they allocate a correctly-sized buffer and repeat the operation

28 Troubleshooting with Regmon If you suspect registry data is causing problems, rename the key and re-run the application Most applications re-create user settings when run In this way, the data won’t be seen by the application Can always rename the key back

29 Regmon Example: Misleading IE Error Message Internet Explorer failed to start with this error:

30 Solution: Misleading IE Error Message Looked backwards from end of Regmon log Saw query of Completed value in Internet Connection Wizard key Value read was 0 Value was 1 on other systems Solution: Set value to 1 and problem went away

31 Example Problem Problem: User somehow disabled all toolbars and menus in Word No way to open files, change settings etc. Solution: With Regmon, captured startup of Word Found location of user-specific settings for Word Deleted this Registry key Re-ran Word, which re-created user settings from scratch

32 Example Problem Internet Explorer hung when started Default internet connection was set, but wasn’t being dialed Dialing the connection first manually and then running IE worked

33 Solution Ran Regmon Looked backwards from end (at the point IE was hung) Found references to ATT under a PhoneBook key Renamed ATT key and problem went away Background information: User had previously installed the AT&T Dialer program, but had uninstalled it and created dial up connection manually Conclusion: registry junk was left from uninstall Note that uninstalls don’t clean up stuff in currently not loaded user profiles!

34 Example: Misleading VBA Error User got this error installing an application: Regmon showed permissions problem Solution: Edited permissions

35 Missing Settings Sometimes queries to what is not there is more interesting than what is there Identify missing Registry keys Search for status “NOTFOUND” May reveal hidden capabilities

36 Example: Hidden Capabilities User tried installing Compaq utility on non-Compaq hardware Regmon showed hidden key to override: Solution: created key and install succeeded

37 Logging Boot Registry Activity Regmon has a “log boot” option Installs its driver to logs Registry activity from boot time to \Windows\Regmon.log Stops logging when you run Regmon Regmon loads as very first driver so it captures virtually everything Remember that it does not see direct calls to the native Registry API from kernel-mode code Lab: Enable boot logging and search resulting log for Winlogon debug Registry values

38 Logging Activity in Unprivileged Accounts or During Logon/Logoff May need to capture Registry activity in an unprivileged account e.g. An application may work when run as Administrator, but not as a normal user Regmon requires the DEBUG and LOAD_DRIVER privileges to execute Or, may need to capture Registry activity during the logon or logoff process E.g. An Explorer shell extension is crashing Explorer during logon In these cases Regmon needs to remain running after logoff Only way to have a process not be destroyed at log off is to have it run as a service or as a child of a service process 1. For a quick, one-time execution, run with “psexec -i –s -d” command 2. Install and run Regmon as a service using Srvany tool in Resource Kit Can configure to start at system boot When you logon Regmon will still be running

39 Further Reading Mark E. Russinovich and David A. Solomon, Microsoft Windows Internals, 4th Edition, Microsoft Press, Chapter 4 - Management Mechanisms; The Registry (from pp. 183)