ADM392

Windows® Server™ 2003 and Windows XP Kernel Changes

Mark RussinovichWinternals Software

David Solomon Expert Seminars

Outline

Overview

Performance

Scalability

64-bit support

File systems

Reliability and recovery

Miscellaneous

About The Speakers

Authors of:Inside Windows 2000, 3rd Edition(Microsoft Press)Inside Windows 2000/XP/2003 Interactive Internals Video Tutorial

Used by Microsoft for worldwide internal training

David Solomon:Teaches Windows internals classes (www.solsem.com)Writes books and articles on Windows internals

Mark Russinovich:Author of tools on www.sysinternals.comCo-founder and Chief Software Architect for Winternals Software (www.winternals.com)Teaches Windows internals classesWrites books and articles on Windows internals

Level Of Kernel ChangeWindows Server 2003 & Windows XP are modest upgrades as compared to the changes from Windows NT 4.0 to Windows 2000Kernel architecture is basically unchanged

No new subsystemsNo new API sets

Internal version numbers confirm thisWindows 2000 was 5.0Windows XP is 5.1 (not 6.0)Windows Server 2003 is 5.2

Not the same kernel as XP (a superset)

But, nonetheless, still lots of interesting kernel changes…

Outline

Overview

Performance

Scalability

64-bit support

File systems

Reliability and recovery

Miscellaneous

The Boot ProcessGoal: From power on to logon screen in under 30 seconds

Boot monitoring tool (Bootvis) developed to help Microsoft and hardware vendors optimize

Prefetching of drivers

I/O overlapped with device initialization

Slow drivers do work asynchronously

Winlogon doesn’t wait for Workstation service to start if

Account doesn't depend on a roaming profile

Domain policy that affects logon hasn't changed since last logon

Prefetch Mechanism

File activity is traced and used to prefetch data the next time

On boot, system monitors first 2 minutes of boot process (stops 30 seconds after the user starts the shell or 60 seconds after all services are started)

Also applies to application startupFirst 10 seconds are monitored

Prefetch “trace file” stored in \Window\PrefetchName of .EXE-<hash of full path>.pf

Boot trace: NTOSBOOT-B00DFAAD.pf

Prefetch Mechanism

When application run again, system automatically

Reads in directories referenced

Reads in code and file dataReads are asynchronous

But waits for all prefetch to complete

In addition, every 3 days, system automatically defrags files involved in each application startup!

Bottom line: Reduces disk head seeksThis was seen to be the major factor in slow application/system startup

Hibernate And Resume

Hibernation file is better compressed

I/O overlapped on IDE drives

Resume is fasterReads are larger

Device parallelization during power up improved

Power up done asynchronously in the background by drivers (specifically power-pagable devices without children)

Other Performance Improvements

Fast system callsUses SYSENTER/SYSEXIT on Pentium II or higher; SYSCALL on AMD

More intelligent working set trimming on MP systems

Pages removed are LRA (Least Recently Accessed)

In Windows 2000, was only done on uniprocessor systems

Outline

Overview

Performance

Scalability

64-bit Support

File systems

Reliability and Recovery

Miscellaneous

SMP Scalability

Scalability improvements made in several areas of the kernel

Some of these are in Windows XP

More are in Server 2003

Several areas:Increased physical memory support

Bigger multiprocessor systems

Improved synchronization

New types of multiprocessor systems

Increases in system virtual memory limits

Physical Memory Limits

32-bit Server 2003 Enterprise Edition supports 32 GB RAM

Windows 2000 Advanced Server limit was 8 GB

32-bit Server 2003 Datacenter Edition supports 128 GB

Windows 2000 Datacenter Server was 64 GB

64-bit Sever 2003 Datacenter supports 512GB (!)

Using Extended Physical Memory

On 32-bit Windows, virtual address space is still 4 GB, so how can you “use” > 4 GB of memory?

1. Although each process can only address 2 GB (or 3 GB), many may be in memory at the same time (e.g. 5 * 2 GB processes = 10 GB RAM used)

2. Files in system cache remain in physical memoryAlthough file cache doesn’t know it, memory manager keeps unmapped data in physical memory

3. Address Windowing Extensions allow Win32 processes to allocate more than 2 GB of memory

Map windows as needed

Large Pages

Large pages allow a single page directory entry to map a larger region

x86: 4 MBItanium: 16 MB

Large pages are used to map NTOSKRNL, HAL, boot drivers, and nonpaged pool if a “large memory system”

Windows 2000: 128 MB or moreWindows XP/2003: 256 MB or more

Advantage: improves performanceSingle TLB entry used to map larger areaNew in Server 2003: applications can VirtualAlloc large pages with MEM_LARGE_PAGE flag

Large Pages

Disadvantage: disables kernel write protectionWith small pages, OS/driver code pages are mapped as read only; with large pages, entire area must be mapped read/write

Drivers can then modify/corrupt system & driver code without immediately crashing system

Can override by changing HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management

LargePageMinimum REG_DWORD -1EnforceWriteProtection REG_DWORD 1

Larger Multiprocessor Systems64-bit Windows Server 2003, Datacenter edition supports 64 CPUs

SMP ScalabilityNew, more efficient locking mechanism (pushlocks)

Doesn’t use spinlocks when no contention

Used for object manager and address windowing extensions (AWE) related locks

Minimized lock contention for hot locksE.g., PFN (Page Frame Database) lock

Some locks completely eliminatedCharging nonpaged/paged pool quotas, allocating and mapping system page table entries, charging commitment of pages, allocating/mapping physical memory through AWE functions

Per-CPU Scheduling QueuesBefore, there was one system-wide list of threads that want to run

System had to lock this database to decide which thread to run next

Now, each CPU has its own list of threads that want to runThreads always go into the ready queue of their ideal processorInstead of locking the dispatcher database to look for a candidate to run, per-CPU ready queue is checked first

If there is one, does context swapElse scans other CPU’s ready queues looking for a thread to run

This scan is done OUTSIDE the dispatcher lockJust acquires per-CPU scheduling database lock

Global dispatcher lock still acquired to wait or unwait a thread and/or change state of a dispatcher objectBottom line: dispatcher lock is now held for a MUCH shorter time

Hyperthreading

Support for logical processors on hyperthreaded Xeon & Pentium 4 processors

Does not count logical processors against CPU license limit like Windows 2000

E.g., Windows Server 2003 Enterprise Edition will use 16 logical processors on an 8 way hyperthreaded Xeon system

Windows 2000 Advanced Server would only use 8

Scheduling algorithms take into account logical vs physical processors

Used in choosing idle CPU to run a thread

NUMANUMA (non uniform memory architecture) systems

Groups of physical processors (called “nodes”) that have local memory

Connected to the larger system through a cache-coherent interconnect bus

Still an SMP system (e.g. any processor can access all of memory)

But node-local memory is faster

Scheduling algorithms take this into accountTries to schedule threads on processors within the same node

Tries to allocate memory from local memory for processes with threads on the node

New Win32 APIs to allow applications to optimize

System Virtual Memory LimitsKey system memory limits raised in XP & Server 2003

Windows 2000 limit of 200 GB of mapped file data eliminated

Previously limited size of files that could be backed up

Maximum System Page Table Entries (PTEs) increasedCan now describe 1.3 GB of system space (960 MB contiguous)

Windows 2000 limit was 660 MB (220 MB contiguous)

Increases number of users on Terminal Servers

Also means maximum device driver size is now 960 MB (was 220 MB)

Registry LimitsSYSTEM hive was limited to 12MB in Windows 2000

Now limited to 200 MB or ¼ of RAM, whichever is lower

Total loaded registry hive data was limited to 376MB in Windows 2000

Limited number of terminal server usersThis was because registry hives were read into paged pool when loaded

Explains why there was a system registry quota

XP/2003: No limit to loaded registry hive dataRegistry no longer in paged poolHives are accessed as memory mapped files

Views are mapped as necessary

Outline

Overview

Performance

Scalability

64-bit support

File systems

Reliability and recovery

Miscellaneous

a

Windows 64-Bit Editions

Supports 64-bit Itanium Intel architecture64-bit Edition 2003 will support AMD Opteron and Athlon 64

ProductsWindows XP Professional 64-bit editionWindows Server 2003 64-bit editions

True 64-bit versions (e.g. pointers are 64-bits)

Much larger address spaceGood for CAD, simulation, other memory-intensive applications

Not a performance boost in and of itself

Itanium Address Space LayoutUser-Mode User SpaceUser-Mode User Space

Kernel-Mode User SpaceKernel-Mode User Space

1FFFFF00000000001FFFFF0000000000 User Page TablesUser Page Tables

Session SpaceSession Space

Session Space Page TablesSession Space Page Tables

System SpaceSystem Space

6FC000000006FC00000000

20000000000000002000000000000000

3FFFFF00000000003FFFFF0000000000

E000000000000000E000000000000000-E000060000000000-E000060000000000

FFFFFF0000000000FFFFFF0000000000 Session Space Page TablesSession Space Page Tables

00

64-bit Windows64-bit Windows 32-bit Windows32-bit Windows

User Address SpaceUser Address Space 7152 GB (6.9 TB)7152 GB (6.9 TB) 2 or 3 GB2 or 3 GBSystem PTEsSystem PTEs 128 GB128 GB 1.3 GB1.3 GBSystem cacheSystem cache 1024 GB (1 TB)1024 GB (1 TB) 960 MB960 MBPaged poolPaged pool 128 GB128 GB 470 MB470 MBNon-paged poolNon-paged pool 128 GB128 GB 256 MB256 MBPage file sizePage file size 32 TB32 TB 16 TB16 TB

32-Bit Application Support

“Wow64” - allows execution of Win32 32-bit applications on 64-bit OS

Wow64.dll - provides core emulation infrastructure and thunks for Ntoskrnl.exe entry-point functions

Loads the x86 version of Ntdll.dll and runs its initialization code, which loads all necessary 32-bit DLLs

32-bit Kernel32.dll, ntdll.dll, etc., are loaded from %systemroot%\SysWOW64

Wow64win.dll - provides thunks for Win32k.sys entry-point functions

Wow64cpu.dll - provides x86 instruction emulation; executes mode-switch instructions on Itanium

Wow64

Some advanced Win32 APIs not supported (e.g. scatter/gather I/o)Interoperability

COM, cut/paste interoperateCannot load 32-bit DLLs in 64-bit process and vice versa

On Itanium, slower execution than on native 32-bit machineImages marked large address space aware get a full 4 GB process virtual address space

OS isn’t mapped there, so space is available for process

Win64 Disk Partitioning

Win64 boot.ini is in non-volatile RAMExtensible Firmware Interface (EFI)

First partition is FAT

GUID Partition Table (GPT)64-bit only

Overcomes limitations of MBR partitioning64-bit offsets and lengths

Partition table is mirrored

No nesting

Outline

Overview

Performance

Scalability

64-bit support

File systems

Reliability and recovery

Miscellaneous

File System Enhancements

FAT32 on DVD-RAM

Read-only NTFS volumes

UDF 2.01 (new standard for DVD-ROM, DVD-RAM, DVD-RW, DVD video)

Encrypting File System (EFS)No longer a separate driver—integrated into NTFS

Supports multi-user access to encrypted files (supports file sharing)

The Defrag API

Completely rewritten API

Can defrag MFT and other metadata files (except log file, paging file)

Can defrag encrypted files

No 4KB-cluster limit on NTFS

Command line interface (scriptable)

Volume Shadow Copy

Volumes can be “snapshotted”

Allows “hot backup” (including open files)

Uses copy on writeChanges to volume after snapshot cause original contents of cluster to be stored in snapshot file

Later, reads to changed data return contents at time of snapshot

Applications can tie in with mechanism to ensure consistent snapshots

Volume Snapshots

Volume Shadow Volume Shadow Copy DriverCopy Driver

(volsnap.sys)(volsnap.sys)Mirror providerMirror provider

OracleOracle

SQLSQLVolume ShadowVolume Shadow

Copy ServiceCopy Service

Backup Backup ApplicationApplication

1.1. Backup Backup application application requests requests shadow copyshadow copy

2. Writers told 2. Writers told to freeze to freeze activityactivity

3. Providers asked to 3. Providers asked to create volume shadow create volume shadow copiescopies

4. Writers told 4. Writers told to resume to resume (“thaw”) (“thaw”) activityactivity

WritersWriters

ProvidersProviders

5. Backup application5. Backup applicationsaves data from volume saves data from volume Shadow copiesShadow copies

Shadow Copies of Shared Folders

When enabled, 2003 Server uses shadow copy to periodically create snapshots of volumes

Schedule and space used is configurable

Shadow Copies on Shared Folders

Shadow copies only exposed as network sharesClients install Explorer extension that integrates with server that let’s them

View the state of folders and files within a snapshotRollback individual folders and files to a snapshot

Outline

Overview

Performance

Scalability

64-bit support

File systems

Reliability and recovery

Miscellaneous

System RestoreRollback system to previous state:

Registry, COM+ registration database, user profiles, other files not protected by WFP

Windows XP only (not on Server)

Replacement of certain file types causes original version to be stored in a restore point folder

569 file types monitored – see Platform SDK for list

Restore operation replaces these files

Implemented as a service and a filter driver

System Restore

File System Driver (NTFS/FAT)File System Driver (NTFS/FAT)

System Restore FilterSystem Restore Filter

ApplicationsApplications

File system request

Change.log1Change.log1

A0009653.exeA0009653.exe

A0009654.iniA0009654.ini

\System Volume Information\\System Volume Information\_restore{XX-XXX-XXX }\_restore{XX-XXX-XXX }\RP5RP5

User modeUser modeKernel modeKernel mode

System Restore

Restore Points are createdEvery 24 hours

When installing an unsigned driver

When explicitly requested by user or an install program (via an API or script)

WMI interfaces allow scriptable controlCreate/delete restore points, change configuration

Driver RollbackSystem saves updated driver in \Windows\System32\ReinstallBackups\nnnn\DriverFiles

New button on device properties to roll back driver

If you choose roll back, also saves a copy in \Windows\LastGood \System32\Drivers

Will then automatically roll back driver when booting from “last known good”

Driver Verifier Enhancements

New verification options:DMA verification – detects improper use of DMA buffers, adapters, and map registersDeadlock detection – detects lock hierarchy violations with spinlocks, mutexes, fast mutexesSCSI verification - monitors the interaction between a SCSI miniport driver and the port driverEnhanced I/O Verification tests drivers' support for power management, WMI, and filters

Simpler wizard-style GUI (verifier.exe)Defaults verify unsigned drivers

Side-By-Side Assemblies

Microsoft wants to end DLL hell by letting applications specify DLLs they use by version

Support multiple versions simultaneously installed

Application will use updates only if backward compatible

Application that uses assemblies has a manifest fileXML file that specifies application version number and DLLs

DLLs are identified by GUIDs and version number and are stored either in the application’s directory or in SystemRoot\Winsxs

Theme-Aware Common Controls

Example: Windows XP Common Control DLL (comctrl32.dll)

Windows XP version is 6, which supports Luna themes

Windows 2000 version is 5, which doesn’t support themes

Non-theme aware applications can behave incorrectly if used with v6 controls,

If an application doesn’t have a manifest that specifies v6, it gets v5, which is in the SystemRoot\System32 directory

Outline

Overview

Performance

Scalability

64-bit support

File systems

Reliability and recovery

Miscellaneous

Miscellaneous

Boot and execute from ROMOS and drivers copied to RAM

Applications can execute from ROM

Hot plug memory

Hot plug PCI

Headless server support (no keyboard, video, mouse)

Remote Installation Service

EMS (Emergency Management Service) allows remote disaster recovery/control via serial port or network

Terminal Services

Terminal Services included with Windows XP supports multiple sessions

Home Edition: Supports “disconnect and switch users”

Professional: Remote Desktop ConnectionRemote desktop redirection for audio, serial/parallel port, file system (local drives)

Server 2003: Load balancing support, remote audio, local drive & printer mapping

Services InfrastructureMore services run in generic service host process (svchost.exe)

Reduces number of processes

Two new less privileged accounts for built-in servicesLOCAL SERVICE, NETWORK SERVICE

Less rights than SYSTEMReduces possibility of damage if system compromised

Four instances of Svchost (at least)SYSTEM

SYSTEM (2nd instance – for RPC)

LOCAL SERVICE

NETWORK SERVICE

Debugging

Can now detach debugger without killing debuggee

See new Win32 DebugActiveProcessStop

Kernel debuggingLive local system kernel debugging (kd –kl or windbg –kl)

Kernel debugging over 1394 (in addition to serial)

Auto load of updated drivers to target

Registry Callbacks

Up until now Regmon has relied on system call “hooking” to intercept Registry accesses

Hooking isn’t supported by the kernel

As of XP the system call table is write-protected by default if a system has < 256 MB, requiring a trick

Server 2003 introduces a Registry callback mechanism

Driver can see and modify Registry behavior

Latest version of Regmon comes with two drivers: one for Server 2003 and one for previous versions

System Area NetworksSystem Area Networks (SAN) is a connection-oriented server interconnect

Not to be confused with Storage Area Networks (SAN)Provides reliable, in-order delivery

Both network and bus semantics:MessagesRemote DMA (memory semantics)

Segmentation/reassembly in hardwareInterconnect types include

InfiniBandEthernetFiberChannelProprietaryEven shared memory

System Area NetworksData Center

Web Tier Front End(Web Servers) Business Logic

DatabaseBackend

High-Speed SAN Fabric

Internet Traffic viaStandard WAN

System Area Networks

WinSock Direct (WSD) allows applications to get performance benefits of SANs

No application modification needed

Provides third generation task offload

System Area Networks

Socket App

Winsock

TCP/IP WinSock

Provider

TCP/IP Transport

Driver

NDIS

Miniport

NIC

Socket App

Winsock

TCP/IP WinSock

Provider

TCP/IP Transport

Driver

SAN NDIS

Miniport

Winsock Switch

SAN Winsock

Provider

SAN

Proxy

Driver

User Mode

Kernel Mode

NDIS

WinSock

SPI

Traditional Model Winsock Direct Model

SAN Hardware

Private interface

Summary

Server 2003 & XP represent a modest evolution of the NT kernel

More reliable, more secure, and much more scalable than Windows 2000

Upgrade today!!

For More Information

December 2001 MSDN Magazine articleKernel Improvements Create a More Robust, “Windows Powerful, and Scalable OS”http://msdn.microsoft.com/msdnmag/issues/01/12/XPKernel/XPKernel.asp

XP/2003 update to our internals video

4th edition of our bookTo be called “Windows Internals”

Will cover Windows 2000, XP, and Server 2003

To be available end of 2003

Community Resources

Community Resourceshttp://www.microsoft.com/communities/default.mspx

Most Valuable Professional (MVP)http://www.mvp.support.microsoft.com/

NewsgroupsConverse online with Microsoft Newsgroups, including Worldwidehttp://www.microsoft.com/communities/newsgroups/default.mspx

User GroupsMeet and learn with your peershttp://www.microsoft.com/communities/usergroups/default.mspx

evaluationsevaluations

© 2003 Microsoft Corporation. All rights reserved.© 2003 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.