Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
1
Academic Days 2005
Windows OS Course Supplement Kit:
What should be there?
Arkady RetikMicrosoft
2
Presentation Plan
1. Core OS education: is there a problem?
2. What should our goals be? 3. Is our current thinking correct?4. What’s next?
3
What is the problem?
Windows specific:• Use of Windows OS in Core OS courses is low
• Is this a Microsoft only problem?• Why isn’t Linux a solution?
--------------------------------------------------------------------------
Operating System courses in general:• CS intake overall is shrinking (USA, UK, ...)• In particular, OS courses are ‘not attractive’
4Why is Windows underutilized?
• No Windows source code (Almost: Windows Source Code is available for ~ 150 Universities WW under Shared So urce )
• Current license ‘discourages’ teaching and publication
• Insufficient environment to run software out of the box
• No Windows based instructional system (such as Nachos, MINIX)
• Few Windows OS oriented labs & exercise materials
• Inadequate and outdated textbook coverage of Windows
• No academic oriented reference books of Windows
• Limited academic research publications using Windows
• Windows source footprint is large (~ 50% large than Linux)
• ABM
5
Are students interested in OS classes?Are students interested in OS classes?Are students interested in OS classes?Are students interested in OS classes?
• Most of the CS programs offer a core OS class Most of the CS programs offer a core OS class Most of the CS programs offer a core OS class Most of the CS programs offer a core OS class
during 2during 2during 2during 2ndndndnd or 3or 3or 3or 3rdrdrdrd year of studyyear of studyyear of studyyear of study
• About 90% will not take further OS classesAbout 90% will not take further OS classesAbout 90% will not take further OS classesAbout 90% will not take further OS classes
6
Program Goal, Objectives and ScopeOur goal is to develop and deliver a program for co mputer science and engineering undergraduate and graduate courses that provides modern and up to date OS knowledge by leveraging Microsoft soft ware and products
The program’s top level objectives are:• Offer Windows based teaching and learning, tools an d materials• Promote presence of Windows in OS courses world-wide• Increase knowledge of Windows Internals among stude nts and faculty• Demonstrate the innovations underlying the Windows solutions• Provide infrastructure for the future academic rese arch • Support reproducibility requirements for OS researc h publications e.g.
OSDI, SOSP • Complement other MS Academic initiatives (Shared Pre mium Source,
Rotor, WinCE)
Scope: CS OS courses a. Undergrad coursesb. Graduate coursesc. Research and Publications
7
Our current thinking
Program components:1. Limited Source – Alternative to existing Shared Source Premium
a. Simplify current license b. Add tools out of DDK that academics can use to rebuild the kernelc. Textbook: Internals book acceptable to CS faculty
2. OS Project Course (Project OZ) – New offeringa. Undergrad projects based on NT Kernel APIs b. Uses NT subsystem model, with licensed wrapper for NTAPIs (no
Windows sources)c. C/C++, C#, and maybe Java supportd. Companion lab project book will supplement OS textbook
3. Curriculum Development Kit (CDK) – New offeringa. Aimed at teaching OS coursesb. Provides ACM/IEEE Core OS curriculum content
8
Background
Connecting NT and Shared Source:
Brief History and Architecture overview
9
OS E
volu
tio
n
1970
1980
1990
2000
VMS v1.0
Windows NT3.1
Windows NT4.0
Windows XP
Windows Server 2003
Unix
Unix Public
Unix v6
Linux v1.0
Linux v2.0
Linux v2.6
Linux v2.4
10
History of NT
Team forms November 1988
Developers from DEC and Microsoft
Build from the ground up
• Advanced PC Operating System
• Designed for for desktops and servers
• Secure, scalable SMP design
• All new code
Initial effort targeted at Intel i860 code-named N1 0, hence the name NT which doubled as N-Ten and New Technology
11
NT Timeline first 15 years
2/1989 Coding Begins
7/1993 NT 3.1
9/1994 NT 3.5
5/1995 NT 3.51
7/1996 NT 4.0
12/1999 NT 5.0 Windows 2000
8/2001 NT 5.1 Windows XP
3/2003 NT 5.2 Server 2003
4/2004 NT 5.2 Windows XP 64 Bit Edition
12
Goal Setting
High level goals:
• Portability – Ability to target more than one processor, avoid assembler, abstract away machine dependencies.
• Reliability – Nothing should be able to crash the OS. Anything that crashes the OS is a bug.
• Extensibility – Ability to extend the OS over time
• Compatibility – With DOS, OS/2, POSIX, or other popular runtimes.
• Performance – All of the above are more important than raw speed!
13
NT Architecture
14
Overview of Windows Architecture
NT is not a microkernel, but does support user-mode OS personalities (i.e. for posix, OS/2, Win32)
Primary supported programming interface: Win32
Win32 and other subsystems built on native NT APIs
NT APIs generally not documented (not intended as t he supported programming model) – but specific APIs are documente d in the DDK
Kernel implementation organized around the object manager
NT APIs are rich (many parameters) and need re-fact oring and simplification for student use
15
Windows Architecture
User-mode
Kernel-mode Trap interface / LPC
ntdll / run-time library
Win32 GUIProcs & threads
Kernel run-time / Hardware Adaptation Layer
Virtual memoryIO ManagerSecurity refmon
Cache mgr
File filters
File systems
Volume mgrs
Device stacks
Scheduler
Kernel32 User32 / GDI
DLLs
Applications
System Services
Object Manager / Configuration Management
FS run-time
exec synchr
Subsystemservers
Login/GINA
Critical services
16
Windows Kernel Organization
Kernel-mode organized into
NTOS (kernel-mode services)– Run-time Library, Scheduling, Executive services,
object manager, services for I/O, memory, processes, …
HAL (hardware-adaptation layer)– Insulates NTOS & drivers from hardware
dependencies– Providers facilities, such as device access, timers,
interrupt servicing, clocks, spinlocksDrivers
– kernel extensions (primarily for device access)
17
Major Kernel Services
Process managementProcess/thread creation
Security reference monitorAccess checks, token management
Memory managerPagefaults, virtual address, physical frame, and pagefile
managementServices for sharing, copy-on-write, mapped files, GC support,
large appsLightweight Procedure Call (LPC)
Native transport for RPC and user-mode system services. I/O manager (& plug-and-play & power)
Maps user requests into IRP requests, configures/manages I/O devices, implements services for drivers
Cache managerProvides file-based caching for buffer file system I/OBuilt over the memory manager
Scheduler (aka ‘kernel’)Schedules thread execution on each processor
18
OS Project Course(code name Project OZ)
(see examples of the projects in the Appendix)
19
The Project OZ Course
Objectives
• Provide an environment to build a rich set of projects that explore OS principles by leveraging the NT subsystem model for implementing OS personalities
• Use real OS features rather than a ‘toy’ simulation• Reduce the complexity required to learn / build
experiments
• A simple development environment, using standard tools for building, debugging, and instrumentation
• Encourage ‘out-of-the-box’ thinking by students
20
The Project OZ Course
OZ OverviewLibrary of functions that wrap the native NT APIs to
provide access to low-level primitives to provide address spaces, threads, exceptions, and IPC
Languages: C/C++, C#, Java A runtime of support functions that simplify student
projectsDocumentation for the OZ functions/runtimeA rich set of projects, with many variations, that allow
students to explore qualitatively (& quantitatively) a large assortment of OS principles
Tools for instrumentation and measurement
21
CDK
22What will CDK cover?CDK modules
1. cover all OS topics (based on Windows XP/Server 2003) 2. scaleable to multiple levels3. modular (can be used in whole / in part).
� Basic Module will provide materials to incorporate into a complete basic level OS course of one semester in length. The module will cover the Windows OS specific topics in the core and elective units of the OS BOK of Computing Curricula 2001.
� Advanced Module will provide materials to incorporate into an advanced level OS course of one semester in length. The module will cover the Windows OS specific topics in the core and elective units of the “CC2001” OS BOK as well as relevant Networking and other units.
All curriculum materials (syllabi, course outlines, lecture notes, labs, exercises, etc.) will be available in the MSR Curriculum Repository.
23
What OS Body of Knowledge topics will CDK cover?
a. Core topics OS1. Overview of operating systems OS2. Operating system principles OS3. Concurrency OS4. Scheduling and dispatch OS5. Memory management
b. Elective topicsOS6. Device managementOS7. Security and protectionOS8. File systemsOS9. Real-time and embedded systemsOS10. Fault toleranceOS11. System performance evaluationOS12. Scripting
c. Advanced topicsA13. Windows networkingA14. Comparing the Linux and Windows Kernels A15. Windows – Unix Interoperability
d. Labs and Exercises to reinforce the topics
Anything
else?
24What will CDK consist of?
• Instructor’s material:1. An exemplar course syllabus and outline.2. Classroom materials (i.e. lecture slides, notes and other
supporting materials). 3. Lab exercises, assignments and testing materials with notes,
manuals and instructions. 4. Software tools referenced in lecture materials or used in labs or
exercises. 5. Copy of Windows Internals 4th edition MS Press book.
• Student material:Any material that should be delivered to student by professor (i.e. lab assignments, lab set up and descriptions, tools use instructions) will be part of Instructor’s material as stand alone ‘redist’ packages. Windows Internals 4th edition will be a recommended textbook for the course.
25
How do we achieve best content and top quality?
• Engage the best available experts
• Tailor to the academic style and requirements
• Conform to the latest ACM/IEEE Computing Curricula – de facto educational standard
• Capitalize on 4th Edition Windows Internals book - best Windows reference material
• Build on rich experience of Microsoft OS training and experienced OS faculty
• Validate by independent and randomly selected academic reviewers: WW pilot.
26
What’s next• Our current plans to provide initial input for next
academic year (see Appendix for specific topics):
• Core topics will be available early July 2005
• Elective topics will be available in Fall 2005
• Will be looking for participants in pilots and tria ls
• If you are interested - let your academic contact a nd me know:
� Leave your business card /contact details
• More information – watch this space in June:� Shared Source
http://www.microsoft.com/resources/sharedsource
� Curriculum Repository on MSDN AA
http://www.msdnaa.net/curriculum
27
Appendix: Support Information
28
Teaching Core OS ClassesTeaching Core OS ClassesTeaching Core OS ClassesTeaching Core OS Classes
TheoreticalTheoreticalTheoreticalTheoreticalCoursesCoursesCoursesCourses
MixedMixedMixedMixedCoursesCoursesCoursesCourses
PracticalPracticalPracticalPracticalCoursesCoursesCoursesCourses
few labs labs many labs
SysadminSysadminSysadminSysadmin/programming/programming/programming/programming System programSystem programSystem programSystem program KernelKernelKernelKernel modifymodifymodifymodify SystemSystemSystemSystem programprogramprogramprogram OS designOS designOS designOS design lab
resources
teaching
resources
products
Academic
papers
CDKCDKCDKCDK ProjectProjectProjectProjectOZOZOZOZ
WindowsCE/XPe
SharedSharedSharedSharedSourceSourceSourceSource
Advanced Courses Research
Textbooks/Project Books
~50% ~15-20%~30-35%
29
Virtual PC
http://www.microsoft.com/windowsxp/virtualpc/
• Virtual PC 2004 as a tool for running several versions of Windows simultaneously on one machine.
• It could also be used to run one or more independent Unix (or Linux, Natchos, etc) sessions as hosted operating systems under Windows. You don’t have to reboot, an OS can crash without taking Windows down, and each session runs real Unix (or Windows, if you choose).
• The virtualized file system allows you to wipe out changes made during a session, so you can experiment without rendering the OS unbootable. And every Virtual PC-hosted OS automatically inherits (through virtualization) all of the devices and networking you’ve set up for Windows.
• It gives us exactly what we need – and also allow students and teachers utilize their Windows machines.
• Virtual PC Availability and Pricing - an estimated retail price is $129.• It also will be included in Microsoft's MSDN® subscriptions.• (based on my conversation with VPC GPM – they can produce ‘an academic’ / free
version ).
http://www.microsoft.com/windowsxp/virtualpc/
• Virtual PC 2004 as a tool for running several versions of Windows simultaneously on one machine.
• It could also be used to run one or more independent Unix (or Linux, Natchos, etc) sessions as hosted operating systems under Windows. You don’t have to reboot, an OS can crash without taking Windows down, and each session runs real Unix (or Windows, if you choose).
• The virtualized file system allows you to wipe out changes made during a session, so you can experiment without rendering the OS unbootable. And every Virtual PC-hosted OS automatically inherits (through virtualization) all of the devices and networking you’ve set up for Windows.
• It gives us exactly what we need – and also allow students and teachers utilize their Windows machines.
• Virtual PC Availability and Pricing - an estimated retail price is $129.• It also will be included in Microsoft's MSDN® subscriptions.• (based on my conversation with VPC GPM – they can produce ‘an academic’ / free
version ).
30
What is our approach?
Faculty requirements:
Comprehensive, modular and up-to-date curricula & content:
• Continuation of subjects / Spiral learning / Non-prescriptive
• Flexible in levels (institutions; courses) and delivery (web-based teaching)
• Cool topics and technology to attract and retain students (e.g. Gaming Technology, Mobility, Embedded, Web development, Robotics, etc. )
• Multidisciplinary topics (engineering; business)
• Reference material and textbooks
to provide content in three areas:
1. undergraduate, 2. graduate, 3. research and publ ications
University Requirements
Meet existing curriculum accreditation & industry recognition guide lines:
• USA: ACM/IEEE ‘Body of Knowledge’
• WW: varies by country (incl. government/industry requirements)
Motivators:
• Incentive to Faculty 1. Modern, best teaching OS, easy to introduce; modular offerings; saves work (No special lab equipment - use Virtual PC )2. Software Engineering tools (Dev. Analysis and Test Tools)3. Research/publication potential
“Why should I change my material?”
• Incentive to Students 1. Employability 2. Engaging3. Up to date knowledge
“Will this class help me to get a job or a place in Graduate School?”
31
CDK Core Topics – available in early July 2005
Overview of Operating Systems – (Core) OS1• Windows Operating System Internals Course Overview (Core)• The Evolution of Operating Systems (Core)• Windows Operating System Family – Concepts & Tools (Core)
Operating System Principles – (Core) OS2• Structuring of the Windows Operating System (Core)• The Windows API – Naming Conventions, Types (Core)• History of the Windows NT/2000/XP/2003 operating system (Core)• OS Principles labs, quizzes, and assignments
Concurrency – (Core) OS3• Critical Sections, Semaphores and Monitors (Core) • Windows Object Manager, Trap Dispatching, Synchronization (Core)• Windows Inter-process Communication (Core/Advanced)• Concurrency labs, quizzes, and assignments
Scheduling and Dispatch – (Core) OS4• The Concept of Processes and Threads (Core)• Windows Processes and Threads (Core)• Windows Process and Thread Internals (Core/Advanced)• Windows Thread Scheduling (Core) • Advanced Windows Thread Scheduling (Core/Advanced)• Scheduling and Dispatch labs, quizzes, and assignments
Memory Management – (Core) OS5• Memory Management for Multiprogramming (Core)• Windows Memory Management Internals (Core)• Windows Memory Structures (Core)• Advanced Windows Memory Management (Core/Advanced)• Memory Management labs, quizzes, and assignments
Protection and Security – (Elective) OS7• The Security Problem (Elective) • Windows Security Components and Concepts (Elective)• Windows Security Descriptors (Elective/Advanced)• Security labs, quizzes, and assignments
32
CDK elective & supplementary topics – available in Fall 2005
Device Management - The Input/Output System – (Electi ve) OS6• Principles of I/O Systems (Elective)• The Windows I/O System Components (Elective)• Windows I/O Processing (Elective/Advanced)• Device Management labs, quizzes, and assignments
File System – (Elective) OS8• Background: Unix File Systems (Elective)• The Windows File System (NTFS) (Elective)• Encrypting File System Security in Windows OS (Elective/Advanced)• NTFS – Recovery Support (Elective/Advanced)• Windows File and Directory Management (Elective)• File System labs, quizzes, and assignments
Real-time and Embedded Systems – (Elective) OS9• Introduction and Vocabulary (Elective) • Real-Time Systems with Windows (Elective)• Embedded Systems with Windows XP Embedded (Elective)
Fault-tolerance (Elective) OS10 System Performance Evaluation and Troubleshooting ( Elective) OS11Scripting (Elective) OS12
SUPPLEMENTARY UNITSWindows Networking – (Supplementary/Advanced)
• Networking Components in Windows OS (Supplementary/Advanced)• Windows Socket Programming (Supplementary/Advanced)• Microsoft-specific extensions to Sockets and other Networking APIs (Supplementary/Advanced) • Networking labs, quizzes, and assignments
Comparing the Linux and Windows Kernels (Supplement ary/Advanced)Windows – Unix Interoperability (Supplementary/Advan ced)
• File and Command Interoperability (Supplementary/Advanced)• Programming (Supplementary/Advanced)
33
Project OZ examples by Dr. Dave Probert
(in no particular order)
34
Loading Program Images
Take the sections in a file and load into an addres s space• dynamic relocation• shared libraries• creating stacks and initial thread contexts• create environment and other parental state• Initialize IO descriptors from parent
Using• NtCreateProcess to create an NT process container• NT VM to manage/modify process contents• NT library functions to take apart images and set environment• NT thread APIs to execute thread in new process• NT handle duplication to set IO descriptors
35
System Calls
Communicate requests for system services• implement basic system calls• access data cross domains• parameter validation and penetration testing• asynchronous operations• servicing models
Using• NT LPC to perform RPC• NT VM to access/modify child process state• NT threads to support flow-of-control
36
Manage address spaces, physical memory & ptes
Build data structures for managing VM-related resou rces• basic data structures and algorithms• support various page-table models
Using• NT VM to implement actual mappings (and shared memory) for
address spaces• NT VM to implement dirty bits• NT VM shared memory provides page sharing and simulates
physical pages with virtual pages• NT threads provide DMA (for IO)
37
Implementing Virtual Memory
Build various kinds of virtual memory systems• use previous resource management for addresses, mem, page
tables • page file organization• replacement algorithms• shared memory, object/file-backed memory regions• IO/DMA simulation• performance measurement of algorithms
Using• NT LPC/exceptions to handle page faults• NT VM to control actual mappings• NT IO for access to page file
38
Create Processes
Implement processes• create/delete processes in the Oz world • use loader project for loading programs• experiment with different models for process creation and program
execution• implement handle tables for referencing objects• manage run-down and synchronization issues
Using• NT processes to create real VM state (but for little else)
39
Create Threads / Scheduling
Implement threads• create/delete threads in an Oz process • build appropriate data structures to represent thread• tie into Oz process data structures• create stacks and manage thread execution context (PCB)• experiment with ideas like scheduler activations• implement a scheduler, using different policies, priorities, etc• add multi-processor support
Using• NT threads to represent processors and execute Oz threads on NT
threads (Oz threads are essentially user-mode threads)• NT APCs (Asynchronous Procedure Calls) to deliver timer
interrupts to running Oz threads
40
Synchronization
Implement various synchronization primitives• build user-mode, kernel-mode, and hybrid synchronization
primitives of various kinds (events, rw locks)• extend Oz scheduler to support blocking of threads• experiment with deadlock issues, priority inversion
Using• NT compare&swap primitives
41
Other project areas
IO ArchitectureFile SystemsNetworkingSystem bootstrapPerformance measurement & instrumentation (e.g. tra cing)Error handlingExceptions and traps, stack unwindingManagement of physical resourcesManagement of resource sharing policiesNUMA multiprocessorsSecurity, authentication, ACLsObject supportNamespacesVirtual machines