01intro

I–1

CS 167 I–1 Copyright © 2006 Thomas W. Doeppner. All rights reserved.

CS 167: Operating Systems

I–2


Workload: 167

• Write five easy-to-moderate-to-difficult programs (40%)

– shell– multithreaded programming/high-performance

I/O– user-level threads package– system calls and high-level file system– low-level file system

• Hand in five short homeworks (20%)• One in-class midterm exam (10%)• Final exam (30%)• See http://www.cs.brown.edu/courses/cs167/docs/syllabus.167.htm

http://www.cs.brown.edu/courses/cs167/docs/syllabus.167.htm

I–3


Workload: 169

• One large program with a number of milestones, resulting in a working operating system

– Kernel 1– Kernel 2– Virtual file system– S5 file system– Virtual memory

• See http://www.cs.brown.edu/courses/cs167/docs/syllabus.169.htm

http://www.cs.brown.edu/courses/cs167/docs/syllabus.169.htm

I–4


Skills Needed

• Ability to write and debug largish programs in C with POSIX threads

– CS 36– or CS32 with

- C/C++ minicourse (this week)- POSIX threads lectures/labs (next week)

• Basic computer architecture– CS 31

I–5


What are Operating Systems?

• Possible definitions:– the code that {Microsoft, Sun, Apple, Linus}

provides– the code that you didn’t write– the code that runs in privileged mode– the code that makes things work– the code that makes things crash– etc.

I–6


Operating Systems

• Abstraction– providing an “appropriate” interface for

applications• Concerns

– performance- time and space

– sharing and resource management– failure tolerance– security– marketability

The purpose of an operating system is to make the underlying hardware useful. Rather than forcing programmers to deal with the complicated details of such hardware, more convenient idealized abstractions of the hardware are devised, implemented, and made available, replacing the direct interface to the hardware. Thus, as discussed in upcoming slides, we deal with files rather than disks, virtual memory rather than real memory, threads rather than processors, etc.

However, in operating systems, even more so than in “ordinary” applications we must cope with the concerns listed in the slide. Performance is a huge issue. Some portions of the OS might be invoked often, say thousands of times per second. Reducing the number of instructions used is extremely important; we must also be concerned with how effectively the code uses the processor’s caches. Use of memory, both primary and secondary, is also important. If too much memory is used, the operating system might not “fit” in the available resources. Furthermore, too much memory use may require shuffling of code and data between primary and secondary storage, thus wasting time.

We must also provide various forms of sharing: not only must different applications coexist on a computer at once, but different users might be using the same computer at the same time. Our operating system must be tolerant of failures: bugs in one application must not bring down another. Problems with hardware should not cause catastrophic failure.

Security is increasingly a problem. We must protect the operating system, the applications running on it, and data maintained by it, from attack. (This is clearly not a solved problem!)

Finally, for an OS to be successful, people must want to use it. It’s one thing to build an operating system that does an excellent job dealing with everything mentioned above, but if none of our existing applications can be made to run on it and we have to write everything from scratch, no one is going to want to use it.

I–7


Hardware

Memory

Net

wor

k

Processor

Processor

Disk Disk

As we’ve just mentioned, one of the major purposes of an operating system is to give its users an easy-to-use abstraction of a fairly complicated reality. A typical computer system consists of a number of disk drives (which the operating system accesses through specialized hardware called controllers), a network interface (also accessed via a controller), some sort of display/keyboard/mouse combination, and some amount of primary storage.

I–8


Abstractions

• Hardware– disks– memory– processors– network– monitor– keyboard– mouse

• Operating system– files– programs– threads of control– communication– windows,

graphics– input– locator

The typical user has no desire to deal with the complexities of the underlying hardware, preferring instead to deal with abstractions and depend upon the operating system to map these abstractions into reality. For example, the user may have information that needs to be accessed and perhaps updated over long periods of time. While such information may well be stored on disk, the user probably doesn’t care how.

The user’s computation will certainly use temporary storage that need not exist longer than the computation, but should be quickly accessible during the lifetime of the computation. The user’s computation will need a “compute engine” to carry out the computation; in some cases it will need multiple compute engines. Finally, the user may need to communicate with other users or computations on other computers, and may need to access information stored elsewhere.

Designing the most suitable abstractions for an operating system to present to its users is a subject of continuing research. Soon we examine the abstractions provided by a particular operating system, Unix, that is in widespread use on computer workstations and larger machines. Later in the course we discuss alternate abstractions.

Unix provides a relatively simple collection of abstractions (though many argue that Unix has become much too complicated). In particular, permanent storage is represented as files. We think of temporary storage as being built from the abstraction of primary memory, known as virtual memory. Each computation has its own private virtual memory—computations are known in Unix as processes. Traditionally, each process is embodied as a single abstract compute engine, but more recently the concept of multithreaded processes has been introduced. In this model, each process may contain multiple compute engines, or threads, each capable of independent computation. Communication is not a very well developed concept in Unix systems. The general idea is that independent processes can transfer data back and forth among one another. We explore this idea and newer approaches to handling communication in the last section of the course.

I–9


Files

Memory

Disk Disk

File systems provide a simple abstraction of permanent storage, i.e., storage that doesn’t go away when your program terminates or you shut down the computer. A user accesses information, represented as files, and the operating system is responsible for retrieving and storing this information. We expect files to last a long time and we associate names (as opposed to addresses) with files.

I–10


Issues

• Naming• Allocating space on disk (permanent storage)

– organized for fast access– minimize waste

• Shuffling data between disk and memory (high-speed temporary storage)

• Coping with crashes

Among the issues in implementing the notion of files are those mentioned above.

I–11


Programs

Data

Code

Memory

Processor

Disk Disk

Processor

A program is an abstraction that we view as consisting of data and code. Somehow we’ve got to build programs from the pieces of hardware available to us, in particular memory, disks, and processors.

I–12


MemorySharing (1)

Memory

Program 1

Program 2

Program 3

OperatingSystem

So that multiple programs, each with their own private libraries and each accessing shared libraries, can coexist without destroying each other, we must have some means of protecting one program’s memory from another’s. Furthermore, we must protect the operating system from being inadvertently or maliciously attacked by programs.

Using various hardware features, each program is prevented from accessing private portions of other programs. Shared portions can be accessed, but only in a read-only fashion. When the operating system is executing, it can access the user programs. But when user programs are executing, they can access the operating system only to do prescribed requests.

I–13


MemorySharing (2)

Program 1

Program 2

Program 3 Memory

Another memory-sharing issue arises when the programs that are to share memory are too big to fit in all at once (or perhaps even individually).

I–14


Virtual MemoryProgram 1

Program 2

Program 3

Memory

Disk Disk

One popular approach for dealing with memory-protection and -fitting issues is to employ virtual memory. A program executes in a “virtual address space,” which is implemented partially on real memory and partially on disks. A facility supported as a joint effort of hardware and the operating-system kernel “maps” each program’s references of virtual memory into real memory and disk space. Since the operating system controls this mapping, it determines who can access what.

I–15


History of Virtual Memory

• 1961: Atlas computer, University of Manchester, UK

• 1962: Burroughs B5000• 1972: IBM OS/370• 1979: 3 BSD Unix, UC Berkeley• 1993: Microsoft Windows NT 3.1• 2000: Apple Macintosh OS X

The slide lists milestones in the history of virtual memory, from its first instance on Manchester’s Atlas computer in 1961 (when a working prototype was completed), to Apple’s recent announcement of VM support for the Macintosh.

I–16


Apple Invents VM …

Welcome to the Brave New World of Crash-Resistant ComputingLet’s start with the notion of protected memory, and why it’s so important. … One of the ways an operating system ensures reliability is by protecting applications through a mechanism called protected memory (essentially walling off applications from each other). …Along with the protected memory mechanism, Darwin provides a super-efficient virtual memory manager to handle that protected memory space. So you no longer have to worry about how much memory an application like Photoshop needs to open large files. …

— Apple website, September 2000

Apple is the latest of a long line of companies to “discover” virtual memory. The passage quoted in the slide is from Apple’s web page and is copyright (by Apple) 2000. “Darwin” was the code name of the kernel of Apple’s OS X operating system.

I–17


Concurrency

Processor

Virtual Processor

Virtual Processor

Virtual Processor

Computers are typically multiplexed among a number of users. Multiplexing is a concept that is well over thirty years old—many users share a computer by dividing its processing time among one another. It’s often known as multitasking.

Another equally old, if not older, concept is the notion of concurrency within a computation. This is the ability of one computation to utilize multiple logical “compute engines,” or threads. Concurrency means that multiple threads are in progress at one time; on a computer that employs only a single real processor, the execution of these threads is multiplexed over time.

Why do this?• Optimize the use of computers

— programs typically compute, perform I/O, compute, perform I/O …— when one program is performing I/O, another should be computing

• Optimize the use of people— want several applications active at once

– interactive applications· editing· drawing

– background applications· mail monitor· printing· file transfer

I–18


Parallelism

Processor

Virtual Processor

Virtual Processor

Virtual Processor

Processor

Processor

Parallelism means that multiple threads are executing simultaneously: parallelism requires multiple processors.

In this course we make this distinction between parallelism and concurrency; note, however, that not everyone distinguishes between these terms in this way.

I–19


History of Concurrency

• Multiprogramming– 1961, 1962: Atlas, B5000– 1965: OS/360 MFT, MVT

• Timesharing– 1961: CTSS (developed by MIT for IBM 7094);

BBN time-sharing system for DEC PDP-1– mid 60s

- Dartmouth Timesharing System (DTSS)- TOPS-10 (DEC)

– late 60s- Multics (MIT, GE, Bell Labs)- Unix (Bell Labs)

Multiprogramming refers to the notion of having multiple programs active at the same time so that when the current program can no longer run (for example because it’s blocked waiting for I/O), another is available to run. Timesharing is an extension of multiprogramming in which the execution of the active programs is time-sliced: each program runs for a short period of time, then another is run. A good web site with useful information on Multics is http://www.multicians.org/. A brief description of CTSS can be found at http://www.multicians.org/thvv/7094.html.

http://www.multicians.org

http://www.multicians.org/thvv/7094.html

I–20


Another Apple Innovation …

With Preemptive Multitasking, Everything Happens at OnceIn today’s fast-paced world, you rarely get to do one thing at a time. Even in the middle of transforming, say, a Photoshop file, you may need to find a crucial piece of information on the web while you compose an urgent reply to a customer. What you need is a computer that can handle several different tasks at once, giving priority to your primary application, but still crunching away at other jobs in the background. …Darwin makes this possible by incorporating a powerful concept called preemptive multitasking. …

— ibid.

So what’s preemptive multitasking? It’s multiprogramming in which one program can preempt the execution of another. In other words, it’s a necessary ingredient of timesharing, the notion that was first implemented in the 1960s.

(Note that Microsoft “innovated” this idea in 1993 with their introduction of Windows NT.)

I–21


Brief History ofOperating Systems

• Hardware technology– first generation: vacuum tubes– second generation: transistors– third generation: integrated circuits– fourth generation: graphics, personal

computers, networking• OS architecture

– monolithic operating systems– microkernels– personal-computer operating systems– network operating systems

In the next several slides we look at the history of operating systems, showing that the design for the CS169 final project is the culmination of years of operating-system research and development. We look first at the history of hardware developments and how they affected OS design, and then at various architectures of operating systems.

I–22


First-Generation Hardware

• “What you write is all you’ve got”– no operating system– no libraries– no compilers– no assemblers– no nothing

The earliest electronic computers were, by our standards, amazingly primitive. The hardware was characterized by the use of vacuum tubes. The software was characterized by its paucity. Programmers supplied all software that they were to use. Furthermore, this software was written directly in “machine language” and input to the computer via awkward means, such as paper tape.

I–23


IBM 701

The IBM 701 is, reportedly, the first computer to have an operating system (designed by General Motors (not IBM) and called the Input/Output System). The man on the right is computer pioneer Herb Grosch, in whose hotel room the notion of an operating system was hashed out in 1953. He later served as president of ACM. The man on the left is an actor who later served as president of a large country. The picture is from http://www.fwtunesco.org/musi/museu8.html (the link no longer works), the web site of FWT UNESCO Computer Museum, Padova, Italy, and was taken in 1956.

http://www.fwtunesco.org/musi/museu8.html

I–24


Second-Generation Hardware

• Batch systems• Concurrent I/O and computation• Compilers• Assemblers• Libraries

The second generation of computer hardware is characterized by the use of transistors. Improvements in hardware came along with major improvements in software—compilers, assemblers, and loaders were invented. Libraries were developed and linkers were made available to ease their use. The early operating systems were known as “batch” systems—a collection, or batch, of jobs was submitted and their output was written to magnetic tape. After the batch was completed, the operator would take the tape to another computer that drove a printer and start the next batch. Thus while one batch was running, the results of the previous batch were printed.

Also at this time the use of I/O devices such as disks, tapes, and drums became more advanced. In early systems, all activity on a machine ceased while the computation was waiting for an I/O operation to complete. In the second generation, the use of separate hardware to manage I/O and the concept of interrupts made it possible for a program to continue to compute while I/O was being performed. The difficulty of managing such concurrency helped to spur the development of operating systems.

I–25


IBM 7094

The picture, from http://www.ecs.csun.edu/~dxs/history/histcomp.html, is of an IBM 7094, a midrange second-generation scientific computer. Among its features were core memory, a disk drive, and a subroutine-call instruction.

http://www.ecs.csun.edu/~dxs/history/histcomp.html

I–26


Early Third-Generation Hardware

• Multiprogramming• Time sharing

• Big operating systems– OS/360– Multics

• Little operating systems

– Unix

In the third generation, characterized by the use of (small-scale) integrated circuits, OS technology begins to resemble what we have today. Large machines were multiprogrammed, allowing several jobs to run concurrently. This made security very important, since jobs now had to be protected from one another. Another important development was time sharing, in which a programmer could actually interact with a running program. This was considered a rather expensive frill at first, but has since become the dominant style of computing.

The most commercially important OS of the day was IBM’s OS/360. This system was based on the radical concept that a whole range of hardware, from small, slow machines to huge, fast machines, could all run the same operating system. Unfortunately, it didn’t work exactly as planned, and OS/360 had to be split into a number of different versions, each targeted to a particular class of machines. The largest of these OSs, known as OS/MVT, is the forerunner of the OSs run on large data-processing machines today.

The most interesting OS of the day was Multics, which was initially a joint project of GE, MIT, and Bell Laboratories. The latter two eventually dropped out and GE’s computer business was bought by Honeywell, but under the Honeywell nameplate this OS survived into the ’80s. Many of the concepts that have just now become commercially viable were pioneered in the Multicssystem.

The dark-horse operating system was Unix, which was developed at Bell Labs after it dropped out of the Multics project. It gradually became popular among researchers at Bell Labs. Bell Labs made the radical decision to release of the source code of this OS to universities, and it soon became very popular in a large number of computer science departments and a few enlightened industrial organizations. Unix ran principally on DEC PDP-11s, but was ported to a few other machines and gained the reputation of being the first portable operating system (another very radical concept for software that had traditionally been written in assembler language and took advantage of every arcane feature a machine had to offer). A group at the University of California at Berkeley took Bell Lab’s port of Unix to DEC’s new architecture, the VAX, and embellished it considerably. This version of Unix, known as BSD (Berkeley Software Distribution), or simply Berkeley Unix, was adopted by a large number of academic CS departments around the world.

I–27


$

• If you had $10,000 to invest on January 1, 1962, would you have put it in IBM?

I–28


Later Third-Generation Hardware

• Minicomputers– proliferation of platforms and OSes

- Digital PDP-11• RSX-11, RSTS-11, RT-11, etc.

- Data General Nova, Eclipse• RDOS, SOS, AOS

- Hewlett Packard 2100, 3000• BCS, DOS, MPE, MTS

– see http://www.cs.wvu.edu/~jdm/classes/cs258/OScat/minis.html

The ’70s saw an amazing number of minicomputers.

http://www.cs.wvu.edu/~jdm/classes/cs258/OScat/minis.html

I–29


$$

• It’s now January 1, 1970; your stake in IBM is worth $22,557

I–30


PDP-11

The PDP-11 shown in the left half of the picture (a relatively small configuration) is around 6-feet tall, making one wonder about the appellation “mini”! On the right is a closeup of the console of a model 45 (an early high-end model) and a closeup of the backplane of a model 50 (a later, midrange model).

I–31


Late 70s, Early 80s

• Super minicomputers– DEC VAX-11, etc.

• Workstations– Apollo, Sun

• Personal computers– MITS Altair, Apple II, TRS-80, IBM PC, etc.

• Xerox PARC– Alto, Dorado, etc.

During the late ’70s and early ’80s, the minicomputer manufacturers (particularly DEC) thrived with big sales of high-end minicomputers. The early 80s saw the beginning of workstation-class computers, with first the establishment of Apollo, then Sun.

Meanwhile, computer hobbyists got into the act with the development of the MITS Altair (the first affordable piece of hardware that you could reasonably call a computer; it came complete with 256 8-bit words of memory). Steve Wozniak and Steve Jobs developed the successful Apple II computer. Then IBM galvanized the entire industry with the introduction of the IBM PC in the early ’80s.

Perhaps the most important work was being done at Xerox PARC, where the whole notion of computer workstations and personal computers was invented. (This work actually started in the early-to-mid ’70s.)

I–32


¢¢

• It’s now January 1, 1975; your stake in IBM has gone down to $15,323

I–33


VAX-11

• Two operating systems– VMS (sort of the predecessor of Windows XP)– Berkeley Unix (definitely the predecessor of

Solaris, Linux, and FreeBSD)

The Digital VAX-11 (a model 780, the first, is shown in the picture) was the most successful of the superminis. While Digital developed, marketed, and pushed their VMS operating system, Unix, as modified and developed by a group at UC Berkeley, became extremely popular as well.

I–34


Apollo

The CS Department at Brown was one of Apollo’s first customers (receiving the 5th and 6th computers delivered to customers).

I–35


Apple II

• See http://apple2history.org/

The Apple II was a very successful hobbyist computer that eventually made it into the business world, as well as into the educational world.

http://apple2history.org

I–36


PC Operating Systems

• Apple II– DOS 3.x– SOS, ProDOS

• Z80, 8080– CPM

- see http://en.wikipedia.org/wiki/Gary_Kildall• 8086

– MS-DOS- See, for example,

http://en.wikipedia.org/wiki/MS-DOS

The early history of PC operating systems is controversial, interesting, entertaining, etc. Check out the links given in the slide.

http://en.wikipedia.org/wiki/Gary_Kildall

http://en.wikipedia.org/wiki/MS-DOS

I–37


$$$

• It’s now January 1, 1980; your IBM stake is worth $27,493

I–38


Further PC Development

• Apple introduces the Apple III– thud …

• Microsoft, selling software, starts doing better in the PC market than IBM, selling hardware

• Apple introduces the Lisa …– thud …

• Apple introduces the Macintosh

–BOINNNG!!!

– $$$$$$$$$$$$$$$

I–39


$$$$

• It’s January 1, 1985; your IBM stake has soared to $76,340

I–40


$$$$$

• It’s March 13, 1986. Microsoft is going public. Do you put $10,000 into their IPO?

(you didn’t …)

(though my broker has a client who did …)

(and it wasn’t me)

($10,000 got you approximately 477 shares @ $21)

I–41


Toy Operating Systems

• 1987: Andrew Tanenbaum of VrijeUniversiteit, Amsterdam, publishes Operating Systems: Design and Implementation

– included is source code for a complete, though toy, operating system: Minix, sort of based on Unix

• 1991: Linus Torvalds buys an Intel 386 PC– MS-DOS doesn’t support all its features (e.g.,

memory protection, multi-tasking)– “soups up” Minix to support all this

• January 1992: Torvalds releases Linux 0.12• January 1992: Tanenbaum declares Linux

obsolete

I–42


$$$$$$

• It’s January 1, 1990. Your stake in IBM is worth $64,775.

• The $10,000 you didn’t put into Microsoft would now be worth $66,071.

(though note that, unlike Microsoft’s, IBM’s stock has always returned dividends. Microsoft started returning dividends in 2004)

I–43


Late 80s/Early 90s

• 1988: Most major Unix vendors get together and form OSF to produce a common Unix: OSF/1, based on IBM’s AIX

• 1989: Microsoft begins work on NT• 1990: OSF abandons AIX, restarts with Mach• 1991: OSF releases OSF/1• 1992: Sun releases Solaris 2

– many SunOS (Solaris 1) programs are broken• 1993: All major players but DEC have

abandoned OSF/1• 1993: Microsoft releases Windows NT 3.1• 1994: Linux 1.0 released

I–44


$$$$$$$

• It’s January 1, 1995. Your IBM stake has dropped to $57,476

• Your Microsoft stake would have been worth $391,339

I–45


Late 90s

• IBM has three different versions of Unix, all called “AIX”

• 1996: DEC renames its OSF/1 “Digital Unix”• 1996: Microsoft releases Windows NT 4• 1996: Linux 2.0 released• 1998: DEC is purchased by Compaq; “Digital

Unix” is renamed “Tru64 Unix”• 1999: Sun’s follow-on to Solaris 2.6 is called

Solaris 7

I–46


The ’00s• 2000: Microsoft releases Windows 2000 and Windows

Me• 2000: Linux 2.2 is released• 2000: IBM “commits” to Linux (on servers)• ~2000: Apple releases OS X, based on Unix (in

particular, OSF/1)• 2001: Linux 2.4 is released• 2001: Microsoft releases Windows XP• 2002: Compaq is purchased by HP• 2003: SCO claims their code is in Linux, sues IBM; IBM

countersues– (trial currently scheduled for Feb. 2007)

• 2004: Linux 2.6 is released• 2005: IBM sells PC business to Lenovo

I–47


$$$$$$$$$$$$$$$$

• It’s January 1, 2000• Your IBM stake is worth $377,423• Your Microsoft stake would have been worth

$5,927,143• You should have sold instantly

– on September 5, 2006- your IBM stake is worth $282,894- your Microsoft stake would have been

worth $3,518,199• If you’d traded all your MS stock for Google at

their IPO on 8/18/2004, you’d now have $17,209,146

On September 5, 2006 you own 3499 shares of IBM stock. Check outhttp://www.microsoft.com/msft/download/IPOsharecalc.xls for information on Microsoft stock. On 8/18/2004, your Microsoft investment would have been worth $3,765,942. Google sold their stock for $85/share at their IPO on 8/18/2004. On 9/5/2006, it sold for $384.36/share.

http://www.microsoft.com/msft/download/IPOsharecalc.xls

I–48


CS167/169 Operating SystemsAn Abbreviated History—Part 1

• Prehistory– IBM 360-like simulator written in PL/1– simple OS written in IBM 360 assembler

• Dawn of civilization– Intel 8086-like architecture– simple OS written in pseudo code

- paper-only design• Classical period

– Motorola 68000 simulator written in C- first ran on 68000-based Apollos, then on 68000-

based Suns, then on SPARC-based Suns– very simple OS design– OSes first written in 68000 assembler, later in C

We finish this lecture with a brief history of the OS project of CS 167 and 169.

I–49



• The Enlightenment– new simulator for “simple architecture” written in C

- no virtual memory- much faster than previous simulators

– first OS: Unix-like, no virtual memory– second OS: SLO (Solaris-Like OS)

- first to have a name- microkernel architecture

• an architecture designed to ease OS writing• few students completed the project ...

I–50



• The Industrial Revolution: Objects for the Masses

– same simulator as used in the enlightenment– OS design based on Sun’s Spring object-

oriented OS; OSes written in C++- FallOS- Doors 95- Doors NT- OpenDoors

I–51



• The Post-Modern Age– new simulator, written in C

- supports dynamic address translation– OS design based on Linux

- virtual memory- “network capable”- completely documented (well, almost …)

- marketed under the name Weenix™– consumer version

- Weenie™Gone the way of

Windows Me

Technology

01intro