Chapter 4: Threads - JUST

1

Chapter 4: ThreadsChapter 4: Threads

4.2 Silberschatz, Galvin and Gagne ©2005Operating System Concepts – 7th edition, Jan 23, 2005

Chapter 4: ThreadsChapter 4: Threads

Overview

Multithreading Models

Threading Issues

Pthreads

Windows XP Threads

Linux Threads

Java Threads

2


Processes Processes

Recall that a process includes many things

An address space (defining all the code and data pages)

OS resources (e.g., open files, child processes) and accounting information

Execution state (PC, SP, regs, etc.)

Creating a new process is costly because of all of the data structures that must be allocated and initialized

Recall struct proc in Solaris

…which does not even include page tables, perhaps TLB flushing, etc.

Communicating between processes is costly because most communication goes through the OS

Overhead of system calls (interrupt) and copying data


Parallel ProcessesParallel Processes

Think of a Web server example that forks off copies of itself tohandle multiple simultaneous requests

Or any parallel program that executes on a multiprocessor

To execute these programs we need to

Create several processes that execute in parallel

Cause each to map to the same address space to share data

They are all part of the same computation

Have the OS schedule these processes in parallel (logically or physically)

This situation is very inefficient

Space: each process needs PCB, page tables, etc.

Time: create data structures, fork and copy addr space, etc.

3


Rethinking ProcessesRethinking Processes

What is similar in these cooperating processes?

They all share the same code and data (address space)

They all share the same privileges

They all share the same resources (files, sockets, etc.)

What don’t they share?

Each has its own execution state: PC, SP, and registers

Key idea: Why don’t we separate the concept of a process from its execution state?

Process: address space, privileges, resources, etc.

Execution state: PC, SP, registers

Exec state also called thread of control, or thread

A thread is bound to a single process

Processes, however, can have multiple threads

Threads become the unit of scheduling

Processes are now the containers in which threads execute

Result: allow multiple executions to take place in the same process environment


ConclusionConclusion

A traditional (or heavyweight) process has an address space and single thread of control (process)

Modern applications are multithreaded. Each thread does a task.

The thread defines a sequential execution stream within a process (PC, SP, registers)

The process defines the address space, resources, and general process attributes (everything but threads of execution)

A thread (or lightweight process) is a basic unit of CPU utilization. It consists of:

Program counter.

Register set.

Stack space.

Threads are useful when the application performs several similar tasks.

Web server: Process-based or Thread-based.

4


Threads in a Process


Multithreading

Multithreading: a process executing an application is divided into multiple threads of execution.

All threads (of the same process) run concurrently.

They share address space: code section & data section.

Share other operating-system resources, such as files.

No need for process context switching

Switching between threads is faster.

However, there is no protection between threads

one thread can read, write, or even completely wipe out another thread’s stack since they share the same address space

– Protection is not necessary, since unlike multiple processes, threads are created by the same user to cooperate not to fight.

Note: the process is the unit of resource management, not the thread

5


Single and Multithreaded ProcessesSingle and Multithreaded Processes


Multithreading BenefitsMultithreading Benefits

Responsiveness: allow an interactive application to continue running even if part of it is blocked or is performing a lengthyoperation.

Resource Sharing: threads share the memory (code and data) and the resources of the process to which they belong.

Since they do not have any resources attached to them, threads are easier to create and destroy than processes

Economy: generally, it is much more time consuming to create and manage processes than a thread.

Solaris: process creation 30X slower. Context switch 5X more.

Utilization of MP Architectures: threads may be running in parallel on different processors (A single-threaded process can only run on one CPU).

6


Thread Design Space

Single thread /single task Single thread/Multitask

Multithread/Single task Multithreads/Multitasks


Process/Thread Separation

Separating threads and processes makes it easier to support multithreaded applications

Creating concurrency does not require creating new processes

Concurrency (multithreading) can be very useful

Improving program structure

Handling concurrent events (e.g., Web requests)

Writing parallel programs

So multithreading is even useful on a uniprocessor

7


Threads: Concurrent Servers

Using fork() to create new processes to handle requests in parallel is overkill for such a simple task

Recall our forking Web server:

while (1) {

int sock = accept();

if ((child_pid = fork()) == 0) {

Handle client request

Close socket and exit

} else {

Close socket

}

}


Threads: Concurrent Servers

Instead, we can create a new thread for each request

web_server() {

while (1) {

int sock = accept();

thread_fork(handle_request, sock);

// usually, a threads creates new threads by calling a library procedure, for example, thread_create. the creating thread is usually returned a thread identifier that names the new thread.

}

}

handle_request(int sock) {

Process request

close(sock);

}

When a thread has finished its work, it can exit by calling a library procedure, say, thread_exit. It then vanishes and is no longer schedulable

8


Kernel Threads

We have taken the execution aspect of a process and separated it out into threads

To make concurrency cheaper

As such, the OS now manages threads and processes

Keeps track of all the required attributes of all the threads in the system

All thread operations are implemented in the kernel

The OS schedules all of the threads in the system

System calls are used to manage all thread operations

OS-managed threads are called kernel-level threads or lightweight processes

Kernel threads – supported and managed directly by the operating system.

Almost all contemporary operating systems support kernel threads.

Windows XP/2000

Solaris

Linux

Tru64 UNIX

Mac OS X


Kernel Thread Limitations

Kernel-level threads make concurrency much cheaper than processes

Much less state to allocate and initialize

However, for fine-grained concurrency, kernel-level threads still suffer from too much overhead

Thread operations still require system calls

Ideally, want thread operations to be as fast as a procedure call

Kernel-level threads have to be general to support the needs of all programmers, languages, runtimes, etc.

For such fine-grained concurrency, need even “cheaper” threads

9


User-Level Threads

To make threads cheap and fast, they need to be implemented at user level

Kernel-level threads are managed by the OS

User-level threads are managed entirely by the run-time system, above the kernel, and managed without kernel support

User-level threads are small and fast

A thread is simply represented by a PC, registers, stack, and small thread control block (TCB)

Creating a new thread, switching between threads, and synchronizing threads are done via procedure call

No kernel involvement

User-level thread operations 100x faster than kernel threads

As far as the kernel is concerned, it is managing ordinary, single-threaded processes

a user-level threads package can be implemented on an operating system that does not support threads


User-Level Thread Limitations

But, user-level threads are not a perfect solution

As with everything else, they are a tradeoff

User-level threads are invisible to the OS

They are not well integrated with the OS. For example, each process needs its own private thread table to keep track of the threads parameters and state in that process

As a result, the OS can make poor decisions

Scheduling a process with idle threads

Blocking a process whose thread initiated an I/O (page read, forexample), even though the process has other threads that can execute

Unscheduling a process with a thread holding a lock

If a thread starts running, no other thread in that process will ever run unless the first thread voluntarily gives up the CPU

Within a single process, there are no clock interrupts, making it impossible to schedule user threads round-robin fashion

Solving this requires communication between the kernel and the user-level thread manager

10


Kernel vs. User Threads

Kernel-level threads

Integrated with OS (informed scheduling)

Slow to create, manipulate, synchronize

User-level threads

Fast to create, manipulate, synchronize

Not integrated with OS (uninformed scheduling)

With user-level threads, the run-time system keeps running threads from its own process until the kernel takes the CPU away from it (or there are no ready threads left to run).

With kernel-level threads When a thread blocks, the kernel may run either another thread from the same process (if one is ready), or a thread from a different process

Understanding the differences between kernel and user-level threads is important

For programming (correctness, performance)

For test-taking

Best to use both kernel and user level threads

Multithreading models (next slides)


Thread Libraries

A thread library provides the programmer with an API for creating and managing threads.

Implemented either in the user space (function call) or the kernel space (system call)

Three primary thread libraries:

POSIX Pthreads.

May be either a user- or kernel-level library.

Win32 threads.

Kernel-level library.

Java threads

Typically implemented using a thread library available on the JVM’s host system.

11


Pthreads

A POSIX standard (IEEE 1003.1c) API for thread creation and synchronization.

The Pthreads API standard is a specification for thread behavior. Implementation is left up to the developer of the library.

Common in UNIX operating systems (Solaris, Linux, Mac OS X, Tru64 Unix).

Sharewares available for the Windows OS

Very useful example of Pthreads usage is explained in page 133


Multithreading ModelsMultithreading Models

Specifies the relationship between user threads and kernel threads.

Use kernel-level threads and then multiplex user-level threads onto some or all of the kernel threads.

The kernel is aware of only the kernel-level threads and schedules those.

Three models of doing so:

Many-to-One

One-to-One

Many-to-Many

12


ManyMany--toto--OneOne

Many user-level threads mapped to single kernel thread

On older Unix, only one “kernel thread” per process

Thread management done by the thread library routines at user level.

Entire process will block if a thread makes a blocking system call.

Multiple threads unable to run in parallel on multiprocessors since only one thread can access the kernel at a time.

May be used on systems that do not support kernel threads.

Examples:

Solaris Green Threads

GNU Portable Threads


ManyMany--toto--One ModelOne Model

13


OneOne--toto--OneOne

Each user-level thread maps to a kernel-level thread

Provides more concurrency than many-to-one (or many-to-many) by allowing another thread to run during blocking system calls.

Multiple threads can run in parallel on multiprocessors.

Most implementations restrict the number of kernel threads supported by the system due to overhead ( since creating a user thread requires creating the corresponding kernel thread, which can affect the system performance. Thus, the number is limited)

Examples

Windows NT/XP/2000

Linux

Solaris 9 and later


OneOne--toto--one Modelone Model

14


ManyMany--toto--Many ModelMany Model

Allows many user level threads to be mapped to a smaller or equal number of kernel threads

Allows the operating system to create a sufficient number of kernel threads to server the user threads.

Does not suffer from the blocking of all threads during a blocking system call, and the user threads are not limited by the number of kernel threads

Solaris prior to version 9

Windows NT/2000 with the ThreadFiber package


ManyMany--toto--Many ModelMany Model

15


Multithreading Models


TwoTwo--level Modellevel Model

Similar to M:M, except that it allows a user thread to be bound directly to a kernel thread

Examples

IRIX

HP-UX

Tru64 UNIX

Solaris 8 and earlier

16


TwoTwo--level Modellevel Model


Implementing Threads

Implementing threads has a number of issues

Interface (thread libraries)

Context switch

Preemptive vs. non-preemptive

Scheduling

Synchronization

Focus on user-level threads

Kernel-level threads are similar to original process management and implementation in the OS

17


Threading IssuesThreading Issues

Semantics of fork() and exec() system calls

Thread cancellation

Signal handling

Thread pools

Thread specific data

Scheduler activations


Semantics of fork() and exec()Semantics of fork() and exec()

Does fork() create a new process which duplicates only the calling thread, or all threads?

May have two versions of fork(). One that duplicates all threads and another that duplicates only the thread that invoked the fork() system call

Which one to use? Depends on the application if it call exec() directly after fork or not.

If a thread invokes exec(), the new program replaces the entire process, including all threads.

18


Thread CancellationThread Cancellation

Terminating a thread before it has finished

Press stop button before a browser load all images.

Two general approaches:

Asynchronous cancellation terminates the thread to be cancelled (target thread) immediately

Deferred cancellation allows the target thread to periodically check if it should be cancelled, allowing it an opportunity to terminate itself in an orderly fashion

Difficult in situations where recourses have been allocated to the target thread or when a thread is cancelled while in the middle of updating date it is sharing with other threads


Signal HandlingSignal Handling

Signals (synchronous, asynchronous) are used in UNIX systems to notify a process that a particular event has occurred

A signal handler is used to process signals

1. Signal is generated by particular event

2. Signal is delivered to a process

3. Signal is handled by the signal handler (default or user-defined)

Options for delivering a signal

Deliver the signal to the thread to which the signal applies

Deliver the signal to every thread in the process

Deliver the signal to certain threads in the process

Threads can specify which signals to receive.

Assign a specific thread to receive all signals for the process

19


Thread PoolsThread Pools

At process startup, create a number of threads and put them in a pool, where they await work

Advantages:

Usually slightly faster to service a request with an existing thread than create a new thread

Allows the number of threads in the application(s) to be bound to the size of the pool. Important on systems that cannot support a large number of concurrent threads.


Thread Specific DataThread Specific Data

Allows each thread to have its own copy of data

Useful when you do not have control over the thread creation process (i.e., when using a thread pool)

20


Scheduler ActivationsScheduler Activations

Both M:M and Two-level models require communication to maintain the appropriate number of kernel threads allocated to the application

lightweight process is an intermediate data structure between the user and kernel threads, which appears as a virtual process to the user-thread library..

Scheduler activations provide upcalls - a communication mechanism from the kernel to the thread library

Allows the kernel library to schedule (LWPs) across the virtual processors provided by the kernel threads.

This communication allows an application to maintain the correct number kernel threads


Thread Scheduling

The thread scheduler determines when a thread runs

It uses queues to keep track of what threads are doing

Just like the OS and processes

But it is implemented at user-level in a library

Run queue: Threads currently running (usually one)

Ready queue: Threads ready to run

Scheduling more than one thread means that we need to switch between threads

In other words, it context switches to another thread

21


Thread Context Switch

The context switch routine does all of the magic

Saves context of the currently running thread (old_thread)

Push all machine state onto its stack (not its TCB)

Restores context of the next thread

Pop all machine state from the next thread’s stack

The next thread becomes the current thread

Return to caller as new thread

This is all done in assembly language


Windows XP ThreadsWindows XP Threads

Implements the one-to-one mapping

Each thread contains

A thread id

Register set

Separate user and kernel stacks

Private data storage area

The register set, stacks, and private storage area are known as the context of the threads

The primary data structures of a thread include:

ETHREAD (executive thread block)

KTHREAD (kernel thread block)

TEB (thread environment block)

22


Linux ThreadsLinux Threads

Linux refers to them as tasks rather than threads

Thread creation is done through clone() system call

clone() allows a child task to share the address space of the parent task (process)


Threads Summary

The operating system as a large multithreaded program

Each process executes as a thread within the OS

Multithreading is also very useful for applications

Efficient multithreading requires fast primitives

Processes are too heavyweight

Solution is to separate threads from processes

Kernel-level threads much better, but still significant overhead

User-level threads even better, but not well integrated with OS

Now, how do we get our threads to correctly cooperate with each other?

Synchronization…

23

End of Chapter 4End of Chapter 4

Documents

Chapter 4: Threads - JUST