OSPP: Concurrency

Threads1TopicsThreadIntroductionMultithreading modelsThread librariesThreading issuesLinux ThreadsBut first..Shell slides I found3Operating Systems: A Modern Perspective, Chapter 2Shell Command Line InterpreterOS

Interactive User

Shell ProgramApplication& SystemSoftwareOS System Call InterfaceOperating Systems: A Modern Perspective, Chapter 2The Shell StrategyShell ProcessProcessto executecommand% grep first f3f3read keyboardfork a processread fileBack to ThreadsOperating Systems: A Modern Perspective, Chapter 2Abstract Machine EntitiesProcess: A sequential program in executionResource: Any abstract resource that a process can request, and which may can cause the process to be blocked if the resource is unavailable.File: A special case of a resource. A linearly-addressed sequence of bytes. A byte stream.Operating Systems: A Modern Perspective, Chapter 2Algorithms, Programs, and ProcessesDataFilesFilesFilesOtherResources

AlgorithmIdeaSourceProgramBinaryProgramExecution EngineProcessStackStatusOperating Systems: A Modern Perspective, Chapter 2Process AbstractionHardwareDataProcessStackProcessorExecutableMemoryProgramOperating System9Operating Systems: A Modern Perspective, Chapter 2Classic ProcessOS implements {process environment} one per taskMultiprogramming enables N programs to be space-muxed in executable memory, and time-muxed across the physical machine processor.Result: Have an environment in which there can be multiple programs in execution concurrently*, each as a processes* Concurrently: Programs appear to execute simultaneously10Operating Systems: A Modern Perspective, Chapter 2Multithreaded Accountant

Purchase OrdersInvoiceInvoiceSeparate ProcessesFirst AccountantSecond Accountant

Purchase OrdersInvoice

Accountant & CloneDouble Threaded ProcessDefinitionsA thread is a single execution sequence that represents a separately schedulable taskOperating Systems: A Modern Perspective, Chapter 2A Process with Multiple ThreadsDataFilesFilesFilesOtherResourcesBinaryProgramProcessStackStatusStackStatusStackStatusThread (Execution Engine)Single and Multithreaded Processes

14Thread LifecycleThread Specific DataAllows each thread to have its own copy of dataUseful when you do not have control over the thread creation process (i.e., when using a thread pool)16Single and Multithreaded Processes

Thread Data17Shared vs. Per-Thread StateOperating Systems: A Modern Perspective, Chapter 2Modern Process & ThreadDivide classic process:Process is an infrastructure in which execution takes place address space + resourcesThread is a program in execution within a process context each thread has its own stackDataProcessStackProgramOperating SystemThreadThreadThreadStackStackProcess Control BlocksPIDTerminated childrenLinkReturn codeProcess Control BlockPIDTerminated childrenLinkReturn codePIDTerminated childrenLinkReturn codeTCB LinkMultithreaded Server Architecture

threadthreadthreadthreadMotivationOperating systems need to be able to handle multiple things at once (MTAO)processes, interrupts, background system maintenance Servers need to handle MTAOMultiple connections handled simultaneouslyParallel programs need to handle MTAOTo achieve better performancePrograms with user interfaces often need to handle MTAOTo achieve user responsiveness while doing computationNetwork and disk bound programs need to handle MTAOTo hide network/disk latencyMulticore ProgrammingMulticore systems putting pressure on programmers, challenges includeDividing activitiesBalanceData splittingData dependencyTesting and debugging

Thread AbstractionInfinite number of processorsThreads execute with variable speedPrograms must be designed to work with any scheduleExecution model: each thread runs on a dedicated virtual processor with unpredictable and variable speed.24Thread ImplementationThreads can be implemented in any of several waysUser ThreadsKernel ThreadsThread management done by user-level threads library

Three primary thread libraries: POSIX Pthreads Win32 threads Java threads

25User ThreadsThreads can be implemented in any of several waysUser ThreadsThread management done by user-level threads libraryMultiple user-level threads, inside a UNIX process (early Java)Multiple single-threaded processes (early UNIX)Three primary thread libraries: POSIX Pthreads Win32 threads Java threads

Mixture of single and multi-threaded processes and kernel threads (Linux, MacOS, Windows)To the kernel, a kernel thread and a single threaded user process look quite similarScheduler activations (Windows)

Thread management done by user-level threads library

Three primary thread libraries: POSIX Pthreads Win32 threads Java threads

26Kernel ThreadsSupported by the Kernel

ExamplesWindows XP/2000SolarisLinuxTru64 UNIXMac OS X

Mixture of single and multi-threaded processes and kernel threads (Linux, MacOS, Windows)To the kernel, a kernel thread and a single threaded user process look quite similar

Scheduler activations (Windows)

Thread management done by user-level threads library

Three primary thread libraries: POSIX Pthreads Win32 threads Java threads

27Multithreading ModelsMany-to-One

One-to-One

Many-to-Many

User Thread to - Kernel Thread28Multithreading ModelsMany-to-One

One-to-One

Many-to-Many

User Thread to - Kernel Thread29Many-to-OneMany user-level threads mapped to single kernel threadExamples:Solaris Green ThreadsGNU Portable Threads30Many-to-One Model

31One-to-OneEach user-level thread maps to kernel threadExamplesWindows NT/XP/2000LinuxSolaris 9 and later32One-to-one Model

33Many-to-Many ModelAllows many user level threads to be mapped to many kernel threadsAllows the operating system to create a sufficient number of kernel threadsSolaris prior to version 9Windows NT/2000 with the ThreadFiber package34Many-to-Many Model

35Thread LibrariesThread library provides programmer with API for creating and managing threadsTwo primary ways of implementingLibrary entirely in user spaceKernel-level library supported by the OSJava ThreadsJava threads are managed by the JVM

Typically implemented using the threads model provided by underlying OS

Java threads may be created by:

Extending Thread classImplementing the Runnable interface

37PthreadsMay be provided either as user-level or kernel-levelA POSIX standard (IEEE 1003.1c) API for thread creation and synchronizationAPI specifies behavior of the thread library, implementation is up to development of the libraryCommon in UNIX operating systems (Solaris, Linux, Mac OS X)

38Thread Operationspthread_create(func, args)Create a new thread to run func(args)pthread_yield()Relinquish processor voluntarilypthread_join(thread)In parent, wait for forked thread to exit, then returnpthread_exitQuit thread and clean up, wake up joiner if any

Helpful linkshttp://www.yolinux.com/TUTORIALS/LinuxTutorialPosixThreads.html

https://computing.llnl.gov/tutorials/pthreads/

Best: http://www.cs.cmu.edu/afs/cs/academic/class/15492-f07/www/pthreads.html

Threading IssuesSemantics of fork() and exec() system callsThread cancellation of target threadAsynchronous or deferredSignal handlingThread poolsThread-specific dataScheduler activations41Semantics of fork() and exec()Does fork() duplicate only the calling thread or all threads?42Thread CancellationTerminating a thread before it has finishedTwo general approaches:Asynchronous cancellation terminates the target thread immediatelyDeferred cancellation allows the target thread to periodically check if it should be cancelled

43Signal HandlingSignals are used in UNIX systems to notify a process that a particular event has occurredA signal handler is used to process signalsSignal is generated by particular eventSignal is delivered to a processSignal is handledOptions:Deliver the signal to the thread to which the signal appliesDeliver the signal to every thread in the processDeliver the signal to certain threads in the processAssign a specific threa to receive all signals for the process44Thread PoolsCreate a number of threads in a pool where they await workAdvantages:Usually slightly faster to service a request with an existing thread than create a new threadAllows the number of threads in the application(s) to be bound to the size of the pool45Thread SchedulingProgrammer vs. Processor ViewPossible ExecutionsMain: Fork 10 threadscall join on them, then exitWhat other interleavings are possible?What is maximum # of threads running at same time?Minimum?Two threads call yieldThread PitfalsRace conditions

Ensuring Thread safe code

Mutex Deadlock

Race conditions: While the code may appear on the screen in the order you wish the code to execute, threads are scheduled by the operating system and are executed at random. It cannot be assumed that threads are executed in the order they are created. They may also execute at different speeds. When threads are executing (racing to complete) they may give unexpected results (race condition). Mutexes and joins must be utilized to achieve a predictable execution order and outcome.Thread safe code: The threaded routines must call functions which are "thread safe". This means that there are no static or global variables which other threads may clobber or read assuming single threaded operation. If static or global variables are used then mutexes must be applied or the functions must be re-written to avoid the use of these variables. In C, local variables are dynamically allocated on the stack. Therefore, any function that does not use static data or other shared resources is thread-safe. Thread-unsafe functions may be used by only one thread at a time in a program and the uniqueness of the thread must be ensured. Many non-reentrant functions return a pointer to static data. This can be avoided by returning dynamically allocated data or using caller-provided storage. An example of a non-thread safe function isstrtokwhich is also not re-entrant. The "thread safe" version is the re-entrant versionstrtok_r.Mutex Deadlock: This condition occurs when a mutex is applied but then not "unlocked". This causes program execution to halt indefinitely. It can also be caused by poor application of mutexes or joins. Be careful when applying two or more mutexes to a section of code. If the first pthread_mutex_lock is applied and the second pthread_mutex_lock fails due to another thread applying a mutex, the first mutex may eventually lock all other threads from accessing data including the thread which holds the second mutex. The threads may wait indefinitely for the resource to become free causing a deadlock. It is best to test and if failure occurs, free the resources and stall before retrying.

50Race ConditionsWhile the code may appear on the screen in the order you wish the code to execute, threads are scheduled by the operating system and are executed at random. It cannot be assumed that threads are executed in the order they are created. They may also execute at different speeds. When threads are executing (racing to complete) they may give unexpected results (race condition).

Producer while (true) { /* produce an item and put in nextProduced */ while (count == BUFFER_SIZE); // do nothing buffer [in] = nextProduced; in = (in + 1) % BUFFER_SIZE; count++;} 52Producer while (true) { /* produce an item and put in nextProduced */ while (count == BUFFER_SIZE); // do nothing buffer [in] = nextProduced; in = (in + 1) % BUFFER_SIZE; count++;} 53Race Conditioncount++ could be implemented as

register1 = count register1 = register1 + 1 count = register1count-- could be implemented as

register2 = count register2 = register2 - 1 count = register2Consider this execution interleaving with count = 5 initially:S0: producer execute register1 = count {register1 = 5}S1: producer execute register1 = register1 + 1 {register1 = 6} S2: consumer execute register2 = count {register2 = 5} S3: consumer execute register2 = register2 - 1 {register2 = 4} S4: producer execute count = register1 {count = 6 } S5: consumer execute count = register2 {count = 4}

54Thread Safe Code"thread safe".This means that there are no static or global variables which other threads may clobber or read assuming single threaded operation.

If static or global variables are used then protection must be applied or the functions must be re-written to avoid the use of these variables.

In C, local variables are dynamically allocated on the stack. Therefore, any function that does not use static data or other shared resources is thread-safe.

Thread-unsafe functions may be used by only one thread at a time in a program and the uniqueness of the thread must be ensured. Many non-reentrant functions return a pointer to static data.

DeadlockThread1 has a resource that Thread2 wants

while

Thread2 has a resource that Thread1 wants.

Operating Systems: A Modern Perspective, Chapter 6Process Manager ResponsibilitiesDefine & implement the essential characteristics of a process and threadData structures to preserve the state of the executionDefine what things threads in the process can reference the address space (most of the things are memory locations)Manage the resources used by the processes/threadsTools to create/destroy/manipulate processes & threadsTools to time-multiplex or schedule the process/threads in the CPU Tools to allow threads to synchronization the operation with one another Mechanisms to handle deadlock Mechanisms to handle protection 57

Waiting

Running FinishedReadyInitThread Creation

SchedulerResumes Thread Thread Exit

Thread Yields/Scheduler

Suspends ThreadThread Waits for EventEvent Occurs

e.g.,sthread_create()

e.g., sthread_yield()e.g.,

sthread_join()

e.g., sthread_exit()

e.g., other threadcalls

sthread_join()

State

GlobalVariables

Heap

Code

PerThreadState

Stack

SavedRegisters

Thread ControlBlock (TCB)

ThreadMetadata

Stack Information

PerThreadState

Stack

SavedRegisters

Thread ControlBlock (TCB)

ThreadMetadata

Stack Information

Shared

Programmer Abstraction Physical Reality

Threads

Processors1 2 3 4 5 1 2

Running Threads

Ready Threads

1 2 3 4 5 1 2 3 4 5

Programmers View

.

.

.x = x + 1;y = y + x;z = x +5y;

.

.

.

Possible Execution

#1...

x = x + 1;y = y + x;

z = x + 5y;...

Possible Execution

#2...

x = x + 1..............

thread is suspendedother thread(s) runthread is resumed

...............y = y + x

z = x + 5y

Possible Execution

#3...

x = x + 1y = y + x...............

thread is suspendedother thread(s) runthread is resumed

................z = x + 5y

Thread 1Thread 2Thread 3

Thread 1Thread 2Thread 3

Thread 1Thread 2Thread 3

a) One execution b) Another execution

c) Another execution

4.4 Implementation details 167

Logical ViewThread 1 Thread 2while(1){ while(1){

thread_yield() thread_yield()} }

Physical RealityThread 1s instructions Thread 2s instructions Processors instructionscall thread_yield call thread_yieldsave state to stack save state to stacksave state to TCB save state to TCBchoose another thread choose another threadload other thread state load other thread state

call thread_yield call thread_yieldsave state to stack save state to stacksave state to TCB save state to TCBchoose another thread choose another threadload other thread state load other thread state

return thread_yield return thread_yieldcall thread_yield call thread_yieldsave state to stack save state to stacksave state to TCB save state to TCBchoose another thread choose another threadload other thread state load other thread state

return thread_yield return thread_yieldcall thread_yield call thread_yieldsave state to stack save state to stacksave state to TCB save state to TCBchoose another thread choose another threadload other thread state load other thread state

return thread_yield return thread_yield... ... ...

Figure 4.13: Interleaving of instructions when two threads loop and call thread_yield().

Then, we will describe a few small additions needed to support multi-threaded processes.

Multi-threaded kernel with single-threaded processes

Figure 4.14 illustrates two single-threaded user-level processes running ona multi-threaded kernel with three kernel threads. Notice that each user-level process includes the processs thread. But, each process is more thanjust a thread because each process has its own address space process 1has its own view of memory, its own code, its own heap, and its own globalvariables that differ from those of process 2 (and differ from those of thekernel).