MultiThreaded Concepts MultiThreaded Concepts and Programmingand Programming

Copyright, 1999 © Elbit Medical Imaging -- Tor Systems

Liran Zvibel

4.5.99

Organization of the Organization of the lecture:lecture:• Historical review.Historical review.

• What is a tread.What is a tread.

• Why and when to use threads.Why and when to use threads.

• When NOT to use threads.When NOT to use threads.

• Can threads cause problems?Can threads cause problems?

• Some programming techniques.Some programming techniques.

• How the hell do I debug threads !?!?!How the hell do I debug threads !?!?!

Subjects not includedSubjects not included

• Thread implementation in different Thread implementation in different OSes.OSes.

• Usage of threads in specific OS.Usage of threads in specific OS.

• Advanced programming techniques Advanced programming techniques and algorithms.and algorithms.

• Theory of deadlocks.Theory of deadlocks.

Historical ReviewHistorical Review

In the course for better system In the course for better system utilization:utilization:

• One job at a time systemsOne job at a time systems

• Batch systems with SPOOLingBatch systems with SPOOLing

• MultiProgramming / MultiTaskingMultiProgramming / MultiTasking

• MultiThreaded programmingMultiThreaded programming

Timesharing

Thread -- A definitionThread -- A definition

Thread (thred) Thread (thred) nn. very thin twist of wool, . very thin twist of wool, cotton, linen, silk, etc.; filament as of cotton, linen, silk, etc.; filament as of gold, silver; prominent spiral part of gold, silver; prominent spiral part of screw; consecutive train of thought.screw; consecutive train of thought.

-- -- New Webster’sNew Webster’s

Dictionary.Dictionary.

What is a thread?What is a thread?

A thread consists of:A thread consists of:

• A Program CounterA Program Counter

• Register SetRegister Set

• Stack SpaceStack Space

• PriorityPriority

• Thread IDThread ID

A thread shares with A thread shares with its peer threads:its peer threads:

• Code SectionCode Section

• Data SectionData Section

• Operating System Operating System resourcesresources

• Process IDProcess ID

A thread is a basic unit of CPU utilization.

What is MultiThreaded What is MultiThreaded programming?programming?

Basically multithreaded programming is Basically multithreaded programming is implementing your software so that two or implementing your software so that two or more activities can be performed in parallel more activities can be performed in parallel within the same application.within the same application.

Programmers should be aware of the OS Programmers should be aware of the OS rules and the way it governs the different rules and the way it governs the different threads, and how threads can interact with threads, and how threads can interact with the “outer world” (other threads/processes)the “outer world” (other threads/processes)

Thread SchedulingThread Scheduling

The OS schedules each thread in a RR The OS schedules each thread in a RR fashion. Each thread is allocated a fashion. Each thread is allocated a small amount of time called small amount of time called quantumquantum, to run on the CPU., to run on the CPU.

When switching between two threads When switching between two threads the OS performs a the OS performs a context switch context switch..

• Intra-process Context SwitchIntra-process Context Switch

• Inter-process Context SwitchInter-process Context Switch

Thread States:Thread States:

• RunningRunning

• ReadyReady

• BlockedBlocked

• SuspendedSuspended

Why write a Why write a MultiThreaded program?MultiThreaded program?

• Increased/Maximum ParallelizationIncreased/Maximum Parallelization

• Faster processingFaster processing

• Simplified designSimplified design

• Increased robustnessIncreased robustness

• Increased responsiveness to the userIncreased responsiveness to the user

• Better use of the CPUBetter use of the CPU

When NOT to use threadsWhen NOT to use threads

• You don’t have a really good reason You don’t have a really good reason to use threads. (Just for the heck of to use threads. (Just for the heck of it isn’t good enough…)it isn’t good enough…)

• The OS overhead would not make it The OS overhead would not make it worthwhile. If your thread does not worthwhile. If your thread does not perform a lot of work, the context perform a lot of work, the context switch does not worth it.switch does not worth it.

Can threads cause Can threads cause problems?problems?

• DeadlocksDeadlocks

• StarvationStarvation

• Destruction of data structuresDestruction of data structures

• Race conditions.Race conditions.

• Problems with synchronizationProblems with synchronization

Race ConditionsRace Conditions

thread Athread A

Fetch Fetch numnum into register into register

increment and returns increment and returns numnum

thread Bthread B

Fetch Fetch numnum into register into register

increment and returns increment and returns numnum

IncrementingIncrementing num = 5= 5 twicetwice

Final value ofFinal value of num is 6 and not 7is 6 and not 7

Destruction of Data Destruction of Data Structures, an exampleStructures, an example

Typedef struct tagGADGET

{

struct tagGADGET * pNext;

}

GADGET, *PGADGET;

PGADGET gpFreeGadgets = NULL;

gpFreeGadgets Gadget1 Gadget2 Gadget3

Destruction -- cont.Destruction -- cont.

PGADGET GetAGadget (void)

{

PGADGET pFreeGadget =

gpFreeGadget;

if (pFreeGadget) {

gpFreeGadget =

gpFreeGadget->next;

}

return pFreeGadget;

}

Void FreeAGadget

(PGADGET pGadget)

{

pGadget->next =

gpFreeGadget;

gpFreeGadget = pGadget;

}

Destruction -- Cont.Destruction -- Cont.

Now consider that there are two Now consider that there are two threads, thread A only calls threads, thread A only calls GetAGadget, GetAGadget, and thread B only calls and thread B only calls FreeAGadget.FreeAGadget.

Lets assume that the OS lets B work till Lets assume that the OS lets B work till the first assignment and then the first assignment and then schedules thread A, and then thread schedules thread A, and then thread B continues processing. B continues processing.

What Happens?What Happens?

Destruction -- Cont.Destruction -- Cont.

gpFreeGadgets Gadget1 Gadget2 Gadget3

FreeGadget Beginning of thread B

FreeGadget

gpFreeGadgets

Gadget1

Gadget2 Gadget3

FreeGadget

Gadget1gpFreeGadgets

Gadget2 Gadget3

A scheduled

B Finishes

MultiThreaded ConceptsMultiThreaded Concepts

• Atomic operationsAtomic operations

• Mutual Exclusion (Mutual Exclusion (mutex)mutex)

• Race ConditionRace Condition

• starvationstarvation

MT Concepts -- ContMT Concepts -- Cont

• SemaphoresSemaphores

• Priority InversionPriority Inversion

• DeadlocksDeadlocks

Solving the Mutex Solving the Mutex problemproblem

• Testandset : An atomic operation Testandset : An atomic operation that checks whether a pointer points that checks whether a pointer points to a location that contains 0, then to a location that contains 0, then changes to the requested value.changes to the requested value.

• Spinlock : Spinlock : while (testandset (value)) continue;;

The problems with The problems with SpinLocks:SpinLocks:

• Needs hardware assistanceNeeds hardware assistance

• Busy Waiting.Busy Waiting.

• Fairness (starvation)Fairness (starvation)

• Priority InversionPriority Inversion

Mutex solution in softwareMutex solution in softwareG.L. Peterson, 1981G.L. Peterson, 1981

enam {False = 0, True = 1};

int interested[2] = {False, False}, turn=0;

void EnterCriticalSection(int this_thread){

int other_thread = 1 - this_thread;

interested[this_thread] = True;

turn = this_thread;

while (turn == this_thread &&

interested[other_thread] ) ;

}

void LeaveCriticalSection(int this_thread) {

interested[this_thread] = Flase;

}

Semaphores Semaphores E.W.Dijkstra, 1965E.W.Dijkstra, 1965

Semaphores provide a method of synchronizing access to a shareable resource by multiple threads. Allowing for multiple concurrent access to a finite number of shared resources, where the semaphore value gets initialized to the number of shared resources. Each time a thread needs a resource, the semaphore value gets decremented.

Semaphores -- cont.Semaphores -- cont.

• Usually semaphores do not have a busy-Usually semaphores do not have a busy-wait implementation within the OS.wait implementation within the OS.

• The interface is easy to use, and allows The interface is easy to use, and allows for resource counting.for resource counting.

• Can be used as a simple lock (then called Can be used as a simple lock (then called binary semaphore -- the semaphore is binary semaphore -- the semaphore is initialized to 1)initialized to 1)

BarrierBarrierI.Carmon, 1988I.Carmon, 1988

A barrier is a way to force number of A barrier is a way to force number of threads to work together.threads to work together.

All the threads first have to wait till all All the threads first have to wait till all of them come to a certain point in of them come to a certain point in the code (the barrier), and when the the code (the barrier), and when the last thread arrives, all of the threads last thread arrives, all of the threads are set free, and continue working.are set free, and continue working.

Barrier -- ImplementationBarrier -- Implementation

typedef struct

{

lock in = 0,

out = 1;

int counter = 0;

} barrier;

Barrier(barrier *b, int n)

{ lock(b->in);

if(++b->count < n){

unlock(b->in);

lock(b->out);

}

if(--b->count > 0)

unlock(b->out);

else unlock(b->in);

}

Debugging Threads.Debugging Threads.

• Thinking in parallelThinking in parallel

• Remember that those bugs are Remember that those bugs are (mostly) not easily reproducible.(mostly) not easily reproducible.

• Know your code well!Know your code well!

• Use the Visual C++ Thread and Use the Visual C++ Thread and Callstack windowsCallstack windows

Debuggin Threads -- Cont.Debuggin Threads -- Cont.

• Know your primary thread.Know your primary thread.

• Know what thread you are currently Know what thread you are currently debugging.debugging.

• Watch for context switches while Watch for context switches while stepping.stepping.

• Time does not stand still in the Time does not stand still in the debuggerdebugger

The EndThe End

Any Questions?Any Questions?