Process

CSE421/521:Opera4ngSystemsKarthikDantu

SlidesadoptedfromCS162classatBerkeley,CSE451atU-WashingtonandCSE421byProfKosaratUB

Logis4cs–PriorAc4onItems

•  JoinPiazza•  Setupdevelopmentenvironment:VirtualBox+Ubuntu16.04

•  Implementassignment#0andtestintheenvironment

•  Formgroups

Logis4cs-II

•  Assignment1out•  Recita4onsstartthisweek

– Wed10-10:50(NSC210)– Fri8-8:50(Park250)

•  Recita4on:BasicPintosdiscussionaswellasC/git/Unixtools

•  Scheduleuponwebsite–checkforconflicts!

Logis4cs–Newac4onItems

•  Assignment1out– Readthecode– Compile/test– Learnstructure–useprin%swhereyoucantounderstandflow

•  Testsampleprogramsfromclass

Recall:FourfundamentalOSconcepts•  Thread

–  Singleuniqueexecu4oncontext–  ProgramCounter,Registers,Execu4onFlags,Stack

•  AddressSpacewithTransla4on–  Programsexecuteinanaddressspacethatisdis4nctfromthememoryspaceofthephysicalmachine

•  Process–  Aninstanceofanexecu4ngprogramisaprocessconsis-ngofanaddressspaceandoneormorethreadsofcontrol

•  DualModeopera4on/Protec4on–  Onlythe“system”hastheabilitytoaccesscertainresources–  TheOSandthehardwareareprotectedfromuserprogramsanduserprogramsareisolatedfromoneanotherbycontrollingthetransla-onfromprogramvirtualaddressestomachinephysicaladdresses

6

ProcessConcept•  Processisaprograminexecu4on–  Aprocessimageconsistsofthreecomponents

–  anexecutableprogram–  theassociateddataneededbytheprogram

•  theexecu4oncontextoftheprocess,whichcontainsallinforma4ontheO/Sneedstomanagetheprocess(ID,state,CPUregisters,stack,etc.)

Typical process image implementation

Stallings, W. (2004) Operating Systems: Internals and Design Principles (5th Edition).

user address space

ProcessControlBlock(Assumesinglethreadedprocessesfornow)

•  Kernelrepresentseachprocessasaprocesscontrolblock

(PCB)–  Status(running,ready,blocked,…)–  Registers,SP,…(whennotrunning)–  ProcessID(PID),User,Executable,Priority,…–  Execu4on4me,…–  Memoryspace,transla4ontables,…

•  KernelSchedulermaintainsadatastructurecontainingthePCBs

•  Schedulingalgorithmselectsthenextonetorun

Recall:givetheillusionofmul4pleprocessors?

vCPU3vCPU2vCPU1

Shared Memory

•  Assumeasingleprocessor.Howdoweprovidetheillusionofmul4pleprocessors?– Mul4plexin4me!– Mul4ple“virtualCPUs”

•  Eachvirtual“CPU”needsastructuretohold,i.e.,PCB:–  ProgramCounter(PC),StackPointer(SP)–  Registers(Integer,Floa4ngpoint,others…?)

•  HowswitchfromonevirtualCPUtothenext?–  SavePC,SP,andregistersincurrentPCB–  LoadPC,SP,andregistersfromnewPCB

•  Whattriggersswitch?–  Timer,voluntaryyield,I/O,otherthings

vCPU1 vCPU2 vCPU3 vCPU1 vCPU2

Time

9

ProcessState

•  Asaprocessexecutes,itchangesstate–  new:Theprocessisbeingcreated–  ready:Theprocessiswai4ngtobeassignedtoaprocessor–  running:Instruc4onsarebeingexecuted–  wai+ng:Theprocessiswai4ngforsomeeventtooccur–  terminated:Theprocesshasfinishedexecu4on

Scheduler

•  Scheduling:Mechanismfordecidingwhichprocesses/threadsreceivetheCPU

•  Lotsofdifferentschedulingpoliciesprovide…–  Fairnessor–  Real4meguaranteesor–  Latencyop4miza4onor..

if ( readyProcesses(PCBs) ) { nextPCB = selectProcess(PCBs); run( nextPCB );

} else { run_idle_process();

}

ProcessCrea4on

•  Someeventsthatleadtoprocesscrea4on–  thesystemboots

•  whenasystemisini4alized,severalbackgroundprocessesor“daemons”arestarted(email,logon,etc.)

–  auserrequeststorunanapplica4on•  bytypingacommandintheCLIshellordouble-clickingintheGUIshell,theusercanlaunchanewprocess

–  anexis4ngprocessspawnsachildprocess•  forexample,aserverprocess(i.e.webserver,fileserver)maycreateanewprocessforeachrequestithandles

–  theinitdaemonwaitsforuserloginandspawnsashell

Pukngittogether:webserver

Request

Reply(retrieved by web server)

Client Web Server

Pukngittogether:webserver

Server

Kernel

Hardware

requestbuffer

replybuffer

11. kernel copy from user buffer to network buffer

Network interface Disk interface

12. format outgoing packet and DMA

6. disk request

10. network socket write

1. network socket read

2. copy arriving packet (DMA)

syscall

wait

interrupt

3. kernel copy

RTU

5. file readsyscall

8. kernel copy

RTU

7. disk data (DMA)

interrupt

4. parse request 9. format reply

Request Reply

ProcessTreeinLinux

Recall:3typesofKernelModeTransfer•  Syscall

–  Processrequestsasystemservice,e.g.,exit–  Likeafunc4oncall,but“outside”theprocess–  Doesnothavetheaddressofthesystemfunc4ontocall–  LikeaRemoteProcedureCall(RPC)–forlater– MarshallthesyscallIDandargumentsinregistersandexecutesyscall

•  Interrupt–  Externalasynchronouseventtriggerscontextswitch–  e.g.,Timer,I/Odevice–  Independentofuserprocess

•  TraporExcep4on–  Internalsynchronouseventinprocesstriggerscontextswitch–  e.g.,Protec4onviola4on(segmenta4onfault),Dividebyzero,…

User/Kernel(Privileged)ModeUser Mode

Kernel Mode

Full HW accessLimited HW access

exec

syscall

exitrtn

interrupt

rfi

exception

Implemen4ngSafeKernelModeTransfers

•  Importantaspects:– Separatekernelstack– Controlledtransferintokernel(e.g.,syscalltable)

•  Carefullyconstructedkernelcodepacksuptheuserprocessstateandsetsitaside– Detailsdependonthemachinearchitecture

•  Shouldbeimpossibleforbuggyormalicioususerprogramtocausethekerneltocorruptitself

NeedforSeparateKernelStacks•  Kernelneedsspacetowork•  Cannotputanythingontheuserstack(Why?)•  Two-stackmodel

–  OSthreadhasinterruptstack(locatedinkernelmemory)plusUserstack(locatedinusermemory)

–  Syscallhandlercopiesuserargstokernelspacebeforeinvokingspecificfunc4on(e.g.,open)

–  Interrupts(???)

Kernel Stack

running

main

User Stack proc1proc2

...

ready to run

mainproc1proc2

...

user CPU state

waiting for I/O

mainproc1proc2syscall

user CPU state syscallhandler

I/O drivertop half

Before

code:

foo () { while(...) { x = x+1; y = y-2; }}

SS: ESP

User-level Process

CS: EIPEFLAGS

otherregisters:EAX, EBX,...

Registers Kernel

code:

handler() { pusha ...}

stack:Exception Stack

During

code:

foo () { while(...) { x = x+1; y = y-2; }}

SS: ESP

User-level Process

CS: EIPEFLAGS

otherregisters:EAX, EBX,...

Registers Kernel

code:

handler() { pusha ...}

stack:Exception Stack

SSESP

EFLAGSCSEIP

error

KernelSystemCallHandler•  Vectorthroughwell-definedsyscallentrypoints!

–  Tablemappingsystemcallnumbertohandler•  Locatearguments

–  Inregistersoronuser(!)stack•  Copyarguments

–  Fromusermemoryintokernelmemory–  Protectkernelfrommaliciouscodeevadingchecks

•  Validatearguments–  Protectkernelfromerrorsinusercode

•  Copyresultsback–  Intousermemory

Hardwaresupport:InterruptControl•  Interruptprocessingnotvisibletotheuserprocess:– Occursbetweeninstruc4ons,restartedtransparently– Nochangetoprocessstate– Whatcanbeobservedevenwithperfectinterruptprocessing?

•  InterruptHandlerinvokedwithinterrupts‘disabled’–  Re-enableduponcomple4on– Non-blocking(runtocomple4on,nowaits)–  PackupinaqueueandpassofftoanOSthreadforhardwork

•  wakeupanexis4ngOSthread

Hardwaresupport:InterruptControl•  OSkernelmayenable/disableinterrupts

– Onx86:CLI(disableinterrupts),STI(enable)– Atomicsec4onwhenselectnextprocess/threadtorun

– Atomicreturnfrominterruptorsyscall

•  HWmayhavemul4plelevelsofinterrupt– Maskoff(disable)certaininterrupts,eg.,lowerpriority

– CertainNon-Maskable-Interrupts(NMI)•  e.g.,kernelsegmenta4onfault

InterruptController

•  Interruptsinvokedwithinterruptlinesfromdevices•  Interruptcontrollerchoosesinterruptrequesttohonor

–  Maskenables/disablesinterrupts–  Priorityencoderpickshighestenabledinterrupt–  SopwareInterruptSet/ClearedbySopware–  Interruptiden4tyspecifiedwithIDline

•  CPUcandisableallinterruptswithinternalflag•  Non-MaskableInterruptline(NMI)can’tbedisabled

Network

IntID

Interrupt

Interrupt Mask

ControlSoftwareInterrupt NMI

CPU

Priority Encoder

Tim

er

Int Disable

Howdowetakeinterruptssafely?•  Interruptvector

–  Limitednumberofentrypointsintokernel•  Kernelinterruptstack

–  Handlerworksregardlessofstateofusercode•  Interruptmasking

–  Handlerisnon-blocking•  Atomictransferofcontrol

–  “Singleinstruc4on”-liketochange:•  Programcounter•  Stackpointer•  Memoryprotec4on•  Kernel/usermode

•  Transparentrestartableexecu4on–  Userprogramdoesnotknowinterruptoccurred

Canaprocesscreateaprocess?•  Yes!Uniqueiden4tyofprocessisthe“processID”(orPID)•  fork()systemcallcreatesacopyofcurrentprocesswithanewPID

•  Returnvaluefromfork():integer– When>0:

•  Runningin(original)Parentprocess•  returnvalueispidofnewchild

– When=0:•  RunninginnewChildprocess

– When<0:•  Error!Musthandlesomehow•  Runninginoriginalprocess

•  AllstateoforiginalprocessduplicatedinbothParentandChild!– Memory,FileDescriptors(nexttopic),etc…

fork1.c#include <stdlib.h>#include <stdio.h>#include <string.h>#include <unistd.h>#include <sys/types.h>

#define BUFSIZE 1024int main(int argc, char *argv[]){ char buf[BUFSIZE]; size_t readlen, writelen, slen; pid_t cpid, mypid; pid_t pid = getpid(); /* get current processes PID */ printf("Parent pid: %d\n", pid); cpid = fork(); if (cpid > 0) { /* Parent Process */ mypid = getpid(); printf("[%d] parent of [%d]\n", mypid, cpid); } else if (cpid == 0) { /* Child Process */ mypid = getpid(); printf("[%d] child\n", mypid); } else { perror("Fork failed"); exit(1); } exit(0);}

fork2.c int status; … cpid = fork(); if (cpid > 0) { /* Parent Process */ mypid = getpid(); printf("[%d] parent of [%d]\n", mypid, cpid); tcpid = wait(&status); printf("[%d] bye %d(%d)\n", mypid, tcpid, status); } else if (cpid == 0) { /* Child Process */ mypid = getpid(); printf("[%d] child\n", mypid); } …

ProcessRaces:fork3.c

•  Ques4on:Whatdoesthisprogramprint?•  Doesitchangeifyouaddinoneofthesleep()statements?

int i;cpid = fork();if (cpid > 0) { mypid = getpid(); printf("[%d] parent of [%d]\n", mypid, cpid); for (i=0; i<10; i++) { printf("[%d] parent: %d\n", mypid, i); // sleep(1); } } else if (cpid == 0) { mypid = getpid(); printf("[%d] child\n", mypid); for (i=0; i>-10; i--) { printf("[%d] child: %d\n", mypid, i); // sleep(1); } }

UNIXProcessManagement•  UNIXfork–systemcalltocreateacopyofthecurrent

process,andstartitrunning–  Noarguments!

•  UNIXexec–systemcalltochangetheprogrambeingrunbythecurrentprocess

•  UNIXwait–systemcalltowaitforaprocesstofinish

•  UNIXsignal–systemcalltosendano4fica4ontoanotherprocess

•  UNIXmanpages:fork(2),exec(3),wait(2),signal(3)

UNIXProcessManagement

pid = fork();if (pid == 0) exec(...);else wait(pid);

pid = fork();if (pid == 0) exec(...);else wait(pid);

pid = fork();if (pid == 0) exec(...);else wait(pid);

main () { ...

}

exec

wait

fork

Shell

•  Ashellisajobcontrolsystem– Allowsprogrammertocreateandmanageasetofprogramstodosometask

– Windows,MacOS,Linuxallhaveshells

•  Example:tocompileaCprogramcc–csourcefile1.ccc–csourcefile2.cln–oprogramsourcefile1.osourcefile2.o./program

Signals–infloop.c#include <stdlib.h>#include <stdio.h>#include <sys/types.h>

#include <unistd.h>#include <signal.h>

void signal_callback_handler(int signum){ printf("Caught signal %d - phew!\n",signum); exit(1);}

int main() { signal(SIGINT, signal_callback_handler);

while (1) {}}

Recall:UNIXSystemStructure

User Mode

Kernel Mode

Hardware

Applications

Standard Libs

HowDoestheKernelProvideServices?

•  Yousaidthatapplica4onsrequestservicesfromtheopera4ngsystemviasyscall,but…

•  I’vebeenwri4ngallsortofusefulapplica4onsandInevereversawa“syscall”!!!

•  That’sright.•  Itwasburiedintheprogramminglanguagerun4melibrary(e.g.,libc.a)

•  …Layering

OSRun-TimeLibrary

OS

Proc 1

Proc 2

Proc n…

OS

Appln login Window Manager

…OS library OS library OS library

AKindofNarrowWaistCompilers

Web Servers

Web Browsers

Databases

Email

Word Processing

Portable OS Library

System Call Interface

Portable OS Kernel

Platform support, Device Drivers

x86 ARMPowerPC

Ethernet (1Gbs/10Gbs) 802.11 a/g/n/ac SCSI ThunderboltGraphics

PCIHardware

Software

System

UserOS

Application / Service

KeyUnixI/ODesignConcepts•  Uniformity

–  fileopera4ons,deviceI/O,andinterprocesscommunica4onthroughopen,read/write,close

–  Allowssimplecomposi4onofprograms•  find|grep|wc…

•  Openbeforeuse–  Providesopportunityforaccesscontrolandarbitra4on–  Setsuptheunderlyingmachinery,i.e.,datastructures

•  Byte-oriented–  Evenifblocksaretransferred,addressingisinbytes

•  Kernelbufferedreads–  Streamingandblockdeviceslooksthesame–  readblocksprocess,yieldingprocessortoothertask

•  Kernelbufferedwrites–  Comple4onofout-goingtransferdecoupledfromtheapplica4on,

allowingittocon4nue•  Explicitclose

I/O&StorageLayersHigh Level I/O

Low Level I/O Syscall

File System

I/O Driver

Application / Servicestreams

handles

registers

descriptors

Commands and Data Transfers

Disks, Flash, Controllers, DMA

Summary•  Process:execu4onenvironmentwithRestrictedRights

–  AddressSpacewithOneorMoreThreads–  Ownsmemory(addressspace)–  Ownsfiledescriptors,filesystemcontext,…–  Encapsulateoneormorethreadssharingprocessresources

•  Interrupts–  Hardwaremechanismforregainingcontrolfromuser–  No4fica4onthateventshaveoccurred–  User-levelequivalent:Signals

•  Na4vecontrolofProcess–  Fork,Exec,Wait,Signal

•  BasicSupportforI/O–  Standardinterface:open,read,write,seek–  Devicedrivers:customizedinterfacetohardware

Logis4cs–Newac4onItems

•  Assignment1out– Readthecode– Compile/test– Learnstructure–useprin%swhereyoucantounderstandflow

•  Testsampleprogramsfromclass