Managing the Development of Large Software Systems

MANAGI NGTHEDEVELOPMENTOFLARGESOFTWARESYSTEMS Dr.Winston W. Rovce I NTRODUCTI ON lam going t odescribe mype,-.~onal views aboutmanaging large softwaredevel opments.Ihave had various assignments duri ngthepastnit,.:years,mostl yconcerned wi t hthedevel opment ofsoftwarepackages forspacecraftmissionplanning, commandi ngandpost-fl i ghtanalysis.IntheseassignmentsIhave experienced di f f erent degrees of successwithrespecttoarri vi ngatanoperati onalstate,on-ti me,and wi t hi ncosts.Ihave become prejudicedbymyexperiences andIamgoing torelatesome of theseprejudicesinthispresentation. COMPUTERPROGRAMDEVELOPMENTFUNCTI ONS There are t woessential stepscommontoallcomput erprogramdevel opments, regardless of sizeor compl exi t y. Thereis firstananalysis step,f ol l owedsecondbya codi ngstepas depi ctedinFigure1.Thissort ofverysimplei mpl ement at i on conceptisinfactallthatisrequirediftheef f ort is suffi ci entl ysmallandi fthe fi nal productis tobe operated bythose whobui l t it-asis t ypi cal l ydone wi t hcomput erprogramsfori nternaluse.Itis also theki ndofdevel opment ef f ort forwhi chmostcustomersarehappytopay,sincebothsteps involve genui nel y creative workwhi chdi rect l ycontri butestotheusefulness ofthefi nal product.An i mpl ement at i onplantomanufacture13rger softwaresystems,andkeyedonl ytothese steps,however,is doomed t of ai l ur e. Manyaddi ti onaldevel opment stepsare required,none cont ri but eas di rect l ytothefi nal productas analysis andcoding,andalldriveupthe devel opment costs.Customerpersonnel t ypi cal l ywoul drathernotpay forthem,anddevel opment personnel woul drathernoti mpl ementthem.Thepri mef unct i onofmanagement is toselltheseconceptstobot hgroupsandthenenforcecompl i ance onthepartofdevel opment personnel. ANALYSI S CODI NG Figure1.I mpl ement at i on stepstodel i ver a smallcomput erprogramf ori nternal operati ons. Amore grandiose approachtosoftware devel opment isi l l ustratedinFigure2.Theanalysis andcoding stepsare stillinthepicture,buttheyareprecededbyt wolevels of requi rementsanalysis, areseparated bya programdesignstep,andf ol l owedbya testingstep.These addi ti onsaretreatedseparately f romanalysis and codingbecause theyare di sti nctl ydi f f erent inthewaytheyare executed.Theymustbeplanned andstaffed di f f erent l yforbestut i l i zat i onofprogramresources. Figure3portraystheiterativerel ati onshi p between successivedevel opment phasesforthisscheme. The orderi ng of stepsis basedonthef ol l owi ng concept:thatas each stepprogressesandthedesignis further detai l ed, thereis ani terati on wi t hthepreceding andsucceeding stepsbutrarely wi t hthemoreremotestepsin thesequence.Thevirtueofall of thisis thatas thedesignproceeds thechangeprocessisscopeddownto manageable l i mi ts.At anypoi nt inthedesignprocess aftertherequi rements analysisiscompl etedthereexists a f i rmandc~seup~movi ng baseline towhi(:hto~t ur nintheevent of unforeseen design di ffi cul ti es.Whatwe have is aneffecti vefallbackposi ti onthattendstomaxi mi ze theext ent of earl y workthatis salvageable and preserved. ReprintedfromProceedings, IEEE WESCON, August1970,pages1-9. Co_pyright 1_9_70 by The Institute of Electrical and ElectronicsEt)gineers,,.328 Inc.Originally publishedby TRW. ISYSTEM IANALYSIS PROGRAM DESIGN I coo, . o TESTING IOPERATIONS Figure2.I mpl ement at i on stepstodevel op alarge computerprogramf ordel i very t oa customer. Ibelieve inthisconcept,butthei mpl ement at i on describedaboveis riskyandinvites fai l ure.The probl emisi l l ustratedinFigure4.Thetestingphase whi choccursattheend of thedevel opment cycleis the firstevent f orwhi chti mi ng,storage,i nput / out put transfers,etc.,are experienced as distinguishedf rom analyzed.These phenomena arenotprecisely analyzable.Theyarenotthesol uti ons tothestandardpartial di fferenti al equati ons ofmathemati cal physics forinstance.Yeti fthesephenomena failtosatisfythevarious externalconstraints,theni nvari abl y amaj orredesignisrequired.Asimpleoctalpatchorredo of someisolated code wi l l notf i xthese kindsof di ffi cul ti es.Therequireddesignchanges arel i kel ytobe so di srupti vethatthe softwarerequirements uponwhi chthedesignis basedand whi chprovides therati onal e foreverythi ng are violated.Eithertherequirementsmustbemodi f i ed,ora substantialchangeinthedesignisrequired.Ineffect thedevel opment processhas returnedtotheori gi nandone canexpectuptoal O0-percent overruninschedule and/orcosts. Onemi ghtnote thattherehas been a skipping-over of theanalysis andcodephases.Onecannot,of course,producesoftware wi t hout thesesteps,butgenerally these phasesaremanaged wi t hrelative ease and have l i ttl eimpactonrequirements, design,andtesting.Inmyexperience thereare whol e departments consumed wi t htheanalysis of orbi t mechanics, spacecraftat t i t udedetermi nati on,mathemati calopt i mi zat i on ofpayl oad acti vi tyandso f ort h, butwhenthese departments have compl etedthei rdi f f i cul t andcompl ex work,theresultantprogramstepsi nvol vea fewlines ofserial ari t hmet i ccode.Ifintheexecut i on ofthei rdi f f i cul tandcompl ex workthe analystshave made amistake,thecorrecti onisi nvari abl yi mpl emented byami nor change inthecode wi t hnodi srupti vefeedbacki ntotheother devel opment bases. However,Ibelieve thei l l ustratedapproach tobef undament al l y sound.Theremainder ofthis discussionpresents fiveaddi t i onalfeatures thatmustbe added tothisbasic approach toel i mi nate mostof the devel opmentrisks. 329 ISYSTEM! REQUIREMENTSIBI~ ~"' i so,w.,~I ANALYSIS ~1111~IIpRI~OGRAM~l l l ICODINGIi TESTING OPERATIONS Figure 3.Hopefully, the ~terat=veinteract=onbetweenthe various phasesis confined to successivesteps. ISYSTEM"1 .~oo,.~-,..Sl.,~ Iso,w..~!. IANALYSIS PROGRAM DESIGN I coo,.G I, ~ ! TESTINGI I O.ATO.S! Figure 4.Unfortunately,for the processillustrated, the design iterationsare neverconfined to the successivesteps. 330 STEP1:PROGRAMDESI GNCOMESFI RST The firststep towards a f i xisi l l ustratedinFigure5.Aprel i mi naryprogramdesign phasehas been inserted between thesoftwarerequirements generati on phaseandtheanalysis phase.Thisprocedurecanbe cri ti ci zedonthebasisthattheprogramdesigner is forcedtodesignintherelative vacuumofi ni ti al software requirements wi t hout anyexi sti ng anal ysi s..Asa result,hisprel i mi narydesignmaybe substanti al l yinerroras compared tohis designi fhe weretowai t unti l theanalysis was compl ete.Thiscriticismiscorrectbutitmisses thepoi nt.Bythistechni que theprogramdesigner assures thatthesoftware wi l l notfailbecauseofstorage, ti mi ng,anddataf l uxreasons.Astheanalysis proceeds inthesucceeding phasetheprogramdesignermust impose ontheanalystthestorage, ti mi ng,andoperati onalconstraintsinsucha waythathe senses the consequences.Whenhe j usti fi abl yrequiresmoreofthiski ndof resourceinordertoi mpl ementhisequations itmustbesi mul taneousl y snatchedf romhis analystcompatri ots.Inthiswayalltheanalysts andallthe programdesigners wi l l cont ri but etoameani ngfuldesignprocess whi chwi l l cul mi nateintheproperal l ocati on ofexecuti on ti meandstorageresources.Ifthetotal resources tobeappl i ed arei nsuffi ci entori ftheembryo operati onaldesignis wrongitwi l l berecognized atthisearlier stageandthei terati on wi t hrequhements and prel i mi nary designcanberedone beforefi nal design,codingandtestcommences. Howis thisprocedurei mpl emented?Thef ol l owi ng stepsare required. 1)Begin thedesignprocess wi t hprogram designers,notanalysts orprogrammers. 2)Design, defi ne andal l ocate thedataprocessing modes even attheriskof being wrong.Al l ocate processing, functi ons,designthedatabase,defi ne database processing,allocate executi on ti me,defi ne interfaces andprocessing modes wi t htheoperati ng system,describei nputandout put processing,anddefi ne prel i mi naryoperati ng procedures. 3)Writean overvi ew document thatisunderstandable, i nformati veandcurrent.Eachandevery workermusthave an el ementalunderstandi ng of thesystem.At least onepersonmusthave a deepunderstand- ing of thesystemwhi chcomesparti al l yf romhaving hadtowri t ean overvi ew document. /ALLOCATE~ADESCRIBE /sO..oOO,,./c% I Figure 5.Step1 :Insurethata prel i mi naryprogram designis compl etebefore analysis begins. 331 STEP2:DOCUMENTTHEDESIGN Atthispoi nt itis appropri ate toraisetheissue of-" howmuchdocument at i on?"Myownviewis "qui t eal ot ; "certai nl ymorethanmostprogrammers,analysts,orprogramdesigners are wi l l i ngtodoifl eftto thei rowndevices.Thefirstruleofmanaging softwaredevel opment isruthlessenforcementofdocument at i on requirements. OccasionallyIamcalledupontoreview theprogress ofothersoftwaredesignefforts.Myfirststepis toinvestigate thestate ofthedocument at i on, Ifthedocument at i onisinserious defaul tmyfirst recommendati onis simple.Replace projectmanagement.Stopallacti vi ti esnotrelated todocument at i on.Bringthedocument at i onuptoacceptable standards.Management ofsoftwareissi mpl yimpossible wi t hout a veryhighdegree ofdocument at i on. Asanexampl e, letmeofferthef ol l owi ngestimates forcompari son.In ordertoprocurea 5mi l l i ondol l arhardwaredevice,I woul dexpect thata30page speci fi cati onwoul dprovi de adequate detai l tocontrol theprocurement.Inordertopr ocur e5mi l l i ondollarsofsoftwareI woul destimate ~1[,00pa~e speci fi cati onis aboutrightinordertoachieve comparabl e control , Whysomuchdocument at i on? 1)Eachdesigner mustcommuni cate wi t hi nterfaci ngdesigners, wi t hhismanagement andpossibly wi t hthecustorner.Averbalrecordis t ooi ntangi bl e toprovi deanadequate basisforaninterfaceormanage- ment deci si on. Anacceptable wri t t endescri pti onforcesthedesigner totakeanunequi vocal posi ti onand provide tangi bl e evidence of compl et i on. Itprevents thedesignerfromhi di ngbehi ndt he- " l am90-percentf i ni shed"-syndromemont haftermonth. 2)Duri ngtheearlyphaseofsoftwaredevel opment thedocument at i oni . st hespeci fi cati onand i._~.sthe design.Unt i l codingbegins these threenouns(documentati on,speci fi cati on,design)denot easi ngt et hi ng. If thedocument at i onis badthedesignis bad.Ifthedocument at i on does notyetexistthereis as yetnodesign, onl ypeople t hi nki ngandtal ki ngaboutthedesign whi chis ofsomevalue, butnotmuch. 3)Therealmonetaryvalue ofgooddocument at i onbegins downstreaminthedevel opmentprocess duri ngthetestingphase andconti nues throughoperati ons andredesign.Thevalue ofdocument at i oncanbe describedintermsofthreeconcrete,tangi bl e si tuati ons thateveryprogrammanager faces. a)Duri ngthetestingphase,wi t hgood document at i onthemanager canconcentratepersonnel onthe mistakesintheprogram.Wi t hout good document at i oneverymistake,large orsmall,is anal yzed byoneman whoprobabl ymade themistakeinthefirstplace because heis theonl ymanwhounderstands theprogram area. b)Duri ngtheoperati onalphase, wi t hgood document at i onthemanager canuse operat i on-ori ent ed personnel tooperate theprogramandtodoa better job,cheaper.Wi t hout good document at i onthesoftware mustbe operatedbythose whobui l t it.General l y thesepeople arerel ati vel y disinterestedinoperati ons anddo notdoas effectivea j obas operati ons-ori ented personnel.Itshouldbepoi nt ed out inthisconnecti onthatin anoperati onalsi tuati on,ifthereis somehangup thesoftwareis always bl amedfirst.Inorderei thertoabsolve thesoftwareortof i xtheblame,thesoftwaredocument at i onmustspeakclearly. c)Fol l owi ng i ni ti al operati ons, whensystemi mprovementsareinorder,good document at i onpermits effectiveredesign,updating,andret rof i t t i nginthefield.Ifdocument at i ondoes not exist,general l y theenti re existing f rameworkof operati ng softwaremustbe j unked,even forrel ati vel y modestchanges. Figure6showsa document at i onplan whi chis keyedtothestepsprevi ousl y shown.Notethatsix documentsareproduced,andattheti meofdel i very of thefi nal product,DocumentsNo,1,No.3,No.4, No.5,andNo.6areupdatedandcurrent. 332 / ,I0: wZ /ooi,~ g~ Irl. . oi 0 .i IIII ~,-,,*,1= .~ illl~$~m z~_~u, E X E 8 " 0 IllN ~, .~- r" .2 /" z_,,,.~~E ~OLU a. . ~ N N I Z ,~,- w i - ,