September 8, 2009COMS W41561 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser [email protected]

September 8, 2009 COMS W4156 1

COMS W4156: COMS W4156: Advanced Software Advanced Software

EngineeringEngineeringProf. Gail KaiserProf. Gail Kaiser

[email protected][email protected]

http://bank.cs.columbia.edu/classes/http://bank.cs.columbia.edu/classes/cs4156/ cs4156/


Objectives of this courseObjectives of this course

• To introduce advanced concepts in To introduce advanced concepts in software engineering and their software engineering and their embodiment in selected methodologies embodiment in selected methodologies and technologiesand technologies

• To apply those concepts within a multi-To apply those concepts within a multi-iteration team software development iteration team software development projectproject


Topics covered in this Topics covered in this lecturelecture

• Overview of software engineeringOverview of software engineering• Overview of the courseOverview of the course• Pair design exercisePair design exercise• If time permits: Introduction to pair If time permits: Introduction to pair

programmingprogramming


Software Engineering as a Software Engineering as a DisciplineDiscipline


What is software?What is software?• Computer programs and associated Computer programs and associated

documentation such as requirements, design documentation such as requirements, design models, test plans and user manuals.models, test plans and user manuals.

• Software products may be developed for a Software products may be developed for a particular customer or may be developed for particular customer or may be developed for a general market.a general market.

• New software can be created by developing New software can be created by developing new programs, customizing generic new programs, customizing generic software systems and/or combining existing software systems and/or combining existing software.software.


What is software What is software engineering?engineering?

• Software engineering is an engineering discipline Software engineering is an engineering discipline that is concerned with all aspects of software that is concerned with all aspects of software production.production.– NOT just programmingNOT just programming– NOT just programming [part of] a large software systemNOT just programming [part of] a large software system– NOT just programming as a member of a large team NOT just programming as a member of a large team

• Software engineering is a systematic and Software engineering is a systematic and organized approach to software development organized approach to software development using appropriate tools and techniques depending using appropriate tools and techniques depending on the problem to be solved, the development on the problem to be solved, the development constraints and the resources available.constraints and the resources available.


What do software engineers What do software engineers do?do?

• Anyone can call themselves a software Anyone can call themselves a software engineer, but not all programmers are engineer, but not all programmers are software engineers (and not all software software engineers (and not all software engineers are programmers)engineers are programmers)

• Newly hired software engineers typically Newly hired software engineers typically assigned to test or maintain someone else’s assigned to test or maintain someone else’s pre-existing codepre-existing code

• Maintenance = Everything Maintenance = Everything afterafter initial initial deploymentdeployment– When was the last time you used a commercial When was the last time you used a commercial

or open source program labeled “Version 1.0”?or open source program labeled “Version 1.0”?– 50-90+% of total lifecycle costs50-90+% of total lifecycle costs


What do software engineers What do software engineers do?do?

• Categories of maintenanceCategories of maintenance– Corrective - reactive modification to correct Corrective - reactive modification to correct

discovered problemsdiscovered problems– Adaptive - modification to keep it usable in Adaptive - modification to keep it usable in

a changed or changing environmenta changed or changing environment– Perfective - improve performance or Perfective - improve performance or

maintainabilitymaintainability– Preventive - modification to detect and Preventive - modification to detect and

correct latent faultscorrect latent faults


What is the difference between What is the difference between software engineering and software engineering and

computer science?computer science?• Computer science is concerned with Computer science is concerned with

theory and fundamentals.theory and fundamentals.• Software engineering is concerned with Software engineering is concerned with

the practicalities of developing and the practicalities of developing and delivering useful software.delivering useful software.


Why Why studystudy software software engineering?engineering?

• Writing a Writing a programprogram is is easyeasy– Program ~= code (possibly with comments)Program ~= code (possibly with comments)

• Developing a Developing a software systemsoftware system is is harderharder– System ~= program plus technical documentation System ~= program plus technical documentation

sufficient such that someone other than original sufficient such that someone other than original developers can maintain and extenddevelopers can maintain and extend

• Developing a Developing a software productsoftware product is is very hardvery hard– Product ~= system plusProduct ~= system plus customers customers, fulfilling the , fulfilling the

business needs of those customers, with customer-business needs of those customers, with customer-oriented documentation and supportoriented documentation and support


Why Why studystudy software software engineering?engineering?

• To attain high-quality software products, software To attain high-quality software products, software engineers must be skilled professionals who engineers must be skilled professionals who follow defined processes and “best practices”follow defined processes and “best practices”

• Some schools offer entire degree programs in Some schools offer entire degree programs in software engineering (e.g., CMU, Drexel)software engineering (e.g., CMU, Drexel)

• At present there is no licensing of software At present there is no licensing of software engineers in the US, only specialty “certifications” engineers in the US, only specialty “certifications” offered by vendors (e.g., IBM, Microsoft, Sun) and offered by vendors (e.g., IBM, Microsoft, Sun) and by vendor-neutral organizations (e.g., IEEE, Linux by vendor-neutral organizations (e.g., IEEE, Linux Professional Institute)Professional Institute)

• Most other kinds of “engineer” must be licensed Most other kinds of “engineer” must be licensed by the state government, but licensing of software by the state government, but licensing of software engineers is very controversialengineers is very controversial


THIS Software Engineering THIS Software Engineering coursecourse


Course contentCourse content

• Software processSoftware process– Pair programming, requirements, modeling, Pair programming, requirements, modeling,

architecture, design patterns, refactoring, …architecture, design patterns, refactoring, …• Component-based software developmentComponent-based software development

– Component model frameworks Component model frameworks (COM+, .NET, EJB, Web Services)(COM+, .NET, EJB, Web Services)

• Quality assuranceQuality assurance– Code inspection, unit and integration testing, Code inspection, unit and integration testing,

stress testing, test planningstress testing, test planning


Why is this course called Why is this course called AdvancedAdvanced Software Software

Engineering?Engineering?• Until ~2002, undergraduate CS majors were required to Until ~2002, undergraduate CS majors were required to take COMS W3156 take COMS W3156 Introduction toIntroduction to Software Software Engineering, oriented towards sophomores and juniors, Engineering, oriented towards sophomores and juniors, whereas 4156 was an equivalent elective oriented whereas 4156 was an equivalent elective oriented towards seniors and graduate students towards seniors and graduate students

• Both courses presented a traditional view of the Both courses presented a traditional view of the software lifecycle focused on detailed documentation, software lifecycle focused on detailed documentation, independent of any particular implementation independent of any particular implementation technology, and stepped through the full lifecycle technology, and stepped through the full lifecycle (through delivery to “customers”)(through delivery to “customers”)

• 3156 replaced by COMS W3157 Advanced Programming3156 replaced by COMS W3157 Advanced Programming• 4156 now assumes introductory level software 4156 now assumes introductory level software

engineering background (preferably from the real world)engineering background (preferably from the real world)• Concentrates on recent trends towards component Concentrates on recent trends towards component

technology and emphasizes quality assurancetechnology and emphasizes quality assurance


Who am I?Who am I?• Gail KaiserGail Kaiser• [email protected][email protected] (note the +4156) (note the +4156)• Professor at Columbia since 1985Professor at Columbia since 1985• BS from MIT, MS and PhD from CMUBS from MIT, MS and PhD from CMU• Worked summers and/or consulted for startups, Worked summers and/or consulted for startups,

defense contractors, industry research labs, defense contractors, industry research labs, Software Engineering Institute, Educational Software Engineering Institute, Educational Testing ServiceTesting Service

• Research in software testing, collaborative work, Research in software testing, collaborative work, computer and network security, parallel and computer and network security, parallel and distributed systems, self-managing systems, Web distributed systems, self-managing systems, Web technologies, information management, software technologies, information management, software development environments and tools development environments and tools


Who are the TAs?Who are the TAs?

• Mr. Nipun Arora, third year graduate Mr. Nipun Arora, third year graduate studentstudent

• Mr. Palak Baid, second year graduate Mr. Palak Baid, second year graduate studentstudent

• Mr. Jonathan Bell, senior in SEASMr. Jonathan Bell, senior in SEAS


Who are you?Who are you?

• Graduate and upper level undergraduate students in Graduate and upper level undergraduate students in Computer Science, Computer Engineering, Computer Science, Computer Engineering, Biomedical Informatics, Electrical Engineering, IEOR, Biomedical Informatics, Electrical Engineering, IEOR, maybe othersmaybe others

• Sincerely interested (hopefully) in learning to Sincerely interested (hopefully) in learning to applyapply Software Engineering methodologies and technologies Software Engineering methodologies and technologies – not just programming skills – to developing useful – not just programming skills – to developing useful and usable software productsand usable software products

• 4156 is 4156 is requiredrequired for the CS MS Software Systems for the CS MS Software Systems track (unless you “import” a prior upper-level track (unless you “import” a prior upper-level software engineering course from another institution)software engineering course from another institution)


TextbooksTextbooks

• Required:Required: Ian Sommerville, Ian Sommerville, Software Software EngineeringEngineering, 8, 8thth edition, Addison- edition, Addison-Wesley, ISBN 9780321313799 Wesley, ISBN 9780321313799

• Recommended: Ron Patton, Recommended: Ron Patton, Software Software TestingTesting, 2, 2ndnd edition, Sams Publishing, edition, Sams Publishing, ISBN 0672327988ISBN 0672327988

• Ordered for this class through the Ordered for this class through the Columbia University Bookstore Columbia University Bookstore


Final Grade BreakdownFinal Grade Breakdown

• 50% Team Project50% Team Project– 2 iterations @ 25% each2 iterations @ 25% each

• 40% Individual Assessments [exams]40% Individual Assessments [exams]– 20% midterm20% midterm– 20% final20% final

• 10% Class participation and “pop quizzes”10% Class participation and “pop quizzes”• Assignments must be submitted via Assignments must be submitted via

CourseWorksCourseWorks


Team ProjectTeam Project

• Students work primarily in Students work primarily in pairspairs (2 people) (2 people)• Grouped into 2-pair Grouped into 2-pair teamsteams (4 people) (4 people)• Each team will invent, design, code and test a Each team will invent, design, code and test a

very smallvery small software system using some software system using some component model frameworkcomponent model framework

• 2 rapid iterations of project - each ~3 weeks 2 rapid iterations of project - each ~3 weeks longlong

• AllAll major system functionality should be (more major system functionality should be (more or less) operational by the end of the 1or less) operational by the end of the 1stst iteration iteration

• 22ndnd iteration will focus on quality assurance iteration will focus on quality assurance issuesissues


Pairs and TeamsPairs and Teams

• Pairs ideally self-chosen – find a partner (if Pairs ideally self-chosen – find a partner (if odd enrollment, odd enrollment, oneone triplet permitted but triplet permitted but nono onesies)onesies)

• Meet someone in class or advertise on the Meet someone in class or advertise on the 4156 discussion forum on CourseWorks4156 discussion forum on CourseWorks

• 2 pairs can request each other for a team2 pairs can request each other for a team• Unattached individuals and pairs will be Unattached individuals and pairs will be

assigned by the teaching staffassigned by the teaching staff• Each team will submit a proposal for the Each team will submit a proposal for the very very

smallsmall system they want to develop system they want to develop


Do the Team Project AS A Do the Team Project AS A TEAMTEAM

1.1. Develop and test your project code Develop and test your project code together with your pair partnertogether with your pair partner

2.2. Integrate together with your team’s Integrate together with your team’s other pairother pair

3.3. Co-author project documentation with Co-author project documentation with all team membersall team members

4.4. Demonstrate your project to the Demonstrate your project to the teaching staff teaching staff

All members of the same team All members of the same team mustmust use the same component use the same component model framework (e.g., EJB, .NET)model framework (e.g., EJB, .NET)


Project GradingProject Grading

• Team members who do not contribute Team members who do not contribute appropriately should be brought to the attention of appropriately should be brought to the attention of the teaching staff as soon as possible. the teaching staff as soon as possible.

• Such team members may receive a significantly Such team members may receive a significantly lower grade for the relevant project assignment(s) lower grade for the relevant project assignment(s) than the rest of that team, possibly "zero", may be than the rest of that team, possibly "zero", may be reassigned to another team, and/or may be asked reassigned to another team, and/or may be asked to withdraw from the course. to withdraw from the course.

• Team members who contribute “above and beyond Team members who contribute “above and beyond the call of duty” may receive a higher grade than the call of duty” may receive a higher grade than the rest of the teamthe rest of the team


Frequently Asked Frequently Asked QuestionsQuestions

• Do I have to join a pair/team?Do I have to join a pair/team?

YES!YES!• Can I do the project alone?Can I do the project alone?

NO!NO!• I work full time, I have a heavy class schedule, I have I work full time, I have a heavy class schedule, I have

a long commute, I don’t like working in groups, I a long commute, I don’t like working in groups, I prefer to program in an obscure language that no one prefer to program in an obscure language that no one else knows, etc. - Can I do the project alone?else knows, etc. - Can I do the project alone?

NO!NO!


Frequently Asked Frequently Asked QuestionsQuestions

• Do we have to use a component model framework?Do we have to use a component model framework?

YES!YES!• Can we use a component model framework not Can we use a component model framework not

covered in class?covered in class?

Yes, but only if it provides Yes, but only if it provides component services for component services for developing N-tier developing N-tier applications applications


Individual AssessmentsIndividual Assessments

• Midterm and Final Midterm and Final • Take-home examinations that must be Take-home examinations that must be

completed by each student working completed by each student working individuallyindividually

• Demonstrate your understanding of software Demonstrate your understanding of software engineering principles and technologies, and engineering principles and technologies, and how they are applied (graded)how they are applied (graded)

• Evaluate your own, your partner’s and your Evaluate your own, your partner’s and your team’s work on the project thus far (not team’s work on the project thus far (not graded, but contributes to project grade graded, but contributes to project grade differentials)differentials)


Class ParticipationClass Participation

• Attendance is requiredAttendance is required• Short “pop quizzes” will be given during Short “pop quizzes” will be given during

several class periodsseveral class periods• Please ask questions of general interest Please ask questions of general interest

in classin class


Final Grade BreakdownFinal Grade Breakdown

• 50% Team Project50% Team Project– 2 iterations @ 25% each2 iterations @ 25% each

• 40% Individual Assessments [exams]40% Individual Assessments [exams]– 20% midterm20% midterm– 20% final20% final

• 10% Class participation and “pop quizzes”10% Class participation and “pop quizzes”• Assignments must be submitted via Assignments must be submitted via

CourseWorksCourseWorks


Pair Design ExercisePair Design Exercise

(details announced in class)(details announced in class)


Pair ProgrammingPair Programming


What Is Pair What Is Pair Programming?Programming?

• TwoTwo software engineers work side-by- software engineers work side-by-side at side at oneone computer, sliding keyboard computer, sliding keyboard and mouse back and forth (or simulated and mouse back and forth (or simulated via desktop sharing technologies)via desktop sharing technologies)

• Continuously collaborate on same Continuously collaborate on same design, algorithm, code, test, etc.design, algorithm, code, test, etc.

• Continuous informal Continuous informal reviewreview


What Is NOT Pair What Is NOT Pair Programming?Programming?

• Splitting up the workSplitting up the work• Taking turns doing the workTaking turns doing the work• One person doing all the workOne person doing all the work


What About Productivity?What About Productivity?• Very controversial – common practice in some Very controversial – common practice in some

software development organizations and software development organizations and unthinkable in othersunthinkable in others

• Demonstrated in some studies to improve Demonstrated in some studies to improve productivity and quality of software products productivity and quality of software products under some circumstancesunder some circumstances– Particularly novice-novice pairs (vs. novice solos) or Particularly novice-novice pairs (vs. novice solos) or

expert-expert pairs (vs. expert solos)expert-expert pairs (vs. expert solos)– Two programmers work together nearly twice as fast Two programmers work together nearly twice as fast

and think of more solutions to a problem as two and think of more solutions to a problem as two working aloneworking alone

– While attaining higher defect prevention and defect While attaining higher defect prevention and defect removalremoval


BenefitsBenefits

• Increased discipline.Increased discipline. Pairing partners Pairing partners are more likely to "do the right thing" are more likely to "do the right thing" and are less likely to take long breaks. and are less likely to take long breaks.

• Fewer interruptions.Fewer interruptions. People are more People are more reluctant to interrupt a pair than they reluctant to interrupt a pair than they are to interrupt someone working alone.are to interrupt someone working alone.

• Resilient flow.Resilient flow. Pairing leads to a Pairing leads to a different kind of mental dynamic than different kind of mental dynamic than programming alone. programming alone.


BenefitsBenefits

• Better code.Better code. Pairing partners are less likely to Pairing partners are less likely to produce a bad design due to their immersion, and produce a bad design due to their immersion, and tend to come up with higher quality designs. tend to come up with higher quality designs.

• Multiple developers contributing to design.Multiple developers contributing to design. If If pairs are rotated frequently, several people will be pairs are rotated frequently, several people will be involved in developing a particular feature. This involved in developing a particular feature. This can help create better solutions, particularly when can help create better solutions, particularly when a pair gets stuck on a particularly tricky problem a pair gets stuck on a particularly tricky problem

• Collective code ownership.Collective code ownership. When everyone on a When everyone on a project is pair programming, and pairs rotate project is pair programming, and pairs rotate frequently, everybody gains a working knowledge frequently, everybody gains a working knowledge of the entire codebase. of the entire codebase.


BenefitsBenefits

• Mentoring.Mentoring. Everyone, even junior Everyone, even junior programmers, possess knowledge that programmers, possess knowledge that others don't. Pair programming is a way of others don't. Pair programming is a way of spreading that knowledge. spreading that knowledge.

• Improved morale.Improved morale. Pair programming can Pair programming can be more enjoyable for some engineers than be more enjoyable for some engineers than programming alone. programming alone.

• Team cohesion.Team cohesion. People get to know each People get to know each other more quickly when pair programming. other more quickly when pair programming. Pair programming may encourage team Pair programming may encourage team gelling. gelling.


DrawbacksDrawbacks• Experienced developers may find it tedious to Experienced developers may find it tedious to

tutor a less experienced developer in a paired tutor a less experienced developer in a paired environment. environment.

• A less experienced developer may feel intimidated A less experienced developer may feel intimidated pairing with a more experienced developer, which pairing with a more experienced developer, which may result in less participation. may result in less participation.

• Differences in coding style may result in conflict. Differences in coding style may result in conflict. • Personality conflicts can result in one or both Personality conflicts can result in one or both

developers feeling awkward or uncomfortable. developers feeling awkward or uncomfortable. • Some software engineers prefer to work alone, Some software engineers prefer to work alone,

and may find the paired environment and may find the paired environment cumbersome. cumbersome.


DrawbacksDrawbacks

• In the case where the team has slightly In the case where the team has slightly different work schedules, which is common in different work schedules, which is common in an environment that values work-life balance, an environment that values work-life balance, the pair is only available during the overlap of the pair is only available during the overlap of their schedules. Therefore, not only does it their schedules. Therefore, not only does it require more man-hours to complete a task, a require more man-hours to complete a task, a typical day has fewer pair-hours available, typical day has fewer pair-hours available, which further increases the overall task which further increases the overall task completion time. completion time.

• Where a company values telecommuting Where a company values telecommuting (working from home) or when an employee (working from home) or when an employee must work from outside the office for whatever must work from outside the office for whatever reasons, pair programming can be difficult. reasons, pair programming can be difficult.


Final NotesFinal Notes


First Assignment Due Next First Assignment Due Next Week!Week!

• Due Tuesday 15 September, 10amDue Tuesday 15 September, 10am• Posted on course websitePosted on course website• Submit via CourseWorksSubmit via CourseWorks• Pair FormationPair Formation


Upcoming DeadlinesUpcoming Deadlines

• Teams announced September 22Teams announced September 22ndnd • Team project concept due September Team project concept due September

2929thth • Project concept feedback by October 6Project concept feedback by October 6thth

First iteration begins October 6First iteration begins October 6thth


COMS W4156: COMS W4156: Advanced Software Advanced Software

EngineeringEngineeringProf. Gail KaiserProf. Gail Kaiser

[email protected][email protected]

http://bank.cs.columbia.edu/classes/http://bank.cs.columbia.edu/classes/cs4156/ cs4156/

Documents

September 8, 2009COMS W41561 COMS W4156: Advanced Software Engineering Prof. Gail Kaiser [email protected]