Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
1
Project Management and Testing
Phases and Milestones
Budget and Resources
Resource Plan
Resource Plan
Activities and Deliverables
2
Introduction - Goal
Organization
Phases and Milestones
Budget and Resources
Risk
Releases
Project Plan
Project Plan
Activities and Deliverables
Project Plan
System Requirements
System Design
Quality Plan
Resource Plan
Configuration Management
Project Portfolio
Project Portfolio
System Test Plan
…
3
PROPS
managing
operative
supporting
Tollgate (TG0) decision to start prestudy
Requirement engineering
Interviews of customer and/or product management
Analysis
Business case analysis
Customers
Competitors
Costs
Benefits
Risks
Results in an assignment for a feasibility study TG1
Prestudy Phase
4
Tollgate (TG1) decision to start feasibility study
Requirement engineering
User-interface prototyping, specification, use cases, etc
System design
Anathomy, architecture, Implementation Proposals
Simulations using tools or role play
Project planning
Risk analysis
Estimation of size, effort, cost and schedule
Resources, competence, organization, etc
Life-cycle model (prototyping, evolutionary, waterfall, etc)
Results in an assignment for an implementation project TG2
Feasibility Phase
Black BoxRequirements
SystemDesign
SystemDevelopment
SystemTest
SystemValidation
FunctionalSpecification
ActorsUse CasesUse Case DiagramsStoryboardsRequirements Specification
Test PlanTest Cases
Diagrams:- Class- Sequence- Statechart- ActivityDesign PatternsSystem Design
Notes and DetailsSignaturesDesign Patterns
We need to manage the artifacts!
Resource PlanProject Portfolio
Milestones!- Activities- Deliverables- Resources- Responsibles- Dates
5
Builds
Relationship Between Use Cases, Iterations and Builds
Iteration 5
build 5.3
Use case 14
Use case 23
6
Relationship between Use Cases, Iterations and Builds
Iteration 5… 4 … 6
5.45.2 build 5.3
Use case 23
Use case 14
Use case 9 *
Use case 7
*
* «extends» or «includes»
*
Final Code Build and Integration Schedule: a Banking Example
week 31
week 23
weekly code builds biweekly daily
release
baseline
7
week 31
week 23
weekly code builds biweekly daily
releaseModule 1
Module 2
Module 3
Integrationof module
intobaseline
baseline
Plan evolutionary increments of the baseline!
activitytask
Final Code Build and Integration Schedule: a Banking Example
week 31
week 23
weekly code builds biweekly daily
releasebank query module
bank deposit module
bank withdrawal module
Integrationof module
intobaselinetasks
baseline
8
Typical Day-by-Day Code Integration Process
week 25
week 26
week 27
week 28
week 29
week 30
week 31
week 24
week 23
weekly builds biweekly daily
release= overnight regression test
development 6 pm 7 am
Runregression tests
time
development
Freeze development Confirm baseline orrevert to previous baseline
Resource Planning
Month 1
1 2 3 4
Month 2
1 2 3 4
Month 3
1 2 3 4
Month 4
1 2 3 4
Month 5
1 2 3 4
MilestonesRelease to production
Complete testing
Requirements
Design
Freeze requirements
Risk Analysis
2 2 2 3 2 2 4
2 2 2 1 1 1
4 4 4 3 3 4
4 4 4 4 3 3 4 4 4 4 4 3 3 4 4 4 4 4
4 4 4 4 4
4 4
4
Given team size:
To be assigned4
Halvacation
Karenvacation
9
Cyclic, Incremental or Evolutionary Planning
1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
1. strategy2. plan3. requirements4. design5. implementation6. test7. postmortem
MilestonesDelivery
1. strategy…
Iteration 1
Iteration 2
1. strategy…
Cycle 1
Week1
Cycle 2Cycle 3
1 1 1 1 1
Iteration 3
1. The total cost of the project,
e.g., increase expenditures
2. The capabilities of the product,
e.g., subtract from a list of features
3. The quality of the product,
e.g., increase the mean time between failure
4. The date on which the job is completed.
e.g., reduce the schedule by 20%
e.g., postpone project's completion date one month
The Variables of Project Management
10
Projekt
Schedule
Cost
Quality
Functionality
On time, within budget and meet requirements!
The Window of Opportunity
Bullseye Figure for Project Variables
cost
capability duration
defectdensity
Target :$70K
Target : 30 wks
Target : 4 defects/Kloc
Target:100%
11
Bullseye Figure for Project Variables
cost
capability duration
defectdensity
Target :$70K
Actual: 100%
Target : 30 wksTarget :
4 defects/Kloc
thisproject
Actual:1 defect/Kloc
Actual:20 wks
Actual:$90K
Target:100%
Coding versus Documenting
12
Urgent versus Important
Triage in Project Management
• Among top items in importance?– if so, place it in do at once category
• otherwise Ignore without substantially affecting project?– if so, place it in last to do category
• otherwise (Do not spend decision time on this)– place in middle category
13
Delegation
Do it youself!
Delegate Ignore?
Delegate and
follow up!Important
NotImportant
NotUrgentUrgent
Look after your own interests - but be a team player. You are a valuable resource in the company, and sometimes it is important to delegate tasks. However, don't delegate things that you find urgent and important - make sure you take responsibility and show initiative. However, protect yourself from small annoying tasks, unless you are told explicitly to leave other important tasks to focus on it. Even if we might not like it there is an hierarchy at work and you are studying a higher education and will be highly qualified (and well paid) and I'm guessing you will have trainees or other subordinates that are less paid and who can take on things you don't want to do.
14
If you are currently having a full workload (40 hours / week) don't accept additional work without letting the person know that this will delay your other projects. Make sure that your boss is alright with the delay and added cost before accepting a task. There is no shame in admitting that you are already working 100%. Swedish companies today actually try to avoid having people work overtime, at least in the IT sector. The reason is that a person in sick-leave is of no use for the development, it's better to keep developers healthy and happy. That's also why they will ask you about your leisure activities in a job interview - they want to know that you have a way to relieve stress etc. after work so that you can remain healthy at work.
More on Project Management…
• Project in Software Enginnering
• Or, most of the other 4th year projects!
• Dedicated Project Management courses at IES etc. Read them!
15
Assignment 6 – Resource Plan
• The resource plan should include information on which main project phases and activities that are planned (further requirements analysis, software architecture design, testing, etc) for the rest of the project and the amount of resources that are planned for each activity. It should also be expressed when these activies are planned to be active.
• Each main project activity should have a start and stop date (it's ok to say 'month1' or 'day30' instead of a real date, it may actually be preferable).
• Each main project activity should have a planned amount of manhours (1 manyear is 1800 manhours).
• Each main project activity should have a number of staff members allocated to the activity, with one team or activity leader.
• You will propable find that some staff members will need to double in some functions or work with more than one activity. This means that you need to plan the work so that a staff member will not be allocated too much or too less.
• Milestones etc should be visible in the plan.
• Remember that your investor is willing to invest 5 ”manyears” in the development of the company’s first system, so the plan should not exceed that ammount of work (note that 5 manmonths are already spent on your initial project portfolio...). How you plan to use the resources is up to you, but should be described in your resource plan.
• There are licenses of Microsoft Project, which you may use to produce Gantt charts etc if you so prefer. You can also use Microsoft PowerPoint or any general purpose software to do plans graphically.
• Keep It Simple! :-)
Testing...
16
Now
• Testing!– Unit Testing
• Black Box Testing
• White Box Testing
• Test Coverage
– Integration Testing
– System Testing
loss
Development Overview
Requirements
Architecture
System code
loss
loss
loss of information
Interface specsDetailed design
Function code
Module (e.g., package) code
loss
loss
Customer
OK?
17
Golden Rules of Testing
Goal of testing: maximize the number and
severity of defects found per dollar spent
– thus: test early
Limits of testing: Testing can only determine
the presence of defects, never their absence
– use proofs of correctness to establish “absence”
Test Plan
Testing
Sys Req
HL Design
LLDesign
Code
TestCases
TestCases
TestCases
TestCases
1h 800h80h8h
$
18
Artifacts and Roles for Integration Testing
Testengineer
Component engineer
Integration tester
System tester
Use-case model
Test case
Test procedure
Test evaluation
Test plan Test
component
. . . . . . . . . . . . . . . . . . responsible for . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Defect management
Testing: The Big Picture
Methods
Combinations of methods in class
Packages of classes
Use-cases & System Functions
Function
Module
Module combination
2. Integration tests
3. System tests
1.Unittests
19
Elaboration
Unified Process
Inception Construction Transition
Requirements
Analysis
Design
Implemen-tation
Test
Prelim.iterations
Iter.#m
Iter.#m+1
Integration tests
Iter.#1
Iter.#n
Iter.#n+1
…..
Unit tests
Iter.#k
…..
System tests
Black-box Testing
Black box… requirements
Actual outputcompared
withrequired
Input determinedby...
Result
20
White-box Testing
Input determinedby...
Result
White box
…designelements
Confirmationof expected
behavior
Black-, Gray-, & White-box Testing
Black box… requirements
Actual outputcompared
withrequired output
White box
Gray box… requirements &key design elements
Input determined by...Result
…designelements
Confirmationof expected
behavior
As for black-and white box
testing
21
Covering Every Statement is Not Sufficient
u>1 andv==0
x = x/u
u==2 orx>0
++x
No
Yes
Code attempt to implement flowchart
if ( (u>1) && (v==0) ) (1)x = x/u; (2)
if ( (u==2) || (x>3) ) (3)++x; (4)
u=2, v=0 and x=3• executes every line (1) - (4) • gives the correct output x= 2.5
However, line (3) is wrongNo
Yes
Required program
Testing Ranges: Elementary Cases
1. within range
2. at the boundaries
of the range
3. outside the range
(“illegal”)
range
Minimize what you need to testor you will never stop testing!
22
Perform Method Testing 1/2
1. Verify operation at normal parameter values(a black box test based on the unit’s requirements)
2. Verify operation at limit parameter values(black box)
3. Verify operation outside parameter values(black box)
4. Ensure that all instructions execute(statement coverage)
5. Check all paths, including both sides of all branches (decision coverage)
6. Check the use of all called objects7. Verify the handling of all data structures8. Verify the handling of all files
Perform Method Testing 2/2
9. Check normal termination of all loops
(part of a correctness proof)
10. Check abnormal termination of all loops
11. Check normal termination of all recursions
12. Check abnormal termination of all recursions
13. Verify the handling of all error conditions
14. Check timing and synchronization
15. Verify all hardware dependencies
23
• VolumeSubject product to large amounts of input.
• UsabilityMeasure user reaction (e.g., score 1-10).
• PerformanceMeasure speed under various circumstances.
• ConfigurationConfigure to various hardware / software
e.g., measure set-up time.
• Compatibility-- with other designated applications
e.g., measure adaptation time.
• Reliability / AvailabilityMeasure up-time over extended period.
Types of System Tests 1
• SecuritySubject to compromise attempts.
• e.g., measure average time to break in.
• Resource usageMeasure usage of RAM and disk space etc.
• Install-abililtyInstall under various circumstances.
• measure time to install.
• RecoverabilityForce activities that take the application down.
• measure time to recover
• ServiceabililtyService application under various situations.
• measure time to service
• Load / StressSubject to extreme data & event traffic
Types of System Tests 2
24
Key Attributes for Usability Testing
• AccessibilityHow easily can users enter, navigate & exit?
• e.g., measure by average time taken to . . .
• ResponsivenessHow quickly does the application allow the user to
accomplish specified goals?• e.g., measure by average time taken
• EfficiencyDegree to which the number of required steps for
selected functionality is minimal• “minimal” deduced in theory • e.g., measure by minimal time / average time
• ComprehensibilityHow easy is the product to understand and use with
documentation and help?• e.g., measure time taken for standard queries
Alpha- and Beta- Releases
In-house and highly trusted users
– Multiplies testing
– Previews customer reaction
– Benefits third-party developers
– Forestalls competition
Selected customers
– Multiplies testing activity
– Gets customer reaction
Alpha
Beta
25
Roadmap for the Transition Iterations
2. Conduct alpha testing.
1. Plan alpha and beta testing.
• Prepare
• Distribute & install
• Carry out (users / customers)
• Gather defect reports
• Observe stopping criteria
• Correct defects3. Conduct beta testing.
• Define population
• Plan defect collection
• Identify stopping criteria
• Completing a particular test methodology– Complete the procedures of a method or tool.
• Estimated percent coverage for each category– predetermine percent of each & how to calculate– e.g., “95% statement coverage”
• Error detection rate– predetermine rate with given severity level – e.g., “2 medium severity defects or less per 100 hours of
operation”• Total number of errors found
– (if possible) computed from a percentage of remaining defects
– predetermine percent– e.g., “95% of estimated existing defects found ”
Stopping Criteria
26
Target: 95%
Stopping Criteria: Graphical Representation
time
Percentage tested or found
%
Stopping Criteria: Graphical Representation
Error detection rate
Errors foundper 1000 hrs
Target: <= 7 per1000 hrs for 4 weeks
time
27
ANSI/IEEE 829-1983 Software Test Documentation
1. Introduction
2. Test planitems under test, scope, approach, resources, schedule, personnel
3. Test design items to be tested, the approach, the plan in detail
4. Test casessets of inputs and events
5. Test procedures steps for setting up and executing the test cases
6. Test item transmittal reportitems under test, physical location of results, person responsible for transmitting
7. Test logchronological record, physical location of test, tester name
8. Test incident reportdocumentation of any event occurring during testing which requires further investigation
9. Test summary reportsummarizes the above
This is how far you will go in D7025E!
28
Project Plan
System Requirements
System Design
Quality Plan
Resource Plan
Configuration Management
Project Portfolio
Project Portfolio
System Test Plan
…
Future
• Guestlectures– Mikael Börjesson on Project Management and System Design
– Kåre Synnes on Extreme Programming & Code review
– Matthias Kranz on HCI
• Cancelled lecture: Wednesday 3/10
• Code review (workshop 1/10)– Prepare yourself by reading assignment
– (If you use different code, bring materials to the lecture and notify me about the source)
29
Agile workshop
• Agile workshop 2/10 and 8/10– Groups 1-3 on 2/10
– Groups 4-7 on 8/10
– Work will not be done in the standard groups• If someone wants to switch to the other date it is ok but let
me know.
Wednesday
• Quiz #2 on Wednesday. – Covers Design Patterns and this lecture
• Mikael Börjesson is teaching… I hope– Mikael is doing some dental work in the morning
– If Mikael is not teaching then I will talk• Rapid prototyping and Presentation techniques.
30
That’s all folks!
Thanks for your attention!
Range of Errors in Estimating Eventual Cost
Range of cost estimates after conceptualization phase,based on actual cost of $1
$1
Design $1
Implementation $1
Integration/Test $1
Requirementsanalysis $1
Conceptualizationphase Range of cost estimates after
requirements analysis phase25c
$4