10 Aug 2010

ECE/BENG-492 SENIOR ADVANCED

DESIGN PROJECTMeeting #7

ECE-492 Meeting#7ECE-492 Meeting#7

Q1: Teams – let’s discuss your Design Reviews

Q2: Any questions about Design Document format and preparation?

TestingTesting

• Systems must be tested to ensure that they meet the engineering requirements

• Testing is used to understand system’s reliability and uncover design flows

• Frequently systems are tested for a variety of conditions and many combinations of conditions – but for complex systems, it’s almost impossible to test against all combinations of conditions

• Testing can be time consuming and costly but at the end testing saves money – (e.g., Hubble Telescope)

• The costs of corrections is lower before systems are deployed• Testing must be considered throughout system

implementation – such testing can save your project !

Testing PrinciplesTesting Principles

• Testing begins at a design process– Write test cases to check

requirements spec

– Write test plan while designing modules

– Write test plan for module integration

– Perform testing/debugging while implementing modules

• The Test-Vee illustrates this process

RequirementsSpecification

SystemDesign

Construct

Debugging

UnitTest

IntegrationTest

AcceptanceTest

Development Testing

Tes

t p

lan

; T

est

case

s

• Every level of design has a corresponding level of testing

• Notice added arrows – they emphasize that the left side develops a plan at a given level, while the right side executes the test plan accordingly

• An acceptance test must be written to test against requirements specification– You will need to develop test cases early next semester

• General benefits:– Test cases define exactly what the system/module must do

– Testing prevents “feature creep”

– The cases motivate developers by providing immediate feedback

– The cases force designers to think about extreme situations

– Test cases are a form of documentation

– Test cases force the designer to re-consider the design of the module before building it

Don’ts and Dos for ECE-492/3Don’ts and Dos for ECE-492/3

• Many ECE-492/3 projects are finished with– Very limited testing (NOT GOOD)

– A demo of “See, it works” attitude – THIS IS NOT ACCEPTABLE ! – (you succeeded because no smoke is visible?)

– Ad hoc testing plan implemented at the end (NOT GOOD)

– Very frequently the test plan is changed

– No data diagrams, statistical analysis, etc. (NOT GOOD)

• Advice– Spend a lot of time on testing – to develop a powerful statement “We

succeeded” rather than “It works”

– Collect data. Process data using statistical analysis. Show diagrams. Derive conclusions and discuss problems.

– Answer to: “Tell me when your system fails” - but use quantitative data, diagram of process test data, or similar. Don’t use intuitive explanations.

Types of TestTypes of Test

• Black box tests– No knowledge of internal organization

– These tests can be more challenging

– Only access to inputs and outputs

– Change inputs and observe outputs

• White box tests– Knowledge of internal organization

– Tests target specific internal nodes of the system to check that they are operating as expected

– Might have expectation of fault module

– Create test instance which reveals physical or logical errors

Constructing TestsConstructing Tests

• Four different types of tests shown in the Test-Vee figure– Debugging– Unit testing– Integration testing– Acceptance testing

• Additional motivation– Testing reduces the number of bugs in existing and new features– Tests reduce the cost of change– Tests improve design– Tests allow you to change and optimize components– Testing forces you to slow down and think more – Testing makes development faster– Tests reduce fear

Experimentation PlanExperimentation Plan

• You need to include a short experimentation plan in your Design Document – it’s required

• In the discussion focus on– Which functionalities you want to test– Method of testing leading to an answer “we succeeded”, “we are

close”, or “not so”– Designing experiments which will give you quantitative data; so you

can process and derive conclusions– Type of graphs demonstrating your test/processed data

• Do not write to ambitious plan (with many tests). Instead focus on few tests but explain them in greater detail. (Sometimes, less is more.)

CASE STUDY#1

< Testing >

CASE STUDY#1

< Testing >

Project ManagementProject Management

• Motivation– Engineers are regularly engaged in projects in their careers

– Middle management continues to shrink

– Industry now organizes more around projects than functions

– Engineers have led the way on project management, it is now “hot and trendy”

– According to a survey: #1 required skill for new engineers = Project Management Skills

• Project Management To complete the project:– On-time

– Within budget

– So that it meets the requirements

Work Breakdown StructureWork Breakdown Structure

• WBS is a hierarchical breakdown of the tasks and deliverables that need to be completed in order to accomplish the project objectives– ACTIVITY – a combination of a task and its associated deliverables

– TASK/CHECKPOINT – an action that accomplishes a job

– DELIVERABLE – an entity that is delivered to the project

• An ACTIVITY must have (see Table 10.1):– Definition of the work to be done

– Timeframe for completion of the activity

– Resources needed

– Person(s) responsible for the activity

– Predecessors – other activities to be completed before

– Checkpoints for monitoring the progress

Graphical Visualization of the Project

Graphical Visualization of the Project

• Gantt chart– Preferred method for project visualization

– See Figure 10.3

– Developed by Henry Gantt (1861-1919) – a bar graph representation of activities on a timeline

– Frequently, each activity must be complemented by a numeric Task Completion Rate (in percentages)

– Include “Milestones” – major in-progress demos given to your FS

ID

Task Name Start Finish DurationJan 2005 Feb 2005

1/16 1/23 1/30 2/6 2/13 2/20 2/27

1 32d2/22/20051/10/20051: Interface Circuitry

2 14d1/27/20051/10/20051.1: Design Circuitry

3 10d2/10/20051/28/20051.2: Purchase Components

4 8d2/22/20052/11/20051.3: Construct & Test Circuits

5 2d2/14/20052/11/20051.3.1: Current Driver Circuitry

6 3d2/15/20052/11/20051.3.2: Level Offset & Gain Circuitry

7 5d2/22/20052/16/20051.3.3: Integrate Components

8 23d2/9/20051/10/20052: LED & Driver Circuitry

9 1d1/10/20051/10/20052.1 Research A/D Converters

10

7d1/19/20051/11/20052.2 Complete Hardware Design

11

10d2/2/20051/20/20052.3 Purchase LED & Driver Components

12

5d2/9/20052/3/20052.4: Construct & Test

13

7d3/3/20052/23/20053: System Integration & Test

• Please use the following simplified version of Gantt charts

1. Robot hardware development

Task NameWeeks

1

1.1 Platform construction

2 3 4 5 6 7 8 9 10 11 12 13 14

CompletionRate

100%

100%

100%

100%

0%

65%

75%

0%

Reporting Period

Project Planning Chart

1.2 Motion system development

1.3 Short-range sensor system development

2. Motion control system development

2.1 Micro-processor system integration

2.2 Motion control software programming

2.3 Motion control system integration

3. Experimentation and testing

3.1 Testing robotic platform

3.2 Experiment #1

3.3 Experiment #2

4. Uprade #1 implementation

4.1 Long-range sensor system development

4.2 Long-range sensor integration

4.3 Testing and Experiment #3

5. Commercial applications analysis

6. Demonstration (milestones)

7. Reporting

100%

100%

100%

0%

0%

Guidance for creating WBSGuidance for creating WBS

• Build the plan after your design is almost complete

• Take the initial time estimates for activities and extend them almost by ~50%

• Assign a lot of time for testing and integration

• Factor in lead times for part ordering

• Do not assign all team members to all tasks

• Track the progress versus the plan

• Don’t become a slave to the plan

• Experience counts

• Next semester – you need to resubmit your revised WBS

CASE STUDY#1

< Work Breakdown Structure>

CASE STUDY#1

< Work Breakdown Structure>