Experiments, Tests, and Datadspace.mit.edu/.../lecture-notes/11exp_dat_pres.pdf · Get feedback on your talk, because it will help. distribution of expertise level in the audience

Experiments, Tests, and Data

Massachusetts Institute of Technology, Subject 2.017

Purpose of Experiments and Tests

• Prove or Support a Hypothesis – The Earth’s diameter is 6500km.

– Multiple propellers on a single shaft can reduce cavitation (Turbinia).

– The archaea prokaryotes in the ocean fix carbon and consume other organisms, and the balance has profound impact on ocean uptake of CO2. (Ed Delong, Ann Pearson, etc.)

– Outriggers provide better roll stability than does a single hull in random beam seas, when wavelength is much larger than the beam.

• Prove a Capability, Support Design– Manned flight to the upper atmosphere can be achieved bi-weekly with a

specialized aircraft (X-Prize).

– Characteristic of lift force as a function of elevator aspect ratio and inflow angle.

– Delay calculation in pulsed 20kHz acoustic signals is possible with the TattleTale Model 8, and the performance obtained is XX.


• Does the work

stand up to

scrutiny?

–Use of controls

–Calibration

–Data quality

–Data processing

– Documentation and

record-keeping!


Controls

• Did you really measure what you thought?

• Rat Maze: Is the maze acoustically navigable?(R. Feynman)

• Mass Spectroscopy: When you put in a sample of known composition, are the other bins clean?

• When measuring electrical resistance, touch the probes together. Check a precision resistor too.

• Resonance in load measurement rigs?

• When measuring hull resistance, does zero speed give zero force?


Calibration

• More time can be spent on calibration than the rest of the experiment!

• Sensors should be calibrated and re-checked using independent

references, such as:

– Manufacturer’s specifications

– Another sensor with very well-known calibration ÅÆ – A tape measure, protractor, calipers, weights & balance, stopwatch, etc..

• Calibration range should include the expected range in the experiment.

• Some statistics of the calibration:

– Precision of fit (r-value or V)

– Linearity (if applicable)

• Understand special properties of the sensor, e.g., drift, PWM


Data and Sensor Quality

• Signal-to-Noise Ratio

(SNR): compares V to the

signal you want

• Repeatability/Precision: If

we run the same test again,

how close is the answer?

• Accuracy: Take the average

of a large number of tests –

is it the right value?


Time and Frequency Domain• Fourier series/transforms establish an exact

correspondence between these domains, e.g.,

X = VTcos( 2S�m t / T ) z(t) dt * 2 / T, m = 0,1,2,…m 0

Y = VT sin( 2S m t / T ) z(t) dt * 2 / T m 0

z(t) = X0 / 2 + 6 Xm cos( 2S m t / T ) + 6 Ym sin( 2S m t / T )


Time Resolution in

Sampled Systems

• The Sampling Theorom shows that the highest frequency that can be detected by sampling at frequency Z = 2S�'t is the Nyquist rate: ZN = Z s / 2.s

• Higher frequencies than this are “aliased” to the range below the Nyquist rate, through “frequency folding.” This includes sensor noise!

• The required rate for “visual” analysis of the signal, and phase and magnitude calculation is much higher, say ten samples per cycle.


Sample Statistics

• Sample mean m:

• Sample standard dev. V:

V = sqrt [ ( (x1-m)2 + (x2-m)2 + … + (xn-m)2 ) / (n-1) ]

• Error budgets for multiplication and addition

(VA is standard deviation of A):

(A + VA)(B + VB) ~ AB + AVB + BVA

Example: (1.0 + V0.2)(3.0 + V0.3) ~ 3.0 + V0.9

(A + VA) + (B + VB) = A + B + V(A+B)

Example: (1.0 + s0.2) + (3.0 + s0.3) = 4.0 + V0.5


Gaussian (Normal) Distribution

Probability Density Function f(x) ~ Histogram

f(x) = exp [ - (x-m)2 / 2V2 ] / sqrt(2S) / V

This is the most common distribution encountered in sensors and systems.

+/- 1V covers 68.3%

+/- 2V covers 95.4%

+/- 3V covers 99.7%

Area under f(x) is 1!


Filtering of Signals

Filterx xf

Use good judgement!

filtering brings out trends, reduces noise

filtering obscures dynamic response

Causal filtering: xf(t) depends only on past measurements – appropriate for real-time implementation

Example: xf(t) = (1-H)xf(t-1) + Hx(t-1)

Acausal filtering: xf(t) depends on all measurements – appropriate for post-processing

Example: xf(t) = [ x(t+1) + x(t) + x(t-1) ] / 3


A first-order filter transfer function in the freq. domain:

xf(jZ) / x(jZ) = O / (jw + O)

At low Z, this is approximately 1 (O�O)

At high Z, this goes to 0 magnitude, with 90 degrees phase lag (O/jZ = -jO�Z)

Time domain equivalent:dxf/dt = O (x – xf)

In discrete time, tryxf(k) = (1-O't) xf(k-1) +

O't x(k-1)


• BUT linear filters will not handle outliers very well!

• First defense against outliers: find out their origin and eliminate them at the beginning!

• Detection: Exceeding a known, fixed bound, or an impossible deviation from previous values. Example: vehicle speed >> the possible value given thrust level and prior tests.

• Second defense: set data to NaN (or equivalent), so it won’t be used in calculations.

• Third defense: try to fill in.

Example:

ifȱabs(x(k)ȱ– x(kȬ1))ȱ>ȱMX,

x(k)ȱ=ȱx(kȬ1)ȱ;end;

ÆLimited usefulness!


Presentations: Written

and Spoken

or

People will pay more attention to

you if you communicate well!


Sources and Ethics

• Somebody has almost certainly thought about what you are doing, and parts of it have almost certainly been solved.

• For specific items, you must give an original source and cite it properly.

• Refereed publications vs. flashy Internet postings.

• Plagiarism: Consider it ILLEGAL.

If there is any question about whether a phrase (or even a particular word) should be cited, protect yourself! … and the associated noise is

“systematically coupled to the

underlying process” [13]. …


Linearity

• Start at the beginning

and go to the end!

Antithesis: Michael

Ondaatji (The

English Patient)

• Flowchart or detailed

outline may help

• Omit needless

words*.

* Strunk, Jr., W. and E.B. White, 1972.

The elements of style. Allyn and

Bacon: Boston.


Introduction:

Approach:

Discussion:

• Bring reader from general to specific

• State hypothesis or objective

• Indicate why work is important

• Review prior work that applies

• etc

• How the experiment or test was designed

• Details of the apparatus or system

• Accuracy and precision issues

• etc

Results:

• Major Result A, with figures and description

• Major Result B

• etc

• Do results support hypothesis?

• Impact of findings

• Future work

• etc

Pointers on Speaking

The audience is here to see YOU, not

just your materials.

Smile and engage them!

Write out your talk so it is

clean from start to end.

Don’t lose anyone!

Practice your talk so you

are confident up there.

Get feedback on your talk,

because it will help.

distribution of expertise

level in the audience

“layman” “expert”

cumulative

average YOU average YOU

Prepare for questions. student (student) professional (professional)

90% of the talk accessible to

90% of the audience


A GOOD FIGURE > 1000 WORDSA bad figure is worth a few bad words


Vehicle trajectory: one

hidden independent

variable; five dependent

variables

Wind speed and direction as a

function of time. Top two plots

are combined into the bottom

plot: one independent

variable, three dependent

variables.


Figure from Principles of Naval Architecture, E. Lewis, ed.,

SNAME: New York, 1988. Original reference: Vossers, G., and

W.A. Swaan 1960. Some seakeeping tests with a Victory model.

Int. Shipbuilding Progress.

Image removed for copyright reasons.Figure and its caption from above-mentioned source.

Shows two independent

and one dependent

variable. Style shows the

effects of varying phase

and period.

Caption injured, and y-axis label missing; gives

three independent variables (length ratio, Froude

number, and heading to waves) and one

dependent variable (added power coefficient).


Documents

Experiments, Tests, and Datadspace.mit.edu/.../lecture-notes/11exp_dat_pres.pdf · Get feedback on your talk, because it will help. distribution of expertise level in the audience