Interpreting Data MHO March 09

8/14/2019 Interpreting Data MHO March 09

1/48


2/48

Learning outcomes

Recap data collection and analysis

Appreciate psychometric properties

and their calculation Identify and interpret descriptive

statistics

Decipher basic inferential statistics

Explain some research jargon

Apply new knowledge to the evidence


3/48

Session plan


4/48

Task 1

Write down up to 5 issues that youthink are important wheninterpreting data

In a group of 4 discuss which youthink are most important and why?


5/48


6/48

Study aim

What is the research question lookingfor? Difference?

Correlation?

Linear association Prediction?

Regression Agreement?

Kappa

ICC


7/48


8/48

Nominal

The lowest and simplest level ofmeasurement

Used to classify or label people,

creatures, behaviour or events Uses categories that are mutually

exclusive e.g. male/female; dead/alive

Sufficient categories needed to allow

every observation to be assigned Often includes other category


9/48

Ordinal

Indicates a rank order in which things are arranged.

from the greatest to the least

the best to the worst

Allows presentation of the order of the observations

but does not provide information on actual values For example, Motor assessment scale, Barthel index,

FIM & FAM


10/48

Interval

This is a true unit of measure

Conveys the order of the observationsAND indicates the distance or degree ofdifference between the observations

Does not have an absolute zero

The difference between each score is

equalfor example, temperature (degrees

centigrade)


11/48

Ratio

Provides the most precise information ofall and includes the maximum amount ofinformation, again a true unit of measure

Has an absolute zero point that has realmeaning, therefore offers an absolutemeasure

The zero point dictates the absence of

the property measured eg. height, weight, speed


12/48

Type of data

P r o p e r t y

C a t e g o r i e s m u t

C a t e g o r i e s lo g i

Adapted from Puri 2002


13/48

Types of data

What type of data are the following? Age

Course of study

Social Class

Year

Weight

Height Profession

Adult shoe size

Pain / functional disability measure


14/48


15/48

Data analysis

The process of gathering,modelling, and transforming datawith the goal of highlighting usefulinformation, suggestingconclusions, and supportingdecision making


16/48

Data analysis

Descriptive statistics Used to describe the data in a sample,

e.g. mean, median, standard deviation.

Refer to any statistics textbook to gainan understanding of appropriate use

Inferential statistics

Infer findings from the sample to thepopulation


17/48

Descriptive statistics

Line

Bar

Histogram Pie chart

Scattergram

Box plot


18/48

Scattergrams

Two-dimensional representations of therelationship between pairs of variables,

The graph represents the points at whichthe two variables intersect for each casein the sample.

Easy visual representation of 3 aspects ofa pairwise relationship: Whether or not it is linear

Whether it is positive or negative The strength of the association. They can be useful aids to the understanding

of the idea of correlation


19/48

Scattergram example

H.A.D.S Anxiety

2520151050

age

atonset

90

80

70

60

50

40


20/48

Boxplots

Also known as a box-and-whiskerdiagram it is a convenient way ofgraphically depicting groups of

numerical data. Can be useful to display differencesbetween populations

The spacings between the different

parts of the box help indicate thedegree of dispersion Displays five summaries of the data


21/48

Boxplot example

111214321065945886521N =

H.A.D.S. Anxiety

20

17

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

0

Missing

age

atonset

100

90

80

70

60

50

40

30

324


22/48


Inferential statistics or statisticalinduction comprises the use ofstatistics to make inferencesconcerning some unknown aspect ofa population

Data can be categorised


23/48


Decisions which need to be made: Qualitative or quantitative? Difference or Correlation? Type of data:

Parametric continuous data

Non-parametric continuous data

Ordinal

Categorical

Number of groups in the sample Paired/ Un-paired data


24/48

Qualitative or quantitative?

Quantitative: Data may be represented numerically

Qualitative: Numerical representation is insufficient

Require words or even images

Examples include personal experiences,life story, perceptions


25/48

Difference or correlation?

Difference Self-explanatory!

Example: Is early mobilisation more

effective than deep breathing exercisespost operatively

Correlation

Looking for a relationship between variables Example: smoking cessation and

improvement in respiratory function


26/48

Parametric data

Conditions:

Interval or Ratio Data

Normally distributed:

Various ways to test if

data is normally

distributed: Mean/ 2 S.D'sKolmogorov-Smirnov

Shapiro-Wilk 66.00 67.00 68.00 69.00 70.00 71.00Height (ins)

250

500

750

1000

1250

Count


27/48

Paired/ un-paired data(Repeated measures)/ (Independent samples)

Paired data are often the result ofbefore and after situations - samemeasurement on the same person on 2

different occasions. Perceived stress level of students on

different programs of study.

Measurements of muscle strength before

and after an exercise to fatigue the muscle.

Attitudes of males and females tophysiotherapy.


28/48

Start

Paired data?

Parametric

t-test(related)

Nonparametric

Wilcoxon

Yes No

DifferencesorCorrelations

DifferencesCorrelations

Parametric

Pearson

Nonparametric

Spearman

Look for differences

Type of data

Chi-Squaretest.

Groups ofnumericaldata

number of groups

Morethan 2groups

2 groups

1 group

One samplet-test

Categoricaldata

Paired data?

Yes No

ParametricNon

parametric

RepeatedMeasuresANOVA

Friedman ParametricNonparametric

t-test(unrelated)

MannWhitney

Parametric Nonparametric

One-WayANOVA

Kruskal-Wallis


29/48

p-value

Output of an inferential statistical test

p (probability) value is used to assess howlikely the results we have obtained are due

to chance Conventionally set at 0.05, or 5% chance

that results obtained from sample are due tochance

This is arbitrary and open to criticism However, important concept to be aware of


30/48

Confidence intervals

This is how confident we are our samplerepresents the population

95% CI can be calculated for given datafrom our sample.

Usually presented in parenthesis eg CI =(O.8,2.7)

So this would mean that 95% of the timethe mean will be between 0.8 and 2.7

A narrow CI implies greater precision This result would be non-significant as

the CI does not cross 0.


31/48

Sample size calculation

Identifies the sample size required for study

Smaller samples show greater variance

Calculated from the primary outcome

measure and previous evidence of its SD inthe population being investigated

Takes into account the relative statisticalsignificance and the power of the study

Often the reason for a pilot RCT

Ethically important


32/48

Blinding

Single

the researcher knows the details of thetreatment but the patient does not

Double one researcher allocates a series of numbers to

'new treatment' or 'old treatment'. The secondresearcher is told the numbers, but not what

they have been allocated to.


33/48

Randomization

Involves the random allocation ofdifferent interventions (treatments orconditions) to subjects.

As long as numbers of subjects aresufficient, this ensures that bothknown and unknown confounding

factors are evenly distributedbetween treatment groups.


34/48


35/48

Psychometric properties

The elements that contribute to thestatistical adequacy of the study interms of Reliability

Validity

Internal consistency

Responsive to change


36/48

Reliability

Data is reliable if it has been shown to bereproducible with the same/similar results

Reliability is inversely proportional to random error

Types of reliability

A measure gives the same results on repeated tests byan individual ( if the respondent has not changed)

A measure gives the same result if different individualsapply it ( at the same time)


37/48

Inter rater reliability

Inter rater reliability is assessed by thedegree of agreement between the 2 sets ofscores

Often assessed using Pearson's or IntraClass Correlation Indicates the strength and direction of a linear

relationship between two random variables.

However, this correlation assesses associationbetween 2 measurers rather than agreement.

For Continuous data


38/48


39/48

Interpreting kappa

Kappa is always less than or equal to 1. A value of 1 implies perfect agreement and values

less than 1 imply less than perfect agreement.

Kappa can be negative. This is a sign that thetwo observers agreed less than would beexpected just by chance.

It is rare that we get perfect agreement.

Different people have different interpretationsas to what is a good level of agreement.


40/48

Responsiveness

Considers the ability to detectchange (that is meaningful topatient)

Simplest way to test is to correlatechange scores from the measurewith changes in other available

measures but is this responsivenessor just the ability to show change


41/48

Validity

The degree to which a test measureswhat it was designed to measure.

The degree to which a study supports

the intended conclusion drawn from theresults

Types of validity

internal external

May be recorded as convergent anddiscriminant validation


42/48

Validity

Many measures have multiple scaleswithin them considering differentconstructs

Ensuring the internal structure of themeasure is also construct validityand is measured through factoranalysis.

This looks at the patterns of itemswithin a measure that togetherassess a single underlying construct


43/48

Internal consistency

A measure usually has several items

Based on the principle that severalobservations are more reliable than one

The items need to be homogeneous One approach split items randomly into

2 halves and assess agreement

Cronbachs Alpha Coefficient estimates

the average agreement between allpossible ways of splitting the 2 halves.


44/48

Summary

Identify study aim What are they looking for

Check type of data collected Nominal, interval etc

Parametric, non-parametric

Are they using the appropriate test

Consider influencing factors Psychometric factors

Sample size, Blinding, Randomization


45/48

Task 2

In groups of four Design a study

Consider What you want to investigate

What you are measuring

What type of data you are collecting

What test would be appropriate inassessing the psychometric properties ofyour outcome measure


46/48

Intention to treat (ITT)

analysis An analysis based on the initialtreatment intent, not on the treatmenteventually administered.

ITT analysis is intended to avoidvarious misleading artifacts. For example, if people who have a more

serious problem tend to drop out at ahigher rate, even a completely ineffective

treatment may appear to be providingbenefits if one merely compares those whofinish the treatment with those who werenever enrolled in it.

.


47/48

Intention to treat (ITT)

analysis For the purposes of ITT analysis,everyone who begins the treatment isconsidered to be part of the trial,whether they finish it or not.

Full application of intention to treatcan only be performed where there iscomplete outcome data for allrandomized subjects.

Although intention to treat is widelycited in published trials, it is oftenincorrectly described and itsapplication may be flawed.


48/48

Summary

Recapped principles of study design,data collection and statisticalanalysis

Considered influencing factors

Applied knowledge to devise a studyinto the dunkability of biscuits

Reviewed how data may bepresented

Documents

Interpreting Data MHO March 09