Error, Bias and Confounding

Presenter: Dr. Mitasha SinghModerator: Dr. SK Raina

30.10.15

Error- Definitions • A false or mistaken result obtained in a study or

experiment• Random error is the portion of variation in

measurement that has no apparent connection to any other measurement or variable, generally regarded as due to chance

• Systematic error which often has a recognizable source, e.g., a faulty measuring instrument, or pattern, e.g., it is consistently wrong in a particular direction

(Last)

Random error• Divergence on the basis of chance alone of an

observation on a sample from the population from the true population values

• ‘random’ because on an average it is as likely to result in observed values being on one side of the true value as on the other side

• Inherent in all observations

• Can be minimized, but never avoided altogether

Sources of random error

1. Individual biological variation

2. Measurement error

3. Sampling error

Measurement error

• errors in measuring exposure or disease• Examples-o Blood Pressure measurement- estimates show that approximately

1/3rd of its observed variability is due to measurement error.

o Nutrient instruments- food records, 24 hour recalls and biomarkers

o Environment risk factors- laboratory and device error

o Hormone levels- lab error

o Tape incorrectly fixed to height boardo Scale consistently reads low by 0.5 kgo Failure to remove heavy clothing before weighingo Misleading questions

Sampling error

• Since inclusion of individuals in a sample is determined by chance, the results of analysis on two or more samples will differ purely by chance. (Last)

• Influenced by:– Sample size (Greater with smaller sample sizes)– Sampling scheme

Assessing the role of chance

1. Hypothesis testing2. Estimation

Hypothesis testing

• Start off with the Null Hypothesis (H0)

• the statistical hypothesis that one variable has no association with another variable or set of variables, or that two or more population distributions do not differ from one another

• the null hypothesis states that the results observed in a study, experiment or test are no different from what might have occurred as a result of operation of chance alone

(Last)

Statistical tests – errors

Null hypothesis (Ho)

(H0) false (H0) true

CONCLUSIONOF

STATISTICALTEST

SIGNIFICANT(H0) Rejected

NOT SIGNIFICANT(H0) Accepted

Type I( α ) error

Type II( β ) error

Power

Fletcher

Statistical tests - errors• Type I ( ) error: error of rejecting a true null α

hypothesis , i.e. declaring a difference exists when it does not

• Type II (β) error: error of failing to reject a false null hypothesis , i.e. declaring that a difference does not exist when in fact it does

• Power of a study: ability of a study to demonstrate an association if one exists

Power = 1- β

Estimation • Effect size observed in a particular study is

called ‘Point estimate’

• True effect is unlikely to be exactly that observed in study because of random variation

• Confidence interval (CI): interval computed with a given probability e.g. 95%, that the true value such as a mean, proportion, or rate is contained within the interval

Confidence intervals

If the study is unbiased, there is a 95% chance that that the interval includes the true effect size. The true value is likely to be close to the point estimate, less likely to be near the outer limits of that interval, and could (5 times out of 100) fall outside these limits altogether.

Fletcher

Precision

Precision in epidemiologic measurementscorresponds to the reduction of random error.

Rothman. Modern Epidemiology. 1986.

Dealing with error

• Increasing the sample size: sample size depends upon

- level of statistical significance ( error)α- Acceptable chance of missing a real effect ( error)β- Magnitude of effect under investigation- Amount of disease in population- Relative sizes of groups being compared

• Systematic quality control procedures to reduce measurement error.

Bias• Deviation of results or inferences from the truth, or

processes leading to such deviation. Any trend in the collection, analysis, interpretation, publication, or review of data that can lead to conclusions that are systematically different from the truth. (Last)

• A process at any stage of inference tending to produce results that depart systematically from true values. (Fletcher)

Relationship b/w Bias and Chance

Chance

Bias

Diastolic Blood Pressure (mm Hg)90

BP measurement(sphygmomanometer)

No.

of o

bser

vati

ons

Types of biases

1. Selection bias2. Measurement / (mis)classification bias

Selection bias

• Errors due to systematic differences in characteristics between those who are selected for study and those who are not.

(Last; Beaglehole)

• When comparisons are made between groups of patients that differ in ways other than the main factors under study, that affect the outcome under study. (Fletcher)

Examples of Selection bias

• Subjects: hospital cases under the care of a physician

• Excluded: 1. Die before admission – acute/severe disease.2. Not sick enough to require hospital care3. Do not have access due to cost, distance etc.• Result: conclusions cannot be generalized

(Last)

Examples: selection bias

• Respondents to study on ‘effects of smoking’ usually are not as heavy smokers as non-respondents hence they volunteer either because they are unwell, or worried about an exposure

• In a cohort study of newborn children, the proportion successfully followed up for 12 months varied according to the income level of the parents

Example: selection bias

• Question: association b/w formaldehyde exposure and eye irritation

• Subjects: factory workers exposed to formaldehyde

• Bias: those who suffer most from eye irritation are likely to leave the job at their own request or on medical advice

• Result: remaining workers are less affected; association effect is diluted

Measurement bias• Systematic error arising from inaccurate

measurements (or classification) of subjects or study variables. (Last)

• Occurs when individual measurements or classifications of disease or exposure are inaccurate (i.e. they do not measure correctly what they are supposed to measure)

(Beaglehole)

• If patients in one group stand a better chance of having their outcomes detected than those in another group.

(Fletcher)

Example: Measurement bias

Theoretical definition• Exposure: passive

smoking – inhalation of tobacco smoke from other people’s smoking

• Disease: Myocardial infarction – necrosis of the heart muscle tissue

Empirical definition• Exposure: passive

smoking – time spent with smokers (having smokers as room-mates)

• Disease: Myocardial infarction – certain diagnostic criteria (chest pain, enzyme levels, signs on ECG)

Example: measurement bias

• analysis of Hb by different methods (cyanmethemoglobin and Sahli's) in cases and controls.

• biochemical analysis of the two groups from two different laboratories, which give consistently different results

Example: measurement bias

• patients suffering from MI are more likely to recall and report ‘lack of exercise’ in the past than controls. (differences in accuracy or completeness of recall to memory of past events or experience.)

• Use of information taken from medical records to determine if women on birth control pills were at greater risk for thromboembolism than those not on pill. (women with thrombophlebitis, if aware of association b/w estrogens and thrombotic events, might report use of ocp more completely than women without phlebitis)

Accuracy

The degree to which a measurement, or anestimate based on measurements, representsthe true value of the attribute that is beingmeasured.

Last. A Dictionary of Epidemiology. 1988

Methods for controlling Selection Bias

During Study Design1. Randomization2. Restriction3. Matching

During analysis4. Stratification5. Adjustment

Dealing with measurement bias

1. Blinding- Subject- Observer / interviewer- Analyser 2. Strict definition / standard definition for

exposure / disease / outcome3. Equal efforts to discover events equally in

all the groups

Confounding 1. A situation in which the effects of two processes are not

separated. The distortion of the apparent effect of an exposure on risk brought about by the association with other factors that can influence the outcome

2. A relationship b/w the effects of two or more causal factors as observed in a set of data such that it is not logically possible to separate the contribution that any single causal factor has made to an effect

(Last)

Confounder … must be

1. Risk factor among the unexposed (itself a determinant of disease)

2. Associated with the exposure under study

3. Unequally distributed among the exposed and the unexposed groups

Examples: confounding

Smoking Lung cancer

Age If the average ages of the smoking and non-smoking groups are very different)

(As age advanceschances of lungcancer increase)

Examples: confounding

Alcohol intake

Myocardial infarction

sex

(Men are more likelyto consume alcoholthan women)

(Men are more at risk for MI)

Examples: confounding

Increased coffee drinking

Increased riskof pancreatic

cancer

Smoking

(many who smoke also drink coffee)

(cigarette smoking is a risk factor for for pancreatic cancer)

Example: multiple biases• Study: Association b/w regular exercise and risk of

CHD

• Methodology: employees of a plant offered an exercise program; some volunteered, others did not

coronary events detected by regular voluntary check-ups, including a careful history, ECG, checking routine heath records

• Result: the group that exercised had lower CHD rates

Example

• Selection: volunteers might have had initial lower risk (e.g. lower lipids etc.)

• Measurement: exercise group had a better chance of having a coronary event detected since more likely to be examined more frequently

• Confounding: if exercise group smoked cigarettes less, a known risk factor for CHD

Methods for controlling Confounders

During Study Design1. Randomization2. Restriction3. MatchingDuring analysis4. Stratification5. Statistical modelling

Bias• Systematic• Is due to mistakes which can be avoided at the planning

stage of study• Control and prevention requires careful attention

Error• Random • Never be completely avoided• Can be controlled by selecting appropriate sample size,

sampling method and precise measurements.

Thank you