After Work Statistics · Ethical Aspects –Sample Size too small • Low sample size causes...

U N I V E R S I T Ä T S M E D I Z I N B E R L I N

After Work Statistics

Institute of Biometry and Clinical Epidemiology

2

We are…

• … open and helpful!

• … active in the statistical methodologic research and in

medical research

• …active in teaching in many ways

Our Service Unit Biometry

• Free biometrical consulting for all medical research

projects, registration online

• “Statistik-Ambulanz” (Walk-in service): Consultation

without prior registration every Tuesday from 9am to 12pm

• Training in biometrical topics and statistical software

• Responsibility for project biometry within cooperation

For further information visit us online:

https://biometrie.charite.de/

Contact: Univ.-Prof. Dr. Geraldine Rauch (Head of Institute),

Institut für Biometrie und Klinische Epidemiologie (iBikE)

Standort Mitte (Charité Campus Mitte)

Reinhardstraße 58, 10117 Berlin

Standort Mitte (Charité Campus Klinik)

Rahel-Hirsch-Weg 5, 10117 Berlin

Slot Topic

1 So many tests! The agony of choice.

2 So many questions! Multiple testing.

3 So many patients? Sample size calculation.

4 What is it this odds ratio? Logistic regression.

5 Missing information? Dealing with missing data.

6 The right time? Survival analysis.

7 The variety of influences - Mixed models.

8 Who fits together? Patient matching.

1 So viele Tests! Die Qual der Wahl.

2 So viele Fragestellungen! Multiples Testen.

3 So viele Patienten? Fallzahlplanung.

4 Was ist dieses Odds Ratio? Logistische Regression.

5 Fehlende Information? Umgang mit fehlenden Daten.

6 Der richtige Zeitpunkt? Analyse von Ereigniszeiten.

7 Die Vielfalt der Einflüsse – Gemischte Modelle.

8 Wer passt zusammen? Matching von Patienten.

3

U N I V E R S I T Ä T S M E D I Z I N B E R L I N

So many patients?

Sample Size Calculation.

Prof. Dr. Geraldine Rauch

Institute of Biometry and Clinical Epidemiology

geraldine.rauch@charite.de

4

Sample Size Calculation

Are you still planning or already evaluating?

Planst Du noch oder wertest Du schon aus?

Have you ever bought furniture for self-assembly from IKEA?

What did you recognise?

5

There are always too many or too few screws!

Picture: Dtraveler www.flickr.com

6

What would happen if

... one is fixing up a cupboard with just 4 screws?

… one uses 360 screws to fix up a table?

© Inter IKEA Systems www.ikea.com

7

Ethical Aspects – Sample Size too small

• Low sample size causes insufficient power

• Even existing relevant differences (e.g. referring to efficacy) can’t be

discovered if the study is too small

• Patients are treated and exposed to study-specific risks, although

no clear result is expected.

Study is unrewarding.

But

weAber we

only

have

three

mices mices!

Photo: dpa-tmn www.rundschau-

online.de

8

Ethical Aspects – Sample Size too large

• Even for a lower sample size a clear statement can be expected.

• Every patient should get the best possible therapy.

• Patients recruited at a later stage are unnecessary getting a

less efficient therapy

9

Screw Calculation

The amount of screws must be large enough to fic the furniture but not

larger.

© Inter IKEA Systems www.ikea.com

Key-Message 1:

The sample size should be as large as necessary, but as

small as possible!

10!

How many screws do you need for…

… a cupboard?

… a table?

… a bed?

© Inter IKEA Systems www.ikea.com

11

Screw Calculation

There is no ideal number of screws being suitable for every kind of

furniture!

© Inter IKEA Systems www.ikea.com

Key-Message 2:

A sample size being suitable for all studies and research

questions does not exist.

12!

How many screws were used here?

13

Screw Calculation

14

More complex furniture usually needs more screws!

© Inter IKEA Systems www.ikea.com

Key-Message 3:

More complex study designs usually need larger sample

sizes.

!

Use help provided!

website: https://www.ikea.com/ms/en_AU/customer_service/ikea_services/assembly.html

15

Screw Calculation

16

For complex furniture, an assembly service may help.

© Inter IKEA Systems www.ikea.com

Key-Message 4:

For sample size calculation should consult an experienced

study biometrician!

!

When is the sample size calculation done?

Just a short biometrical question…

Quelle: https://pixabay.com/de/callcenter-telefon-service-hilfe-1015273/ /

17

When is the sample size calculation done?

Just a short biometrical question…

Quelle: https://pixabay.com/de/callcenter-telefon-service-hilfe-1015273/ /

Key-Message 5:

The sample size calculation is done prior to the start of a

clinical trial and is determined in the study protocol.

18!

2~1 1 x yn n

x y

X YT t

sn n

Influencing factors on the power of the t-test for

unpaired samples

Density function of the test statistic of the t-tests under H0 (blue) and H1 (red)

𝛿

𝜎2

𝑛𝑔𝑟𝑜𝑢𝑝

𝛼-error 𝛽-error

19

Type 1 error =a-error

Type 2 error = b-error

Power = 1- type 2 error

Relation between power & effect δ

The power depends on the size of the absolute effect

The more the true value differs to H0, that is the larger the

absolute effect , the higher the power.

The power increases with !

The larger the difference between

means, the higher the power

20

Relation between power & significance level α

The power depends on the size of the significance level a

The larger the significance level a, the higher the power.

The power increases with a !

The larger the significance level,

the larger the power

21

Relation between power & variance σ2

The power depends on the variance σ2 of the dependent variable.

The smaller the variance σ2, the larger the power.

The power decreases with σ2 !

The larger the variance of the

quantity tested, the smaller the

standardised effect and the smaller

the power

22

Relation between power & sample size n

The power depends on the sample size n.

The larger the sample size n, the higher the power.

The power increases with n!

The larger the sample size, the

more the curves are shifted and the

higher the power

23

Relation power/sample size

The power increases as the type I error 𝛼 increases,

(or the type 2 error decreases) as the true effect 𝛿 increases,

as the sample size 𝑛 increases,

as the variance 𝜎2 decreases.

Or vice verse

The sample size needed increases as the type I error 𝛼 decreases,

as the true effect 𝛿 decreases,

as the power 1 − 𝛽 increases,

as the variance 𝜎2 increases.

Assumptions of the variance σ2 and a (clinical) relevant minimum-effect δ should

be based on previous independent studies! Wrong assumptions of σ2 and δ can

have a key impact on the sample size!

24

Idea of the sample size estimation

Key-Message 6:

The sample size is calculated based on considering which number

of observations is needed, in order to prove a (clinical) relevant

minimum-effect 𝛿0 with a predefined power of (1 − b) for a

statistical test with a significance level 𝛼.

The formula for the sample size depends on the specific

statistical test to be used.

25!

Sample size estimation for the unpaired t-test

Approximate formula for the sample size for the unpaired t-test

(underestimates the actual sample size needed slightly):

𝑛𝑔𝑟𝑜𝑢𝑝 =2 ∙ 𝑧1−𝛼/2 + 𝑧1−𝛽

2

𝛿0𝜎

2

absolute effect 𝛿0 = 𝜇1 − 𝜇2

standardised effect 𝛿0/𝜎

standard deviation 𝜎

26

Example

Are the statements of the paper correct?

Literature: Chenchen Wang MD et al. (2010): A Randomized Trail of Tai Chi for Fibromyalgia, N Engl J Med

363, 743-754.

27

Idea of the sample size estimation

Key-Message 7:

A formal sample size calculation is only possible for studies with

a clearly fixed primary objective (usually confirmatory

studies).

28!

Example

Approximate formula for the sample size for the unpaired t-test

𝑛𝑔𝑟𝑜𝑢𝑝 =2∙ 𝑧1−𝛼/2+𝑧1−𝛽

2

𝛿0𝜎𝑝𝑜𝑜𝑙𝑒𝑑

2 =2⋅ 1.96+0.77 2

0.7 2 = 30.42 ≈ 31

Woohoo, it‘s

right!

29

Exact sample size with G*Power

http://www.gpower.hhu.de/

30

Exact sample size with G*Power

http://www.gpower.hhu.de/

31

Summary

• Sample size estimation needs to be done prior to the commencent

of the study.

• Sample size as large as necessary, but as small as possible

• Sample size (formula) depends on the staitical test used.

• Sample size estimation requires assumptions of 𝛼 and 1 − 𝛽, as

well as assumptions of 𝛿 and 𝜎

• Sample size estimation is not always simple.

32

Recommended Literature

33

Schumacher M, Schulgen G (2008, 3.

Auflage): Methodik klinischer Studien.

Springer. (Kapitel 10, insbesondere 10.1)