Upload
jeogilani
View
227
Download
0
Embed Size (px)
Citation preview
7/27/2019 Test Analysis Report Slides
1/18
Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Test Analysis
ReportDetailed information regarding the statistical
performance of an test
7/27/2019 Test Analysis Report Slides
2/18
Slide 2 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Overview Detailed information regarding the statistical
performance of an test Report Sections
First section presents information in a tabular format
Stats including mean, standard deviation and reliability
Second section: Distribution of tests scores using afrequency distribution table
lists all scores obtained by examinees
histogram graphically displays the range of scores
7/27/2019 Test Analysis Report Slides
3/18
Slide 3 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Test Analysis Report First section presents information in a tabular format
Stats including mean, standard deviation and reliability
7/27/2019 Test Analysis Report Slides
4/18Slide 4 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Test Analysis Report Second section: Distribution of tests scores using a
frequency distribution table lists all scores obtained by examinees
histogram graphically displays the range of scores
7/27/2019 Test Analysis Report Slides
5/18Slide 5 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Report Management and Distribution Distribute via Browser, PDF, Word, Excel, Email,
or Print My Reports
7/27/2019 Test Analysis Report Slides
6/18Slide 6Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Internal Consistency
Questionmark uses Cronbachs Alpha TestReliability
Reason its the gold standard:
Highly documented and very well researched (60+years)
Can be used with a wide variety of question types(partial credit and right/wrong)
Most widely used method in high and medium stakes
7/27/2019 Test Analysis Report Slides
7/18Slide 7 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Cronbachs Alpha Internal consistency test
reliability: A measure of how
well the assessment questionsrelate to one another tomeasure the same topic
Crucial information for legaldefensibility is the testreliablymeasuring what is wasdesigned to measure?
Speaks to the quality ofassessment questions: Are theyof high statistical quality?
)(
1 2
22
T
pT
S
SS
k
k
7/27/2019 Test Analysis Report Slides
8/18Slide 8 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Other Measures Skew
Mode
Kurtosis
Mean (arithmetic)
Median
Mode
Standard Deviation
Variance
Standard Error of measurement
Standard Error of the Mean
7/27/2019 Test Analysis Report Slides
9/18Slide 9 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.
Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
SkewA measure of the symmetry of the distribution
of scores are scores are pushed or skewed to one side or
the other?
Ranges from about -2 to +2
0
2
4
6
8
10
12
14
16
18
20
#
o
f
p
a
r
ti
c
i
p
a
n
t
s
0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%
Participant scores
Positive skew
0
2
4
6
8
10
12
14
16
18
20
#
o
f
p
a
r
ti
c
i
p
a
n
t
s
0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%
Participant scor es
Negative s kew
0
2
4
6
8
10
12
14
16
18
20
#
o
f
p
a
r
ti
c
i
p
a
n
t
s
0% 20% 40% 60% 80% 100%
Participant scores
Normal distribution (no skew)
7/27/2019 Test Analysis Report Slides
10/18
Slide 10 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Kurtosis A measure of the symmetry of a distribution of
scores How peaked/pointed versus flat are the distribution of
scores? What is happening at the tails?
Normal range from about -3 to +3
0
2
4
6
8
10
12
14
16
18
20
#
o
f
p
a
r
ti
c
i
p
a
n
t
s
0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%
Participant sc ores
Normal distribution (zero kurtosis: mesokurtic)
0
2
4
6
8
10
12
14
16
18
#
o
f
p
a
r
ti
c
i
p
a
n
t
s
0 % 1 0% 2 0% 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%
Participant sco res
Negative kurtosis (flat: platykurtic)
0
2
4
6
8
10
12
14
16
18
20
#
o
f
p
a
r
ti
c
i
p
a
n
t
s
0 % 1 0% 2 0 % 3 0% 4 0% 5 0% 6 0% 7 0% 8 0% 9 0% 10 0%
Participant scor es
Positive kurtosis (pointed: leptokurtic)
( )
)3)(2(
)1(3
)3)(2)(1(
)1( 24
+
nn
nz
nnn
nn
7/27/2019 Test Analysis Report Slides
11/18
Slide 11 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Mean (arithmetic) The most commonly used measure of central tendency
(refers to the middle of a distribution of scores)
Range of values depends on scores
7/27/2019 Test Analysis Report Slides
12/18
Slide 12 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Median Another measure of central tendency, less sensitive than
the mean to outliers
Range of values depends on scores Where 50% of participants obtained higher scores and
50% of participants obtained lower scores
7/27/2019 Test Analysis Report Slides
13/18
Slide 13 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Mode A third measure of central tendency, used a great deal in
survey analysis The most common score in a distribution of scores
Range of values depends on scores
34% 43% 56% 56% 56% 63% 67% 76% 88%
Mode = 56%
7/27/2019 Test Analysis Report Slides
14/18
Slide 14 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Standard deviation The spread or variation of scores between participants
Are the scores spread out (e.g., 0 to 100%) or clusteredtogether (e.g., all scores between 55 and 62%)
Range of values depends on scores
Ricks testscore = 75%
Sallys testscore = 83%
Marks testscore = 53%
Ellas testscore = 91%
Standard deviation = 16.36%
7/27/2019 Test Analysis Report Slides
15/18
Slide 15 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
VarianceAnother measure of variation
The first step in calculating a standarddeviation: The standard deviation is thesquare root of the variance
Range of values depends on scores
Used in some advanced calculations (e.g.,
Analysis of variance: ANOVA, Multiple analysisof variance: MANOVA)
7/27/2019 Test Analysis Report Slides
16/18
Slide 16 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Standard Error of
measurement
Theoretical test score = 66.4%
Theoretical test score = 63.7%
Theoretical test score = 67.1%
Theoretical test score = 65.8%
Theoretical test score = 67.5%
Product knowledge test
Theoreticalstandard
deviation =1.26%
1.26% oferror
surroundingRicks observed
scoreRicks observedscore = 66.1%
observed test score = theoretical true score + error
7/27/2019 Test Analysis Report Slides
17/18
Slide 17 Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Standard Error of the Mean
0
5
10
15
20
25
30
35
#
o
f
p
a
rt
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Participant score s
Sample (153 participants)
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
#
o
f
p
a
rt
i
c
i
p
a
n
t
s
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Participant score s
Population (87,000 participants)
Sample mean = 56.78%
Sample standard deviation = 15.21%Standard error of mean = 1.23%
___________________________within plus or minus 1 standard error
of the mean value, 68 times out of 100,the true population mean will reside
Typically the population informationis not known, but if you could see
the information
Population true mean = 57. 81%
7/27/2019 Test Analysis Report Slides
18/18
Copyright 1995-2007 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
More information:
www.questionmark.com