View
216
Download
1
Category
Preview:
Citation preview
Müller-Benedict: grade inflation and other challenges to grading
Grade inflation and other challenges facing grading
An analysis from a long-term perspective
Volker Müller-Benedict
Universität Flensburg
Oslo, Oktober 2015
Müller-Benedict: grade inflation and other challenges to grading
German: 1 2 3 4 5 6
Value: best -----------------------worst | failed
No central authority to rule the system of higher education: 16 federal states, each with autonomy to rule the educational
system Freedom of investigation and teaching as a basic law (Art 5.3): each
university can regulate teaching in its own way
The German grading scale
Norwegian: A B C D E F
2 of 24
Müller-Benedict: grade inflation and other challenges to grading
Diversity of examination regulations
The final grade may be calculated by different weights of the
component testings The final grade may be calculated by a different number of
component testings
The subdivisions of the final grades may be different:
sometimes only whole numbers 1, 2, 3, 4;
sometimes half numbers 1, 1.5, 2, 2.5,..
the majority allows thirds:1, 1.3, 1.7, 2, 2.3, 2.7, 3.0, 3.3, 3.7, 4.0
Leads to a diversity of examination regulations:
3 of 24
Müller-Benedict: grade inflation and other challenges to grading
Types of final degrees: • „Staatsexamen“ (state examinations: lawyers, clergymen,
teachers, physicians)• „Diplom“ (natural and social sciences), • „Diplom (FH)“ (universities of applied sciences)• „Magister“ (humanities)
Since Bologna:• BA Bacchelor• MA Master
For the admission to MA the grades of the BA exam are crucial.
Types of examinations (oral, written, possible repetitions etc.) may
be very different and depend on the specific examination regulations
of a university
Types of degrees and exams
4 of 24
Müller-Benedict: grade inflation and other challenges to grading
Previous investigations
Investigations in the U.S. state „grade inflation“ (increase of better grades without better performance). Some universities prescribe proportions of A‘s and B‘s.
Empirical findings:
In 2003, 2007 and 2012 the German „Wissenschaftsrat“published grades of all German universities. Remarkable differences in mean grades between the universities were detected, also a trend to grade inflation.
Betterment of grades can be• compliant to performance • independent of performance ( only then: grade inflation)
Leading question:
How comparable are grades, between disciplines, universities and over
time spans?
5 of 24
Müller-Benedict: grade inflation and other challenges to grading
Betterment of grades independent of performance
If the proportion of better performance, i.e. A‘s, increases, other performances being equal, the standard deviation remains the same. But:
How to detect grade inflation?
D
C
B
A
Grades in ‘math. teach.‘
Standard deviation of grades in ‘mathematics for teaching‘
3,33,0
2,6
2,2
1,8
1,0
Grades No Ge
6 of 24
Müller-Benedict: grade inflation and other challenges to grading
Research questions, theoretical distribution of grades
Research questions:• Is there grade inflation of German exams?• Are there stable differences in grades between disciplines and
universities?• How can we explain these facts? Are these trends independent of
performance?
Theoretical distribution of grades• ‘Absolute‘ anchor: the actual horizon of knowledge and competencies,
no relations to individual or group levels• Not ‘too simple‘ or ‘too difficult‘: distribution over whole scale
Necessary data:• Dense in the time dimension• Long-term, minimum 25 years
7 of 24
Müller-Benedict: grade inflation and other challenges to grading
Data and methods
Data sampled• about 1960 to 1997: of archives of 7 universities (by Thomas Gaens) • since 1996: electronic database of federal statistical bureau (distatis)• diploma, master of arts or state examination (BA‘s not before 2000)• group discussions with examiners (mostly university professors) (by
Elena Tsarouha)
Two parts:1. Descriptive results: long-term development
methods: graphics2. Analysis: Explanation of cycles and grade inflation
methods: time-series regression
Additionally:• Quotations from group discussions• Some possible recommendations
8 of 24
Müller-Benedict: grade inflation and other challenges to grading
grades – long-term trends of different disciplines (1)
(Smoothed by lowess (0.2))
grade inflation
cycles
1950
1952
1954
1956
1958
1960
1962
1964
1966
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
1.00
1.50
2.00
2.50
3.00
BWL
VWL
Deutsch LA
Mathematik LA
Chemie
Mathematik
Psychologie
Biologie
Busi. econ.Macroecon.Germ. teach.Math. teach.ChemistryMath. dipl.PsychologyBiology
9 of 24
Müller-Benedict: grade inflation and other challenges to grading
grades – long-term trends of different disciplines (2)
„[…] There are also scientific cultures, we didn’t yet speak about that, if for example I had to examine together with these actual colleagues her in [ univ. A], then I would probably, we would quarrel during the exam, about evaluations. Because they have other scientific paradigmas, which I can’t – say, handle, and reversely they not mine […]“
1959
1961
1963
1965
1967
1969
1971
1973
1975
1977
1979
1981
1983
1985
1987
1989
1991
1993
1995
1997
1999
2001
2003
2005
2007
2009
1
1.5
2
2.5
3
3.5
Jura Staatsex-amen
Sozi-ologie Magis-ter
Law: grades restricted by specific laws
Sociology
10 of 24
Müller-Benedict: grade inflation and other challenges to grading
grades – long-term trends of different universities (1)
(Smoothed by lowess (0.2))
Mathematics for teaching secondary level
1960
1962
1964
1966
1968
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
1.4
1.9
2.4
2.9
Göttingen
Braunschweig
Karlsruhe
Berlin
Tübingen
GöttingenBraunschweigKarlsruheBerlinTübingen
11 of 24
Müller-Benedict: grade inflation and other challenges to grading
grades – long-term trends of different universities (2)
(Smoothed by lowess (0.2))
German language Magister (not teaching)
„[…] What worries me most although is the difference of quality between [univ. A] and [univ. B]. There is no way. Who gets a 2 in [univ. A], recieves a 1 in [univ B] […]“
1970
1972
1974
1976
1978
1980
1982
1984
1986
1988
1990
1992
1994
1996
1998
2000
2002
2004
2006
2008
2010
1.4
1.9
2.4
2.9
Göttingen
Berlin
Tübingen
Heidelberg
Münster
Saarbrücken
GöttingenBerlinTübingenHeidelbergMünsterSaarbrücken
12 of 24
Müller-Benedict: grade inflation and other challenges to grading
grades – long-term trends of different types of exams
(Smoothed by lowess (0.2))
„I would say – in state examinations I was more severe, always more severe than in the old Magister. Why? I don’t know exactly but I would think I was more severe because I had a clearer impression of the demands on teaching German in school than I had of the demands on a Magister who can aspire such a lot of different careers “
1960 1970 1980 1990 2000 2010 20201
1.5
2
2.5
Math. dipl
Math.teach
Mathematicsstate examina-tion (for tea-ching in sec.schools)
Mathematics diploma
13 of 24
Müller-Benedict: grade inflation and other challenges to grading
Descriptive results - summary
1. Over four decades grade inflation is proceeding in most disciplines.
2. Remarkable stable differences of mean grades between the disciplines.
3. Considerable differences of mean grades between universities (same discipline), stable over one ore more decades.
4. Stable considerable differences between state examinations and universitarian exams (diploma, magister)
5. Mean grades are fluctuating with periods of different bandwidth around 20 years.
Part 2: Explanation of cycles and of grade inflation
14 of 24
Müller-Benedict: grade inflation and other challenges to grading
Causes and influences on grading
Differentiation between one-time and long-term influences:
• One-time: change of examination regulations, evaluations, options of choicing courses with „good grades“, change of examiners,…
• Long-term: relation student / professor (quality of teaching), intention of grading harsh or soft-gloved, social combination of the students, change of „teaching generations“,…=> are correlated with varying number of tested students (leading indicator)!
Many imaginable influences !
Cycles of the number of a) first semester, b) students in general, and c) of exams have same shape, but time-delayd by half a study length.
15 of 24
Müller-Benedict: grade inflation and other challenges to grading
Synchronous cycles of frequency
Frequency of mathematics state examinations (for teaching in sec. schools)
5 universities, absolute numbers 5 universities, standardized
Synchronous cycles: reaction to a common national job market for teachers with business cycles of about 25 years.
16 of 24
Müller-Benedict: grade inflation and other challenges to grading
Asynchronous cycles of frequency
Frequency of exams in German language (magister, not for teaching in schools, 5 universities standardized)
No common cycles, frequencies don‘t represent a common cause.
17 of 24
Müller-Benedict: grade inflation and other challenges to grading
Two distinct groups of academic careers
• Common national job market (teachers: only one employer, the state)• Clear job profile• Shortage and abundance of applicants are a theme of political and
public discussions• The perception of the job market can influence the examiners
1. Job market - dependent careers (e.g. mathematics for teaching)
• Diverse jobs possible • No common job market cycles• Shortage or abundance of applicants are notperceived by the public• Situation of the job market has no influence on examiners
2. Not job market - related careers (e.g. German language magister)
18 of 24
Müller-Benedict: grade inflation and other challenges to grading
Relation fequeny - grades (1)
Mathematics state examinations (for teaching in sec. schools), all universities
grades
frequency (shifted backward 3 years)
Frequency indicator of actual job market conditions
19 of 24
Müller-Benedict: grade inflation and other challenges to grading
Statistical results
Method: Prais-Winston-Regression
German language magister: same direction of counts and grades
Mathematics state examinations for teaching: reverse direction of counts and grades
Grade_Math_teach Coef. P>|t
Freq_Math_teach (Lead 3)
- 0.208 0.002
time - 0.030 0.061
constant +58.27 0.063
Grade_Germ_MA Coef. P>|t
Freq_Germ_MA +0.580
0.000
time - 0.105 0.000
constant +208.8 0.000
20 of 24
Müller-Benedict: grade inflation and other challenges to grading
Explanation of grade inflation
Why grade inflation? How cycles are related to growth?
Assumption: grades are more elastic in direction „better“ than in direction „worse“
Method:• First differences of data• Disaggregation of the timeseries of frequencies into two
timeseries „only growth“ and „only decrease“
Corollary:If the assumption holds, long term grade inflation follows as a consequence.
21 of 24
Müller-Benedict: grade inflation and other challenges to grading
Different elasticity of grades with respect to counts
Diff_Grade_Math_teach Coef. SE P>|t
Pos growth (shortage) - 0.377 0.057 0.000
Neg growth (abundance)
+0.023 0.067 0.731
Mathematics for teaching
Different significancies according to different effects of shortage or abundance to grades.
1970
1973
1976
1979
1982
1985
1988
1991
1994
1997
2000
2003
2006
-0.5
-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
0.5
In phases of shortage (red curve above the x-axis, frequency increasing) grades are improving (blue curve, marks decrease); in phases of abundance (green curve below the x-axis) grades remain stable or worsen only a bit.
22 of 24
Müller-Benedict: grade inflation and other challenges to grading
Summary
„[…] However my inner grade always was a little bit worse than the grade I gave, because particularly in the exam Magister…Because you have such big doubts, o god, how could those people get along? And if they were thrown into life with a bad grade of Magister additionally, then at the end they will become nothing[…].“
Student frequency is an „overloaded“ indicator, points to various conditions, recomendations are not obvious.
The long-term German data show• Relation of grades to the cyclical movement of student frequency.• Grades have tendency to improve because of less elasticity to
worsen than to improve.
23 of 24
Müller-Benedict: grade inflation and other challenges to grading
Summary
Because grades are exposed to many influences, which do not reflect the
performance, they should generall not be treated as that exact
number/letter, which appears on the certificate!
Thank you very much für your attention!
24 of 24
Recommended