Slide 1
Unequal outcomes: the role of school effectiveness in shaping
learning trajectories
Jack Rossiter, Education Research Officer
CIESAtlanta, 9 March 2017
1
1
Enrolment is not learningMeasuring what matters for learning is
not straightforwardI will review our approach to measuring
meaningful learning in Young Lives school surveysThen suggest how
this may be used to understand student learning trajectories and
for comparative analysis of school effectiveness
MAIN POINTS
Primary global net enrolment rate passing 90% (UIS)But what will
this mean for learning by 2030?FROM SCHOOLING TO LEARNING
Ref. SDGs and shift to learningNER a tricky statistic, somewhat
misleadingLink through from Rhiannons slides on enrolment trends
etc.Make transition to learning and ref. learning commission
3
Young Lives school component introduced in 2010No intervention:
understand, describe and explain using observational data2016-17:
upper primary (Ethiopia) and secondary (Vietnam and India)
gradesYOUNG LIVES SCHOOL EFFECTIVENESS SURVEYS
CountrySitesSchoolsStudentsIndia202059,820Vietnam14528,860Ethiopia306412,140
+ Peru, but with a different approach4
SCHOOL SURVEYS: EXAMPLE RESEARCH QUESTIONS
Benchmarking levels of attainment and progress in key
domains
What are students levels of performance in core curricular and
transferable skills domains?
How do students progress over time (in one academic year &
since previous surveys)?
What is the nature of inequalities in attainment & progress,
and how are these shaped over time?
Benchmarking function: shownAnalytic function: suggesting
hypotheses about relationship between student performance and
context, school, classroom/teacher and student factorsAnalytic
function: identifying cognitive and content domains in which
students have particular strengths or weaknesses.5
Defining learning quality at upper primary and secondary levels
is complex: more than the development of basic skillsMeaningful
learning: not only acquiring knowledge, but also being able to use
knowledge in a variety of new situations (e.g. Mayer 2002)Equip
young people with skills for future labour market or higher
education opportunities (World Bank 2009) more on that
laterASSESSING MEANINGFUL LEARNING: MATHS
Not to suggest that maths is all that matters far from all that
matters in our surveys Maths: Just one of many curricular
competencies, but achievement in basic school subjects is
sufficiently important to figure out how such outcomes are best
accomplished (Scheerens, 1999)
6
ASSESSING MEANINGFUL LEARNING: MATHSNumber sense and basic
number competencyIntegers and Rational numbers, Powers and
BasesFractions, Decimals, Ratios and PercentagesArea and Perimeter,
Volume and Surface Area
Basic Shapes, Geometry and Visual estimationAlgebra: concepts
and applicationsMeasurement, Data Interpretation, Analysis and
GraphsProblem solving and Applications in daily life
Appropriate content domains
Led by curriculum in each country e.g. Ethiopia Minimum Learning
Competencies
Cross-country cognitive domains
TIMSS 2015 frameworkKnowledgeApplicationReasoning
Developing cognitive tests for the secondary school survey.
Worked together with national experts and EI.
Knowledge - the facts, concepts, and procedures students need to
knowApplication - ability of students to apply knowledge and the
conceptual understanding to solve problems or answer
questionsReasoning - goes beyond the solution of routine problems
to encompass unfamiliar situations, complex contexts, and
multi-step problems7
We produced:A curriculum-linked assessment covering all
cognitive domains and the content domains relevant to each country
While retaining cross-country comparability, through use of anchor
items in overlapping content/cognitive domainsWhich allows later
construction of a common scale of mathematics achievementA
CROSS-COUNTRY ASSESSMENT
W1 DESCRIPTIVES (A)
Visualise learning levels on a common scale. Investigate
performance levels by groups
t overall performance changes in each country and split this by
whatever dimension we fancyThe point about the distributions is
that it might allow some insights into lower/higher achieving
groups across countries and gets us beyond a comparison of means.
But might not be that valuable9
W1 DESCRIPTIVES (A)
Visualise learning levels on a common scale. Investigate
performance levels by groups
t overall performance changes in each country and split this by
whatever dimension we fancyThe point about the distributions is
that it might allow some insights into lower/higher achieving
groups across countries and gets us beyond a comparison of means.
But might not be that valuable10
W1 DESCRIPTIVES (B)
Problem solving and applications in daily life
Integers and rational numbers, powers and basesTs father is 6
times as old as T. T's mother is 25 years old. The average age of
this family of three is 20 years. How old is T?
A. 15 yearsB. 10 yearsC. 7 yearsD. 5 years
Understand student performance in different content and
cognitive domains and how it varies between countriesDiscussion of
IRT scaling and fit for these two example items(L) Integers and
Rational numbers, Powers and Bases / Understanding(B) Problem
solving and applications in daily life / Application
11
AND
With this measurement approach we can:
Understand the educational trajectories of individuals and
groups, between testing occasions
Suggest hypotheses about the relationship between student
learning progress and school, classroom, teacher and student
characteristics (school effectiveness model)
Explore country differences in terms of cultures, curricula and
school organisation > not a vehicle for ranking countries but
one for further scrutiny
Design of the secondary school surveys will allow us to look at
learning quality in a similar way to the primary school surveys
Building upon extensive existing YL data allows understanding of
types of lessons for school effectiveness across 3 countries12
www.younglives.org.uk @YLOxford
methodology and research paperschild profiles and
photose-newsletterdatasets (UK Data Archive)
FINDING OUT MORE
13
13
