View
218
Download
0
Tags:
Embed Size (px)
Citation preview
1
Grade Level of Achievement (GLA) Reporting
ASAS Lecture, Universities of Alberta and Calgary
February 6, 2008
2
Introduction
Exploratory Themes The foundation for GLA reporting
Solid classroom assessment theory GLA Reporting: Teacher and Administrator Handbook
Review GLA from two perspectives Can GLA data be collected and reported consistently? Does GLA contribute to our knowledge base regarding
student achievement?
3
Is it possible to balance the Is it possible to balance the decision-making/management decision-making/management information needs of information needs of administrators with teachers’ administrators with teachers’ classroom assessment data classroom assessment data reflecting the learning needs of reflecting the learning needs of students?students?
Key Question…
4
Theoretical Foundation Greater insight is gained when teachers bridge the gap between their assessment efforts and external assessment initiatives Provincial Achievement Tests (PATs)
School Improvement
Grade Level of Achievement (GLA) data
5
A Balanced ModelSee “A Balanced Approach to High Stakes Achievement Testing…” IEJLL Vol. 7, No. 4
FormativeFormative SummativeSummativeN
orm
Ref
eren
ced
Nor
m R
efer
ence
dC
rite
rion
Ref
eren
ced
Cri
teri
on R
efer
ence
dLearner SupportLearner Support
Grade Grade PlacementPlacement DecisionsDecisions
Academic Academic Program Choice Program Choice Advice/DecisionsAdvice/Decisions
Career Choice Career Choice Advice/DecisionsAdvice/Decisions
Program/SchoolProgram/SchoolEffectiveness Effectiveness JudgmentsJudgments
6
“… “… the judgment of the classroom teacher is the judgment of the classroom teacher is an integral part of constructive accountability” an integral part of constructive accountability”
Only when accountability, standards, and Only when accountability, standards, and assessment are fully integrated at the assessment are fully integrated at the classroom level will we achieve the potential classroom level will we achieve the potential for fairness, equity of opportunity, and for fairness, equity of opportunity, and improved academic achievement that teaching improved academic achievement that teaching professionals crave and society demands.professionals crave and society demands.(D. Reeves, (D. Reeves, Accountability For LearningAccountability For Learning, ASCD 2004: 107), ASCD 2004: 107)
7
Theoretical Basis – Understanding by DesignSee Grant Wiggins and Jay McTighe, Understanding by Design, Expanded 2nd edition, ASCD, 2005.
Teaching
Curriculum
Balanced Assessmentcompatible with GLA
Clear Targets
Clear PurposesProp
er M
etho
ds
8
The GLA Reporting Handbook Emphasizes reporting GLA to parents/students as
primary and to Alberta Education as secondary Provides clearer linkages to curricular outcomes Builds on teacher capacity to do good classroom
assessment work Reinforces the link between assessment and
instructional decision-making Emphasizes assessment for learning in ways that
compliment assessment of learning Available on line @
http://www.education.gov.ab.ca/ipr/GLA/TeachAdminHandbook.pdf
Or may be purchased ($6.40 Canadian) @http://www.lrc.education.gov.ab.ca/pro/default.html
9
2006-07 GLA Data Characteristics Alberta Student number is used to align GLA
with related variables No personal identifiers are left in the dataset 3,380 students (1.53%) were on a modified
program (non-graded curriculum) 78% of 217,302 students were non-coded, 2%
had severe disability codes, 8% had mild/moderate disability codes, 1% were coded as gifted and 11% were ESL students
Limitations related to sampling bias
10
For a detailed description of GLA 2005-06 data see… http://education.alberta.ca/media/505076/g
latechnicalreport.pdf 2006-07 data analysis was consistent with
2005-06 data. 2006-07 GLA Pilot Data report to be
published in March.
11
Spearman rho - English LA GLA & Enrolled Grade
Student Groups
2005-06 Correlation Coefficient
(significant at the .01 level -2 tailed)
2006-07 Correlation Coefficient
(significant at the .01 level -2
tailed)
Non Coded .991 .991
Severe Disabilities .853 .868
Mild/Moderate Disabilities
.845 .896
Gifted .994 .994
ESL Canadian Born .974 .966
ESL Foreign Born .925 .945
12
Spearman rho - Math GLA & Enrolled Grade
Student Groups
2005-06 Correlation Coefficient
(significant at the .01 level -2 tailed)
2006-07 Correlation Coefficient
(significant at the .01 level -2 tailed)
Non Coded .994 .992
Severe Disabilities .863 .874
Mild/Moderate Disabilities
.854 .901
Gifted .980 .984
ESL Canadian Born .985 .975
ESL Foreign Born .974 .971
13
Kendall’s tau-b Correlation of PAT by GLA*
2005/06 2006/07
PAT by GLA- Grade and Subject
Corr. Coeff.
PAT by GLA- Grade and Subject
Corr. Coeff.
Gr. 3 Eng. LA .378 Gr. 3 Eng. LA .324
Gr. 6 Eng. LA .406 Gr. 6 Eng. LA .337
Gr. 9 Eng. LA .338 Gr. 9 Eng. LA .323
Gr. 3 Math .388 Gr. 3 Math .342
Gr. 6 Math .403 Gr. 6 Math .366
Gr. 9 Math .399 Gr. 9 Math .409
*all significant at the .01 level -2 tailed
14
Some Conclusions re: Data Utility, Validity and Reliability Correlations between the students’ GLA
and PATs were of moderate strength GLA and PAT correlations were similar in
2006-07 compared to 2005-06. In general, the relationships with enrolled
grade for sub-groups of students were in the expected directions supporting the validity of GLA data
16
GenderGender - Provincial Mathematics
Female Male
Number of students
Percent of total enrolled
(%)Number of students
Percent of total enrolled
(%)
GLA below enrolled grade 9,504 9.0% 11,844 10.6%
GLA equal to enrolled grade 93,918 88.7% 96,508 86.7%
GLA above enrolled grade 464 0.4% 563 0.5%
No GLA provided 2,044 1.9% 2,457 2.2%
Total 105,930 100.0% 111,372 100.0%
Gender - Provincial English Language Arts
Female Male
Number of students
Percent of total enrolled
(%)Number of students
Percent of total enrolled
(%)
GLA below enrolled grade 9,335 8.8% 14,781 13.3%
GLA equal to enrolled grade 92,749 87.6% 92,399 83.0%
GLA above enrolled grade 308 0.3% 232 0.2%
No GLA provided 3,538 3.3% 3,960 3.6%
Total 105,930 100.0% 111,372 100.0%
17
Student Mobility and ELA
Mean GLA per enrolled grade as SMI increases in English Language Arts
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9 10+
SMI
Gra
de
1
2
3
4
5
6
7
8
9
18
Student Mobility and Math
Mean GLA per enrolled grade as SMI increases in Mathematics
0
1
2
3
4
5
6
7
8
9
1 2 3 4 5 6 7 8 9 10+
SMI
Gra
de
1
2
3
4
5
6
7
8
9
19
Student Mobility and Math
Percent of Students Below Grade Level by Mobility Category for each Grade
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
1 2 3 4 5 6 7 8 9
Enrolled Grade
Perc
ent B
elow
Gra
de
Lev
el
High Mobility Low Mobility
20
English GLA compared to Birth Month, Grade 1
Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 1 students
-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00
1stJan
1stFeb
March April May June July Aug Sept Oct Nov Dec 2ndJan
2ndFeb
z-score Linear (z-score)
21
English GLA compared to Birth Month, Grade 2
Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 2 students
-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00
1stJan
1stFeb
March April May June July Aug Sept Oct Nov Dec 2ndJan
2ndFeb
z-score Linear (z-score)
22
English GLA compared to Birth Month, Grade 3
Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 3 students
-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00
1stJan
1stFeb
March April May June July Aug Sept Oct Nov Dec 2ndJan
2ndFeb
z-score Linear (z-score)
23
English GLA compared to Birth Month, Grade 4
Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 4 students
-2.50-2.00-1.50-1.00-0.500.000.501.001.502.00
1stJan
1stFeb
March April May June July Aug Sept Oct Nov Dec 2ndJan
2ndFeb
z-score Linear (z-score)
24
English GLA compared to Birth Month, Grade 5
Z-score of the percent of students at or above grade level in Language Arts, by birth month within cohort - Grade 5 students
-1.50-1.00-0.500.000.501.001.502.002.50
1stJan
1stFeb
March April May June July Aug Sept Oct Nov Dec 2ndJan
2ndFeb
z-score Linear (z-score)
25
Overall Data Observations There is greater variance in GLA for sub-groups
of students with unique needs GLA data can supplement PAT data with
additional insights for teachers and decision makers
GLA provides missing data GLA value added analysis points to research
needed to better understand factors affecting student achievement
26
13
Related issues from “Effective Student Assessment and Evaluation in the Classroom” “Every model of the teaching/learning process requires
that teachers base their decisions- instructional, grading, and reporting- on some knowledge of student progress toward desired learning outcomes” (Rogers, 1991).
Teachers are responsible for ensuring that the assessment of their students is fair, accurate and reflects the results of a range of assessment methods that consistently, in the teacher’s informed professional judgment, point to the students’ reported assessment result.
27
14
Related Issues, Continued…
Research on teaching reveals that the assessment and evaluation of student learning is unquestionably one of the teacher’s most demanding and complex tasks.
Teacher professional judgment is strengthened through reflective practice and through active participations with teacher colleagues.
28
15
Related Issues, Continued…Purposes of Classroom Assessment and Evaluation
Provide opportunities for students to be actively involved in their own assessments;
Relate to the learning outcomes stated in the programs of study; Focus and motivate students to take the steps necessary to
ensure that learning takes place; and Allow for the identification of students’ strengths and areas of
need.
Student learning is the collective responsibility of all members of the school community. Teachers and experts should work collaboratively to positively support student learning.
Q. How bounded is the “school community in this context?
29
16
Related Issues, Continued… Alberta Education, in consultation with teachers and
administrators, can use the information from provincial and other external testing and assessment programs to identify potential changes in the programs of study and approaches to instruction to improve the learning of
students. When using external testing methods, teacher should
be sure to balance these with information gained from other classroom assessment methods. Fair and reasonable assessment is more accurate when based on the application of a range of assessment
methods. Q. Is GLA a legitimate form of triangulating data for
program planning and evaluation purposes?
30
Lessons Learned Implementation must be a combination of top down and
bottom up efforts Sharing GLA data aligns with emerging transformational
or distributed educational leadership models Students and parents can be more involved in
assessment for and of learning GLA Works best when related to existing assessment
work – Teachers don’t need more work to do Teachers need to be assured that the type of
accountability GLA reporting seeks to serve is learner centered to serve student needs
Teachers benefit from collegial discussions of how to best judge a student’s GLA
31
Lessons Learned Continued
Role of policy entrepreneurs at all levels is vital. GLA R is a project of discovery that will
continue to be refined as it evolves before the full-range of positive impacts result.
Communications is vital. Implementation plan has been responsive to
feedback Trust building is essential.
32
In Conclusion, GLA reporting will be a step towards…
Improved program planning More engaged dialogue among educators about
what works for students A more balanced approach to classroom
assessment Data rich critical reflection within professional
learning communities Better informed leadership focused on
improving the overall quality of education
33
Contacts/additional informationContacts [email protected] -
Accountability and Reporting Division, Alberta Education
For more information:
http://www.education.gov.ab.ca/ipr/GLA