2003TIMSS Assessment Frameworksand Specifications 20032nd Edition
Ina V. S. Mullis
Michael O. Martin
Teresa A. Smith
Robert A. Garden
Kelvin D. Gregory
Eugenio J. Gonzalez
Steven J. Chrostowski
Kathleen M. O’Connor
Trends in Mathematicsand Science Study
The International Study CenterLynch School of Education
The International Association for theEvaluation of Educational Achievement
© 2003 International Association for theEvaluation of Educational Achievement (IEA)
TIMSS Assessment Frameworksand Specifications 2003 2nd Edition
February 2003
Publisher: International Study Center,Lynch School of Education, Boston College
Library of CongressCatalog Card Number: 2003101238
ISBN: 1–889938–30–0
For more information aboutTIMSS contact:
TIMSS International Study CenterLynch School of EducationManresa HouseBoston CollegeChestnut Hill, MA 02467United States
tel: +1-617-552-1600fax: +1-617-552-1203
e-mail: [email protected]
http: //timss.bc.edu
Boston College is an equal opportunity,affirmative action employer.
Printed and bound in the United States.
Preface i
Introduction 1Overview 3
The TIMSS Curriculum Model 3
The Development Process for theTIMSS Assessment Frameworksand Specifications 4
The TIMSS Tests 5
Student Populations Assessed 6
Mathematics Framework 7Overview 9
Mathematics Content Domains 11
a Number 12
b Algebra 14
c Measurement 16
e Geometry 18
d Data 21
Mathematics Cognitive Domains 25
f Knowing Facts and Procedures 27
g Using Concepts 28
h Solving Routine Problems 30
i Reasoning 32
Communicating Mathematically 34
Guidelines for Calculator Use 34
Contents
Science Framework 35Overview 37
Science Content Domains 38
a Life Science 40
c Chemistry 47
b Physics 50
d Earth Science 54
e Environmental Science 58
Science Cognitive Domains 61
f Factual Knowledge 63
g Conceptual Understanding 64
h Reasoning and Analysis 66
Scientific Inquiry 69
Contextual Framework 71Overview 73
The Curriculum 73
The Schools 75
Teachers and Their Preparation 76
Classroom Activities and Characteristics 78
The Students 81
Assessment Design 83Scope of the Assessment 85
Dividing up the Item Pool 85
Block Design for Student Booklets 86
Question Types and Scoring Procedures 88
Scales for Reporting Student Achievement 89
Releasing Assessment Materialto the Public 90
Background Questionnaires 90
Endnotes 93
Appendix A 97National Research Coordinators
Appendix B 103Example Mathematics Items
Appendix C 121Example Science Items
Background
Advancing science and mathematics education has long been the focus ofstudies by the IEA, the International Association for the Evaluation ofEducational Achievement. Reflecting the seminal place that these two keycurriculum areas hold in all educational systems as fundamental to devel-oping technologically proficient societies, IEA has been measuring studentachievement and collecting contextual information to facilitate studentlearning in mathematics and science for nearly 40 years.
The conduct of the First International Mathematics Study (FIMS)dates back to 1964, and science first was assessed as part of the Six SubjectStudy in 1970-71. Mathematics and science were again the focus of majorresearch efforts in 1980-82 and 1983-84, respectively. In 1990, the IEAGeneral Assembly determined to assess science and mathematics togetheron a regular basis every four years. This decision marked the first of thelarge-scale international studies to measure trends in student performance,beginning with the original TIMSS (the Third International Mathematicsand Science Study) conducted in 1995, TIMSS-Repeat in 1999, and nowTIMSS 2003 (renamed the Trends in International Mathematics andScience Study), also known as TIMSS Trends.
Frameworks for TIMSS 2003
A particular challenge for TIMSS 2003 was developing this set of frame-works articulating important content for students to have learned in math-ematics and science, as well as describing important home and schoolcontexts influencing achievement in these subjects. It is important thatthese frameworks capture important issues for mathematics and scienceeducation today, while providing the vision necessary to take the TIMSScycle of studies beyond the 2003 assessment. The frameworks, producedat the beginning of the new millennium, are designed to shape future IEAassessments in mathematics and science so that they can evolve with thetimes, while recognizing the axiom – If you want to measure change, do notchange the measure.
The TIMSS International Study Center at Boston College preparedthis second edition of the frameworks to provide examples of the types ofassessment questions contained in the TIMSS 2003 assessment. The exam-ple items for mathematics are presented in Appendix B and for science inAppendix C. The second edition also contains some minor revisions, inparticular to the section on the assessment design.
TIMSS 2003 i
Preface
Acknowledgments
The IEA was founded in 1959 for the purpose of conducting comparativeresearch studies on educational policies, practices, and outcomes. Sincethen, IEA studies have contributed a deeper understanding of the educa-tional process both within and among the nearly 60 member countries.The IEA has a permanent Secretariat in Amsterdam, the Netherlands, anda Data Processing Center in Hamburg, Germany. TIMSS is directed by IEA’sInternational Study Center at Boston College. The strength, quality, andsuccess of IEA’s studies, however, derive from the expertise among itsmembers in curriculum, measurement, and education and their collabora-tion in conducting the research.
Also, extremely crucial to success is securing the funding necessary tocarry out the extensive development and review work required by inter-national projects of this magnitude. Without such support a project likeTIMSS is not possible. The IEA is deeply grateful to the U.S. NationalScience Foundation, the U.S. National Center for Education Statistics, andfor the fees paid by participating countries for helping to fund the develop-ment of the TIMSS frameworks presented herein.
The TIMSS frameworks are the result of considerable collaborationamong individuals from around the world, most notably the specialistscomprising the TIMSS Expert Panel in Mathematics and Science, theNational Research Coordinators from the participating countries, staff fromIEA’s International Study Center at Boston College, and staff from IEA’sSecretariat and Data Processing Center. I am extremely grateful for the con-tribution of each person who devoted his or her energy and time to thisimportant and comprehensive effort. In particular, I would like to acknowl-edge the work of the TIMSS Mathematics Coordinator Robert Garden andthe Science Coordinator Teresa Smith. Kelvin Gregory, the TIMSSCoordinator at the time the frameworks were developed, had specialresponsibility for the contextual framework. The Director of Operations andAnalysis of the International Study Center and TIMSS, Eugene Gonzalez,oversaw development of the assessment design.
Without a dedicated, continuing center from which to coordinateprojects like TIMSS and the experienced staff from the consortium oforganizations that implements the studies, success would be limited. Iwould like to express my thanks to the staff at the International StudyCenter at Boston College, the IEA Secretariat, the IEA Data ProcessingCenter, Statistics Canada, and Educational Testing Service. Finally, I wouldlike to especially thank the Co-Directors of the International Study Centerand TIMSS, Ina V.S. Mullis and Michael O. Martin, for their leadership anddedication to this project.
Hans WagemakerIEA Executive Director
TIMSS 2003ii
TIMSS 2003 iii
The International Study Center at Boston College
The International Study Center (ISC) at Boston College is dedicated toconducting comparative studies in educational achievement. Principally, itserves as the International Study Center for IEA’s studies in mathematics,science, and reading – the Trends in International Mathematics andScience Study (TIMSS), and the Progress in International Reading LiteracyStudy (PIRLS). The staff at the ISC is responsible for the design and imple-mentation of these studies. In developing and producing the TIMSS frame-works, ISC staff conducted a collaborative effort involving a series ofreviews by an Expert Panel and the National Research Coordinators. Thefollowing individuals were instrumental in this process.
Ina V.S. MullisCo-Director, TIMSS
Eugenio J. GonzalezDirector of Operations and Data Analysis
Teresa A. SmithScience Coordinator
Steven J. ChrostowskiDevelopment Specialist
Christine O’SullivanScience Consultant
Michael O. MartinCo-Director, TIMSS
Kelvin D. GregoryTIMSS Coordinator (2000-2001)
Robert A. GardenMathematics Coordinator
Kathleen M. O’ConnorDevelopment Specialist
Eugene JohnsonPsychometric/Methodology Consultant
Hans WagemakerExecutive Director
Dirk HastedtIEA Data Processing Center
Barbara MalakManager, Membership Relations
Oliver NeuschmidtIEA Data Processing Center
TIMSS 2003iv
International Association for the Evaluation ofEducational Achievement (IEA)
In developing the TIMSS Assessment Frameworks and Specifications, theIEA has provided overall support in coordinating TIMSS with IEA’s mem-ber countries and reviewing all elements of the design. The following per-sons are closely involved with TIMSS.
Statistics Canada
Statistics Canada is responsible for collecting and evaluating the sample inTIMSS, and helping participants to adapt the TIMSS sampling design tolocal conditions. Senior methodologists Pierre Foy and Marc Joncasreviewed the frameworks from a sampling perspective, and made manyhelpful suggestions.
Educational Testing Service
Educational Testing Service conducts the scaling of the TIMSS achievementdata. Researchers Matthias Von Davier, Edward Kulick, and KentaroYamamoto reviewed the frameworks from a design perspective.
National Research Coordinators
The TIMSS National Research Coordinators (NRCs) work with internation-al project staff to ensure that the study is responsive to their concerns,both policy-oriented and practical, and are responsible for implementingthe study in their countries. NRCs reviewed successive drafts of the frame-works, and made numerous suggestions that greatly benefited the finaldocument. A list of the NRCs appears in Appendix A.
TIMSS 2003 v
International Expert Panel in Mathematicsand Science
The Expert Panel worked with staff from the International Study Center indeveloping all aspects of the frameworks and particularly the mathematicsand science frameworks. They made recommendations for the contentareas, cognitive domains, problem-solving and inquiry tasks, and focusareas for policy-orientated research.
Mathematics
Khattab Abu-LibdehJordan
Anica AleksovaRepublic of Macedonia
Kiril BankovBulgaria
Aarnout BrombacherSouth Africa
Anna Maria CaputoItaly
Joan Ferrini-MundyUnited States
Jim FeyUnited States
Derek HoltonNew Zealand
Jeremy KilpatrickUnited States
Pekka KupariFinland
Mary LindquistUnited States
David RobitailleCanada
Graham RuddockUnited Kingdom
Hanako SenumaJapan
Science
K.Th. (Kerst) BoersmaThe Netherlands
Rodger BybeeUnited States
Audrey ChampagneUnited States
Reinders DuitGermany
Martin HollinsUnited Kingdom
Eric JakobssonUnited States
Galina KovalyovaRussian Federation
Svein LieNorway
Jan LokanAustralia
Francisco MazzitelliArgentina
Gabriella NoveanuRomania
Margery OsborneUnited States
Jana PaleckovaCzech Republic
Hong Kim TanSingapore
Khadija Zaim-Idrissi Morocco
TIMSS 2003vi
Funding
Funding for the development of the TIMSS Assessment Frameworks andSpecifications was provided by the U.S. National Science Foundation, theU.S. National Center for Education Statistics, and participatingcountries. In particular, much of the work was made possible by a grantfrom the U.S. National Science Foundation. Janice Earle, Finbarr Sloane,Elizabeth Vander Putten, and Larry Suter each played crucial roles inmaking the frameworks possible.
Recommended