ED 403 164

TITLE

INSTITUTION

SPONS AGENCY

REPORT NOPUB DATECONTRACTNOTEAVAILABLE FROM

PUB TYPE

EDRS PRICEDESCRIPTORS

DOCUMENT RESUME

SE 059 728

Science Framework for the 1996 National Assessment ofEducational Progress. NAEP Science ConsensusProject.American Institutes for Research in the BehavioralSciences, Washington, D.C.; Council of Chief StateSchool Officers, Washington, D.C.; National Centerfor Improving Science Education, Washington, DC.National Assessment Governing Board, Washington,DC.

ISBN-0-16-048903-296

RS9008100196p.

U.S. Government Printing Office, Superintendent ofDocuments, Mail Stop: SSOP, Washington, DC20402-9328.Reports Descriptive (141)

MF01/PC04 Plus Postage.Academic Achievement; Elementary Secondary Education;*Evaluation Criteria; *National Competency Tests;*Science Education; *Science Tests

IDENTIFIERS National Assessment of Educational Progress

ABSTRACTThis document contains the Framework and rationale

for assessing science achievement of students throughout the UnitedStates in 1996. It provides a general overview of the NationalAssessment of Educational Progress (NAEP), describes the 1996 NAEPScience Framework'adopted by the National Assessment Governing Board.(NAGB), and reviews the process by which the Framework was developed.Chapters include: (1) "The Nature of Science and the ScienceCurriculum"; (2) "The Framework for the 1996 NAEP ScienceAssessment"; (3) "Desired Attributes of the Assessment"; and (4)"Characteristics of Assessment Exercises." Appendices include Fieldsof Science, Examples of Themes by Grade Level, Science ContentOutlines (Excerpts), and Consensus Committees and Project Staff.Contains 14 references. (JRH)

***********************************************************************

Reproductions supplied by EDRS are the best that can be madefrom the original document. *

***********************************************************************

SCUlot ui- IniLtittbi MU I .,c

The ERIC Facility has assignedthis document for processingto:

In our judgment, this document

0 is also of interest to the Clear-noted to the right.

Oposihnotsuldofrevtielewc.t their

ffou° Ile aCC'6 HElouneomood'onl cf

aiancaanouf s° oueoo

SE

III It I111'IIIlll ll sli; ;fir .11

111

1 BEST COPY AVAILABLE 1

0

000 00

0

U.S. DEPARTMENT OF EDUCATIONOffice Of Educational Research and Improvement

DUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

his document has been reproduced asr wed from the person or organizationoriginating a

O Minor changes have been made to improvereproduction quality.

Points of view or or:rim:ins stated in this document do not necessarily represent officialOERI position or policy

RIAZED Odom Canzemoaro Prod

MCA AssessDepart

matt GavanbanF,onomQ emommoDGD

Hoare

9

c.

What Is NAEP?

The National Assessment of Educational Progress (NAEP) is acongressionally mandated project of the U.S. Department ofEducation's National Center for Education Statistics. It assesses

what U.S. students should know and be able to do in geography, reading,writing, mathematics, science, U.S. history, the arts, civics, and otheracademic subjects. Since 1969, NAEP has surveyed the achievement ofstudents at ages 9, 13, and 17 and, since the 1980s, in grades 4, 8, and 12.

Measuring educational achievement trends over time is critical tomeasuring progress toward the National Education Goals.

The National Assessment Governing BoardThe National Assessment Governing Board (NAGB) was created byCongress to formulate policy for the National Assessment of EducationalProgress (NAEP). Among the Board's responsibilities are developingobjectives and test specifications, and designing the assessment methodologfor NAEP.

MembersMark D. Musick, ChairmanPresidentSouthern Regional

Education BoardAtlanta, Georgia

Charlotte CrabtreeProfessorUniversity of CaliforniaLos Angeles, California

Chester E. Finn, Jr.Founding Partner and

Senior ScholarHonorable William T. Randall, The Edison ProjectVice-Chair Washington, D.C.Commissioner of EducationState Department of EducationDenver, Colorado

Parris C. BattleEducation SpecialistDade County Public SchoolsMiami, Florida

Honorable Evan BayhGovernor of IndianaIndianapolis, Indiana

Michael S. GlodeWyoming State Board

of EducationSaratoga, Wyoming

William J. HumeChairman of the BoardBasic American, Inc.San Francisco, California

Christine JohnsonDirector of Urban Initiatives

Mary R. Blanton Education Commission ofAttorney the StatesBlanton & Blanton Denver, ColoradoSalisbury, North Carolina

Boyd W. BoehljeAttorney and School Board

MemberPella, Iowa

Linda R. BryantDean of StudentsFlorence Reizenstein

Middle SchoolPittsburgh, Pennsylvania

Honorable Naomi K. CohenOffice of Policy and

ManagementState of ConnecticutHartford, Connecticut

John S. LindleyPrincipalGalloway Elementary SchoolHenderson, Nevada

Honorable Stephen E. MerrillGovernor of New HampshireConcord, New Hampshire

Jason MillmanProfessorCornell UniversityIthaca, New York

Honorable Richard P. MillsCommissioner of EducationState Department of EducationMontpelier, Vermont

Carl J. MoserDirector of SchoolsThe Lutheran Church-Missouri SyncSt. Louis, Missouri

John A. MurphySuperintendent of SchoolsCharlotte-Mecklenburg SchoolsCharlotte, North Carolina

Michael NettlesProfessorUniversity of MichiganAnn Arbor, Michigan

Honorable Carolyn PollanArkansas House of RepresentativesFort Smith, Arkansas

Thomas TopuzesSenior Vice PresidentValley Independent BankEl Centro, California

Marilyn WhirryEnglish TeacherMira Costa High SchoolManhattan Beach, California

Sharon P. Robinson (Ex-Officio)Assistant SecretaryU.S. Department of EducationWashington, D.C.

Roy TrubyExecutive DirectorNAGSWashington, D.C.

3 RPM- rnpy A \ /AII API P

Science Frameworkfor the 1996 NationalAssessment ofEducational Progress

The ERIC Facility has assignedthis document for processingto:

In our judgment, this documentis also of interest to the Clear-inghouses noted to the right.Indexing should reflect theirspecial points of view.

NAEP Science Consensus Project

Developed under contract number RS90081001 by the Council ofChief State School Officers, with the National Center for ImprovingScience Education and the American Institutes for Research for theNational Assessment Governing Board.

I

National Assessment Governing Board

Mark D. MusickChair

William T. RandallVice Chair

Roy TrubyExecutive Director

Mary CrovoProject Officer

Science Consensus Project

Ramsay W. SeldenCouncil of Chief State School Officers

Richard C. ClarkCouncil of Chief State School Officers

Senta A. RaizenNational Center for Improving Science Education

Science Assessment Framework for the 1996 NationalAssessment of Educational Progress

Developed for the National Assessment Governing Board undercontract number RS90081001 by the Council of Chief StateSchool Officers with the National Center for Improving ScienceEducation and the American Institutes for Research for theNational Assessment Governing Board

For further information, contact the National AssessmentGoverning Board: 800 North Capitol Street, NW

Suite 825Washington, DC 20002

5For sale by the U.S. Government Printing Office

Superintendent of Documents, Mail Stop: SSOP, Washington, DC 20402-9328

ISBN 0-16-048903-2

Table of Contents

Introduction 1

Background 1

Development of the 1996 NAEP Science AssessmentFramework 3

Steering Committee Guidelines 5

Chapter One: The Nature of Science and theScience Curriculum 7

The Nature of Science 7

The Science Curriculum 9

Chapter Two: The Framework for the 1996 NAEPScience Assessment 11

Introduction 11

Framework Elements 13

The Matrix 13

The Fields of Science 17

Earth Science 17

Physical Science 18

Life Science 19

Knowing and Doing Science 20

Conceptual Understanding 21

Scientific Investigation 23

Practical Reasoning 24

The Nature of Science 26

Science 26

Technology 27

Themes 28

Chapter Three: Desired Attributes of the Assessment 31

Some Assessment Issues 31

The Nature of Testing and the Nature ofKnowledge and Learning 31

iii6

Inferring Understanding From Student Responses 32Developments in Assessment 34Criteria for Assessing Learning and Achievement

in Science 34Preliminary Achievement Levels in Science 35

Chapter Four: Characteristics of Assessment Exercises 41Types of Exercises 41

Performance Exercises 41Open-Ended Pencil-and-Paper Items 42Multiple-Choice Items 43Additional Considerations 43

Pilot Testing 44Procedures for Development of the Assessment 45Special Study in Science 45

References 47

Appendix A: Fields of Science 49Earth Science 51Physical Science 56Life Science 66

Appendix B: Examples of Themes by Grade Level 71Systems 73Patterns of Change 74Models 77

Appendix C: Science Content Outlines (Excerpts) 81Grade 4Earth Science 83Grade 8Life Science 84Grade 12Physical Science 86

Appendix D: Consensus Committees and Project Staff 89

7

iv

Introduction

This document contains the Framework and rationale for assess-ing science achievement of students throughout the UnitedStates in 1996. It provides a general overview of the National

Assessment of Educational Progress (NAEP), describes the 1996NAEP Science Framework adopted by the National AssessmentGoverning Board (NAGB), and reviews the process by which theFramework was developed.

Background

The National Assessment of Educational Progress is authorizedby Congress and funded by the federal government; it is the onlynationally representative and continuing assessment of whatAmerica's students know and can do. For more than 20 years,NAEP has been charged with collecting and reporting informationon student achievement in mathematics, reading, science, U.S.history, writing, and other subjects. NAEP assessments wereconducted on an annual basis until 1981, when they became bien-nial. Originally, assessments were of students at ages 9, 13, and 17,but beginning in 1983, they have included students at grades 4, 8,and 12, as well.

NAEP reports provide descriptive information about studentperformance in various subjects, including basic and higher orderskills, and comparisons of performance by race/ethnicity, gender,type of community, and geographic region. They also show rela-tionships between achievement and certain background variables,such as time spent on homework or educational level of parents.

In the past, only results from a national sample of students havebeen reported for each NAEP subject assessed; but in 1987, anational study group chaired by Lamar Alexander, then Governorof Tennessee, and H. Thomas James, President Emeritus of theSpencer Foundation, recommended to the Secretary of Educationthat NAEP collect representative data on student achievement at thestate level. For the first time, then, in 1990 a trial state assessmentwas conducted for 8th-grade mathematics. The trial continued in1992 with state-level assessments in mathematics at grades 4 and 8and reading at grade 4. In 1994, reading was assessed at the state

1 8

level in grade 4. State-level assessments beyond 1994 are beingplanned.

Assessment is not without critics. The Alexander-James StudyGroup expressed concerns that "national assessments of sciencewere sporadic and almost exclusively devoted to assessing factualinformation," and that they did not attempt to assess abilities toorganize and transform a body of information into a coherentscientific account (The Nation's Report Card: Improving theAssessment of Student Achievement, 1987).

To address this criticism, the study group recommended thatNAEP broaden its scope to include the collection of information onwhether students are able to design, perform, and analyze experi-ments; on whether they have acquired complex thinking abilitiesessential to various fields of science; on whether they are able tointegrate basic concepts of the various scientific fields; and onwhether they can perceive fundamental relationships. To measurethese kinds of knowledge and abilities, NAEP was urged toinclude open-ended items and performance tasks in its assessmenttechniques.

NAEP has great prestige, considerable influence, and well-publicized results. Therefore, the weight of its findings may have asubstantial impact on science education, possibly influencing statecurriculum frameworks and, ultimately, even what teachers teach.Therefore, the assessment should reflect consensus on priorities,best practices, and conclusions from research on science education.

A specific example of how NAEP can influence state frame-works is illustrated by the selection of special assessment tech-niques, such as hands-on science. Performance, for instance, placesa value on the "doing" of science. That priority actually may not bereflected in classroom practice, although it always was consideredto be a standard teaching technique in science. Thus, as the NAEPscience reports are circulated, states and curriculum policymakersbecome sensitized and may incorporate more hands-on activitiesinto classroom lessons.

Science educators, by and large, do not quarrel with the essentialconcept of assessment, but there has been no formal agreement on acommon framework, outcomes, or goals and objectives to assess.The 1996 NAEP Science Assessment attempts to reflect a compre-hensive, contemporary view of science so that those affected by the

92

National Assessment are satisfied that it addresses the complexissues in science education without oversimplification. The framersof this document have attempted to tread a fine line between clearcommunication and technical accuracy in the hope that their effortsrepresent a step forward in building national consensus on the keyoutcomes of science education.

Development of the 1996 NAEP ScienceAssessment Framework

The following factors guided the process for developingconsensus on the Science Assessment Framework:

The first factor is the general process of consensus devel-opment both as it is set forth in law and as it evolves overtime. The process calls for active participation and broadinvolvement of curriculum specialists, science teachers, localscience supervisors, state supervisors, administrators, andparents, and representatives of scientific associations, businessand industry, government officials, unions, cognitive psy-chologists, and science educators, as well as participationfrom the public and private education sectors.

The second factor is emphasis on the important outcomesof science education. As much as possible, this ScienceFramework represents what is considered essential learningin science, setting the stage for the 1996 and subsequentassessments and recommending innovative assessmenttechniques to probe the critical abilities and content areas.

The third factor is recognition that the various "players"in education and in industry often hold diverse and some-times conflicting views. Further, research and general agree-ment in the field is lacking. This lack of agreement on acommon scope of instruction and sequence, components ofscientific literacy, important outcomes of learning, and thenature of overarching themes in science hinders clearcommunication between science educators and the public.

The process of developing the Framework document and accom-panying reports occurred between October 1990 and August 1991.Original plans called for the new NAEP Science Assessment to begiven in 1994. Due to a budget shortfall, however, both the newscience and mathematics assessments were rescheduled for 1996.

3 1 0

For the consensus process, a Steering Committee of 19 members(see appendix D) recommended by the education community andprivate sector was established. Its members developed the prin-ciples that guided creation of the Framework. A smaller PlanningCommittee (see appendix D), composed of practitioners, recog-nized experts in science education, and scientists, was establishedto identify goals and objectives and to produce the Framework.Together with the Steering Committee, it was also responsible forsuggesting ways to assess important outcomes of science education.

The Planning Committee met monthly from November 1990ihrough April 1991 and was joined in the first and final meetings bythe Steering Committee, which reviewed and reacted to all Frame-work drafts. Staff of the American Institutes for Research (AIR), asubcontractor, also attended the meetings and, on the basis of thisinput and with reaction and advice from the committees, AIRformulated specifications for the science assessment.

The NAGB Subject Area Committee #2 and technical staffclosely monitored the consensus project work, and Board memberswere involved in all key phases. In addition, advice was soughtfrom the organizations and sectors that affect and are affected byscience education. For instance:

Opportunities for public input were provided at two hearings(in Washington, D.C., and San Francisco). Opinions wereexpressed by representatives of institutions (e.g., publicand private education and scientific and science educationorganizations) likely to be affected by or concerned with theFramework and subsequent assessments.

The Council of State Science Supervisors was kept abreastof project developments through their electronic communica-tion system, PSInet. This was vital, because state scienceinstruction as well as public opinion may be influenced by the1996 assessment process. State policymakers and those withresponsibility for science education leadership must becomefamiliar with and involved in shaping the process and prod-ucts of the science consensus effort. Input from the Councilof State Science Supervisors was particularly important in

11_

helping the NAEP Science Assessment Project define "bigideas" or themes (see chapter two) in science learning. Forexample:

States with existing frameworks for science instructionwere contacted for copies of their frameworks and assess-ment methods.

Planning and Steering Committee members hosted sessionsat both regional and national science education meetingsto report on the development of the Science AssessmentFramework.

The revised draft of the Framework was widely circulatedin June 1991 within the science education and sciencecommunities for reaction and comments.

Steering Committee Guidelines

At its early meeting, the Steering Committee drafted guidelinesfor the Planning Committee's work and recommended that theFramework and ensuing science assessment have the following fivecharacteristics:

1. The Framework should reflect the best thinking about theknowledge, skills, and competencies needed for a high degree ofscientific understanding among all students in the United States.Accordingly, it should:

Encompass knowledge and use of organized factual informa-tion, relationships among concepts, major ideas unifying thesciences, and thinking and laboratory skills.

Be based on current understandings from research onteaching, learning, and student performance in science.

2. Both the Framework and the new NAEP Science Assessmentshould:

Address the nature and practices of knowing in science, asdifferent from other ways of knowing.

Reflect the quantitative aspects of science as well as theconcepts of the life, Earth, and physical sciences.

5 1 .9

Deal with issues raised by the role of science and technologyin society.

Include practical problem solving that involves design, use ofmaterials, and weighing risks in relation to benefits.

Take into account the developmental levels of students.

Ensure that students with diverse backgrounds are assessed inways that provide them with equal and fair opportunities toreflect their knowledge and performance.

3. Assessment formats should be used that are consistent with theobjectives being assessed. A variety of strategies for assessingstudent performance are advocated, including:

Performance tasks that allow students to manipulate physicalobjects and draw scientific understandings from the materialsbefore them.

Open-ended items that provide insights into students' levelsof understanding and ability to communicate in the sciences,as well as their ability to generate, rather than simply recog-nize, information related to scientific concepts and theirinterconnections.

Collections of student work over time (such as portfolios)that demonstrate what students can achieve outside the timeconstraints of a standardized assessment situation.

Multiple-choice items that probe students' conceptual under-standing and ability to connect ideas in a scientifically soundway.

4. The assessment should contain a broad enough range of items atdifferent levels of proficiency for identifying three achievementlevels for each grade.

5. Information on students' demographic and other backgroundcharacteristics should be collected. Additional informationshould be collected from students, teachers, and administratorsabout instructional programs and delivery systems, so that theirrelationships with student achievement can be ascertained andused to inform program and policy decisions.

13 6

Chapter One

The Nature of Science and theScience Curriculum

The Nature of Science

The various fields of science have their own special ways ofknowing, but the essentials of the natural sciences should bedefined for the purpose of planning appropriate assessments.

The natural sciences are characterized by organized explanationsincorporating both theoretical and empirical elements. Scientistsattempt to construct theories that encompass as much factual andconceptual knowledge, including laws and principles, as possible.The construction of theories is a process involving the consider-ation of factual evidence, insightful questioning, creativity, andimagination.

Atomic theory and evolution theory are good examples ofmodern scientific theories that are central to the sciences as ways ofknowing. They are sources of new hypotheses and logical deduc-tions that can be tested. As these theories are refined, they continueto stimulate new questions and hypotheses. As with other scientifictheories, each time they come into play in experimental situations,they are again subject to testing and the possibility of refutation.Each success broadens their domain and increases their usefulness.Verification of theories does not ensure truth, but does extend theirusefulness in explaining natural phenomena.

Useful theories enable an individual to make predictions under aspecified range of conditions. Scientific theories must be testableaccording to standards of evidence and logical argumentation set bythe scientific community; they are central to the scientific way ofknowing, guiding observation. When scientists observe naturalphenomena, their observations are made within the context ofexisting scientific theories. Nonscientists holding different viewsmay perceive the same phenomena, but, through their subjective

7 1 4

belief systems, may arrive at different conclusions. History isreplete with examples of misunderstandings and miscommunicationbased, at least in part, on the use of different rules of observation.

Science consists of both theoretical and experimental knowledgethat is constructed by the creativity, knowledge, and world view ofscientists. Experimentation plays an essential role in generatingand verifying scientific information. New findings and results ofscientific observation and experimentation accumulate, enlargingthe base of present knowledge, and laying the foundation for futurelearning. Usually, scientific knowledge is structured, forming a webof interrelated concepts, laws, and principles. Whereas sensory dataare sometimes ephemeral, concepts are precisely formulated andare multiply connected to other concepts or sensory data within atheory. Concepts without factual content (sensory data) are empty,while sensory data without concepts are difficult to understand andopen to misinterpretation.

None of the processes usually associated with scienceforexample, observation, measuring, classifying, deduction, infer-enceis unique to science. However, in science, these processesare given meaning by the context of the subject matter underinvestigation. Hence, observations of a mealworm seeminglywalking endlessly around the sides of a container have new mean-ing when mealworm anatomy, behavior, and physiology areunderstood. Likewise, the wiggly lines a physicist observes on aphotographic negative taken of a cloud chamber remain wigglylines to most observers, but they become important sensory data(facts) when the conditions under which the lines were producedare known. A goal of science education, therefore, is to help thestudent recognize the difference between personal opinion and theknowledge gained through scientific investigation and debate.

Although the scientific disciplines are alike in their reliance onevidence, use of logic, and organization of factual information intoconcepts and theories, they differ with respect to what constitutesevidence, specific methods of investigation, and degree of quantifi-cation. Yet, "there are common understandings among [scientists]about what constitutes an investigation that is scientifically valid."(American Association for the Advancement of Science, 1989).This view of the nature of science has profound implications forassessment as reflected in the criteria outlined in chapter three.

158

The Science Curriculum

Because a major purpose of NAEP is to illuminate educationpolicy, assessment of student science learning must take account ofthe science curriculum. Hence, the 1996 NAEP Science AssessmentFramework is based on a consensus regarding desirable elementsof science education against which student attainment is to bemeasured.

In developing the Framework, the Planning Committee reviewedkey blue-ribbon committee reports, examined exemplary practice,studied local- and state-based innovation in the science curriculum,reviewed the science education literature, and noted innovationsemerging in other countries. Recent reports by government agen-cies and professional societies (National Science Board, 1983;American Association for the Advancement of Science, 1989;National Science Teachers Association, 1989; National ResearchCouncil, 1990) express unanimity in their goals for scienceeducation. For example:

Students should acquire a core of scientific understanding,including organized factual information.

Students should acquire the ability to relate scientific conceptsto each other and to problems they encounter in and out ofschool.

Students should be able to apply science knowledge inpractical ways.

Students should be familiar with experimental design andhave the ability to carry out scientific experiments that aredevelopmentally appropriate.

Students should acquire the science knowledge and under-standing that will allow them the opportunity to pursue furtherstudy in scientific fields or enter science- or technology-related careers.

There also are similarities in the reports' recommendations forthe science curricula and instruction needed to achieve these goals,including:

Reduction of the traditional breadth of coverage in favor ofgreater depth, especially in high school science.

9 16

Emphasis on development of such thinking processes asorganizing factual knowledge around major concepts, definingand solving problems, accessing information and reasoningwith it, and communicating with others about one's scienceresults and understandings.

Multi- and interdisciplinary approaches to science teaching.

Approaches that encourage active student engagement andactive participation, such as participating in hands-on science;learning in small, cooperative groups; reflecting orally and inwriting upon experience; and completing sustained projects.

Increased participation of underrepresented populations inchallenging school science.

The design of the 1996 NAEP Science Assessment Framework,while maintaining some conceptual continuity with the 1990 NAEPScience Assessment, takes account of current reforms in scienceeducation. It also is consistent with the science framework used forthe 1991 International Assessment of Educational Progress and thescience framework proposed for the IEA (International Associationfor the Evaluation of Educational Achievement) assessmentsplanned for 1995. This consonance is important as the nationmonitors its progress toward the National Education Goals throughthe year 2000.

1710

Chapter Two

The Framework for the1996 NAEP Science Assessment

Introduction

It is customary to collect relevant curriculum guides, frame-works, and other course outlines to get a sense of what studentsare studying throughout the nation when developing a frame-

work for an educational assessment. The union of all these "learn-ing guides" is then used to develop some overall test specification.Such an approach tends to have an unfortunate consequence: Itoften leads to a broad, trivialized, lowest common denominatorapproach to assessment. However, these materials can serve anotherpurpose, documenting trends and developments in science educa-tion throughout the country. In the 1996 NAEP Science AssessmentFramework development process, reform reports in science fromstates and some large-city science curriculum guides were gatheredand used to establish consonance between the evolving frameworkand the most forward-looking of the reports and guides.

The traditional approach to teaching science has tended toemphasize rote memorization of facts without connection ororganization. While it must not lose sight of the need for factualknowledge fundamental to science literacy, there are severalcompelling reasons why the assessment of science achievementmust change to give more emphasis to conceptual understandingand the application of knowledge and skills. First, the expansion ofscientific information has created far too many facts for anyone tomemorize. It is much more efficient to store them electronically(or in other forms) and access information as it is needed. Second,isolated science facts that are unorganized and remain unused tendto be forgotten quickly and, even when remembered, form a poorbasis for learning. Third, it is desirable to encourage scienceteaching and learning in which science instruction is used bothto deepen understanding and to address challenging problems.

11 1 8

Science education is best served when students can understand anddiscuss ideas rather than simply accumulate unconnected facts.

Therefore, it is important that science assessment cover majortopics like electricity and magnetism, forces and motions, lifecycles, ecosystems, plate tectonics, and climatology. But, eventhese need to be viewed as shorthand for a much richer understand-ing of what students should attain. Several of the current reformreports and frameworks, such as Science for All Americans (Ameri-can Association for the Advancement of Science, 1989), innovativestate frameworks, and the reports of the National Center for Im-proving Science Education (1989, 1990) describe desired outcomesof science instruction in new ways. They advocate mastery oftraditional fundamentals, but in ways that are more likely to resultin students learning them. The approach advocated in these docu-ments is reinforced by the findings of science educators and cogni-tive researchers, demonstrating that if students do not learn deeplythe concepts presented to them, they end up passing through andover the K-12 curriculum without fundamentally changing theconceptual models they learn in their early years.

While it is easy to argue for a test that emphasizes conceptualunderstanding rather than topical listings or the recognition ofdefinitions from a list of choices, it is more difficult to agree onhow science learning should be assessed. The science courses thatwere developed in the late 1950s and early 1960s used deeperconceptual organizing principles for their texts and teachingmaterials. These programs, funded by the National Science Founda-tion, became major influences on the country's secondary sciencetexts from 1960 to 1970 and helped raise standards of students'science achievement (Shymansky et al., 1983). Unfortunately,however, the old-style textbooks have since returned, with ever-expanding numbers of pages to include as many newly emergingtopics as possible. The effect on student learning of science has notbeen beneficial. For 1996, the NAEP Science Assessment Frame-work incorporates a balance of knowledge and skills based oncurrent reform reports, exemplary curriculum guides, and researchon the teaching and learning of science.

1912

Framework Elements

The Framework for the 1996 NAEP Science Assessment isorganized according to two major dimensions: the Fields of Sci-ence and Knowing and Doing Science. The Fields of Science arethe Earth, physical, and life sciences; Knowing and Doing Scienceencompasses conceptual understanding, scientific investigation,and practical reasoning. The two dimensions and subdimensionsare explained in greater detail below and in appendix A.

The Matrix

The 1996 Science Framework is structured according to a matrixsimilar to that used in the 1990 NAEP assessment. The contentareas are organized into the same three fields; however, there is anadditional requirement for some interdisciplinary exercises thatmerge technology with the science content areas. The "Nature ofScience" (which also was part of the 1990 Framework) and"Themes" (which is new in the 1996 NAEP Science Assessment)are categories that should integrate the three fields of science, rather

Knowingand Doing

ConceptualUnderstanding

Scientific

Investigation

PracticalReasoning

Earth

Fields of Science

Physical Life

Nature of Science

ThemesModels, Systems,Patterns of Change

13 20

than represent separate content. The "Knowing and Doing" dimen-sion is a reorganization of the "Thinking Skills" in the 1990 assess-ment, with a clearer delineation of subcategories, particularly withrespect to "practical reasoning." Each of the elements of the 1996Framework is addressed briefly below and in greater detail inseparate appendices.

With respect to the major Fields of Science, the main emphasisof the assessment should be on knowledge in the content areas.

Distribution of content across the three science fields shouldbe approximately equal in grades 4 and 12.

For grade 8, the Assessment Framework places a somewhatheavier emphasis on life science (40 percent in terms ofcontent), with physical and Earth sciences distributed equally(30 percent each). The distribution for grade 8 reflects theimportance for this age group of human biology, whichincreasingly is recognized both in curriculum and instruction.

A limited number of exercises at every grade level shouldaddress technology because of its intimate relationship withscience. Although not every item need do so, exercises andtasks that draw from more than one discipline at the same timeare highly desirable, as they are more likely to mirror scienceproblems occurring in the real world.

More specific guidance on the distribution of content from thethree fields is given in the "Specifications" document.

The major emphasis with respect to Knowing and DoingScience should be on students' active expression of conceptualunderstanding.

At each grade level, 45 percent of content should be devotedto conceptual understandingthe ability to understandbasic concepts and tools used in the process of a scientificinvestigation.

Scientific investigationthe ability to use the appropriatetools and thinking processes in the doing of scienceshouldbe more heavily emphasized in grade 4 (45 percent) than ingrades 8 and 12 (30 percent at each of these grade levels).This is desirable because learning by doing plays a crucialrole for younger students, and ways of knowing in scienceneed to be introduced early.

21 14

Practical reasoning involves suggesting effective solutions toeveryday problems by applying scientific knowledge andskills. The ability to engage in practical reasoning is essentialif one is to understand complex societal problems or be ableto apply previous knowledge to an everyday problem. Theproportions for practical reasoning should be 10 percent atgrade 4, and 25 percent at grades 8 and 12.

Many exercises will involve more than one of thesubdimensions in Knowing and Doing Science.

The percentages cited above are not to be interpreted asimmutable but should serve as a general guide for test devel-opment. As new assessment tasks are developed, assessmentof conceptual understanding may become a part of exercisesthat measure scientific investigations and practical reasoning.

More detailed guidelines on the distribution of Knowing andDoing items are provided in the "Specifications" document.

Every question or task in the assessment should be classifiableby one or more subcategories in each of the two major dimensionsof the matrix. In addition to the two major dimensions, the Frame-work includes two other categories, which pertain to a limitedsubset of items. The first concerns students' understanding of theNature of Science.

Included in this category are the historical development ofscience and technology, the habits of mind that characterizethese fields, and the methods of inquiry and problem solving.

At least 15 percent of the content should measure the Natureof Science. Within this percentage, somewhat more than half(about 60 percent) should deal with the nature of science andsomewhat less than half (about 40 percent) with the nature oftechnology.

In specifying these percentages, it is assumed that these assess-ment items can be developed to do double duty in measuringknowledge of content within a field of science or an area of Know-ing and Doing Science, as well as the Nature of Science.

The second category, Themes, is new to the 1996 NAEP ScienceAssessment. Themes represent big ideas or key organizing conceptsthat pervade science. They cross the traditional science discipline

15 22

boundaries and comprise a group of inquiry tools that scientists useto better investigate and understand the phenomena with which theydeal. These themes include the notion of Systems and their applica-tion in the disciplines; Models and their functioning in the develop-ment of scientific understanding and its application to practicalproblems; and Patterns of Change as they are exemplified innatural phenomena.

The assessment should probe, in a developmentally appropri-ate way, students' understanding of these organizing ideas orthemes.

Students in grade 4 should build beginning notions related tosystems, models, and patterns of change; about one-third ofthe assessmentspread evenly across the three themesshould measure themes as well as content from one or moreof the fields of science.

Fifty percent of the assessment content in grades 8 and 12should assess students' understanding of the themes, spreadevenly across all three themes.

The assessment exercises must embed a given theme in thescience content that supports it, so that an understanding ofthe content and the theme are probed at the same time.

More detailed guidelines on assessing themes are provided in the"Specifications" report, which is used by the test developers tocreate the assessment.

Understanding, doing, and using science often involve tasks thatinclude more than one category in each dimension. Multiple-dutyexercises may present some scoring challenges. Relatively simpleexercises will be scorable according to one or two subdimensionsthat include, for example, a "conceptual understanding in the phy-sical sciences." More complex items may be scorable in severalsubcategories; for example, an open-ended task involving ecosys-tems might yield responses scorable according to "conceptual un-derstanding in the Earth sciences," "scientific investigation in thelife sciences," and "systems." Such items, which contribute to morethan one subcategory, may prove difficult to reproduce throughsimilar items having the same properties in future assessments. Itis important, therefore, to specify carefully the several subdimen-sions or subcategories that each such exercise is intended to probe.

2316

Scoring rubrics for each of the subdimensions also must bedeveloped.

The Fields of Science

The descriptions given here are capsule summaries of the majortopic areas to be probed within each field in the 1996 NAEPScience Assessment. This content represents key elements inscience that all students should be expected to know and under-stand. For a complete explanation, including descriptions of sub-ject matter knowledge to be expected at each grade level, seeappendix A.

Earth ScienceThe 1996 NAEP Science Assessment will probe student under-

standing of how Earth scientists depict data through maps and othermeans to interpret objects, their features and structures, and theevents and processes that caused them. What do students knowabout their own position with respect to objects and structures on,below, and above the Earth's surface? What do children knowabout changes in position of objects and environments throughtime? What do students know about the relative movements of theEarth, Moon, Sun, and the planets. The content to be assessed inEarth science centers on objects and events that are relativelyaccessible or visible: Earth (lithosphere), water (hydrosphere), air(atmosphere), and the Earth in space. With respect to Earth science,the 1996 NAEP Science Assessment should center on the followingconcepts and topics.

Solid Earth:

Composition of the Earth

Forces that alter the Earth's surface

Rocks: their formation, characteristics, and uses

Soil, its changes and uses

Natural resources used by humankind

Forces within the Earth

17 24

Water:

Water cycle

Nature of the oceans and their effects on water and climate

Location of water, its distribution, characteristics, and effectof and influence on human activity

Air:

Composition and structure of the atmosphere, includingenergy transfer

Nature of weather

Common weather hazards

Air quality and climate

Earth in Space:

Setting of the Earth in the solar system

Setting and evolution of the solar system in the universe

Tools and technology that are used to gather informationabout space

Apparent daily motions of the Sun, the Moon, the planets,and the stars

Rotation of the Earth about its axis, and the Earth's revolutionaround the Sun

Tilt of the Earth's axis that produces seasonal variations inclimate

Earth as a unique member of the solar system that may beapproximated in other galaxies in the universe, and thatevolved at least 4.5 billion years ago

Physical ScienceThe physical science component of the 1996 NAEP Science

Assessment relates to basic knowledge and understanding concern-ing the structure of the universe as well as the physical principles

25 18

that operate within it. The assessment should probe the followingmajor topics: matter and its transformations, energy and its trans-formations, and the motion of things. The 1996 NAEP ScienceAssessment should center on the following physical scienceconcepts:

Matter and Its Transformations:

Diversity of materialsclassification and types, particulatenature of matter

Temperature and states of matter

Properties and uses of materialmodifying properties,synthesis of materials with new properties

Resource management

Energy and Its Transformations:

Forms of energy

Energy transformations in living systems, natural physicalsystems, and artificial systems constructed by humans

Energy sources and use, including distribution, energyconversion, and energy costs and depletion

Motion:

An understanding of frames of reference

Force and changes in position and motion

Action and reaction

Vibrations and waves as motion

General wave behavior

Electromagnetic radiation

Interactions of electromagnetic radiation with matter

Life ScienceThe fundamental goal of life science is to attempt to understand

and explain the nature and function of living things. During the20th century, the focus of biological research has changed from

2619

descriptive natural history to experimental investigation, withevolution as the central, unifying theory. The major concepts to beassessed in life science are listed below.

Change and Evolution:

Diversity of life on Earth

Genetic variation within a species

Theories of adaptation and natural selection

Changes in diversity over time

Cells and Their Functions:

Information transfer

Energy transfer for the construction of proteins

Communication among cells

Organisms:

Reproduction, growth, and development

Life cycles

Functions and interactions of systems within organisms

Ecology:

Interdependence of lifepopulations, communities, andecosystems

Knowing and Doing Science

In the 1990 NAEP Science Assessment, three categories wereused: "knowing science," "solving problems," and "conductinginquiries." For the 1996 NAEP Science Assessment, it should benoted that not only has the "knowing science" dimension beenchanged, its meaning has been redefined as well. It has beenreformulated as "conceptual understanding of science" to stress theconnections as well as the organization of factual knowledge inscience.

2? 20

The subdimension of "scientific investigation" has been substi-tuted for the 1990 category of "conducting inquiries." This newsubcategory is intended to probe students' abilities to use the toolsof science, including both cognitive and laboratory tools. Appro-priate to their age and grade level, students should be able toacquire new information, plan appropriate investigations, use avariety of scientific tools, and communicate the results of theirinvestigations.

The 1990 performance category "solving problems" proved to beambiguous. To emphasize the need to assess students' ability to useand apply science understanding in new, real-world applications,this category has been redefined as "practical reasoning." Practicalreasoning subsumes competence in analyzing a problem, planningappropriate approaches, evaluating them, carrying out the re-quired procedures for the approach(es) selected, and evaluatingthe result(s). Each of these subdimensions is described below.

Conceptual UnderstandingMastery of basic scientific concepts can best be shown by a

student's ability to use information in conducting a scientificinvestigation or engaging in practical reasoning. Optimally, essen-tial scientific concepts involve a variety of information, including:

Facts and events the student learns from science instructionand experiences with the natural environment

Scientific concepts, principles, laws, and theories thatscientists use to explain and predict observations of thenatural world

Information about procedures for conducting scientific inquiry

Procedures for the application of scientific knowledge in theengagement of practical tasks

Propositions about the nature, history, and philosophy ofscience

Kinds of interactions between and among science, technology,and society

2821

The goal of school science is to engender conceptual understand-ing. Students should acquire a data base composed of informationstructured in ways that will enable them to apply it efficiently in thedesign and execution of scientific investigations and in practicalreasoning.

A challenge in the design of assessment exercises is to capturechanges in the characteristics of student performance as childrenmature. In the primary years, when the goal of school science isto build a rich collection of information derived from examinedexperiences with the natural environment, the assessment ofconceptual understanding will focus on the breadth of informationabout the natural world and the student's ability to elaborateprinciples with personal experiences. Does the student know thecyclical changes in the apparent shape of the Moon over time?More importantly, can the student relate how he or she knows aboutthe changes? What evidence does the assessment exercise providethat the student's information is based on direct experience? Is therea science notebook in which the student recorded observations ofthe Moon over time? Does the student know that sometimes theMoon is visible during daylight hours? In the primary years, thefocus should not be on explanation or predictionrather, onknowledge obtained from rich experiences in school. Consequently,assessment exercises would not be concerned with having studentsexplain why the Moon appears to change shape but rather withrelationships between time of day, apparent positions of the Sunand the Moon, and times of moonrise and sunset.

In the middle and high school years, the emphasis should shiftfrom richness of experience to reasonable scientific interpretationof observations. In the elementary years, the primary concernshould be with how well reasoned an interpretation the studentpresented, not with whether it reflects the most sophisticatedscientific reasoning. However, at grades 8 and 12, the assessmentshould be concerned increasingly with the congruence of thestudent's interpretations with accepted interpretations, as well aswith the sophistication of their reasoning in moving from observa-tions of the natural world to explanation and prediction. Of specialinterest in the 1996 NAEP Science Assessment will be the extent towhich the student is able to understand and use the notions ofmodels, systems, and patterns of change.

29 22

It is important to note that many aspects of conceptual under-standing as defined for the 1996 NAEP Science Assessment cannotbe tested using exclusively multiple-choice items. Items of this kindmay be satisfactory for assessing individual parts of the informationbase, but they are limited in tapping highly valued aspects ofconceptual understanding.

Scientific InvestigationScientific investigation represents the activities of science that

distinguish it from other ways of knowing about the world. Itincorporates such previously used assessment categories as the"processes of science" and "scientific problem solving". Thiscategory is not just another name for "the scientific method."Indeed, there is great confusion about the scientific method in theteaching of science. Real science is doing what one can in any wayone can, often creative and insightful, often involving flashes ofinsight with little regard for a progression of steps. However, thereis a familiar format and context for reporting the results of experi-ments. It begins with the report of the problem, continues with thehypothesis, the experimental design, the data collected, the analysisof those data, and the conclusions, if any. This convention ofscience is often mistaken for how scientists actually work. Theresults must satisfy logical analysis, but the logical ordering mayappear only when the report is prepared. A great disservice hasbeen done to generations of students because well-meaning peoplehave taught the standard method of reporting science as thestandard method of doing science.

Scientific investigations must be designed at levels appropriateto the development of the students. This has important implicationsfor assessment. Young students are limited in their ability toperceive the scale of things, both very large and very small. Stu-dents' limitations handicap them when they are forced, either bythe textbook or by the curriculum, to deal with developmentallyinappropriate concepts such as atoms or even cells. Young studentsalso are developmentally limited in their ability to understand time.The distant past and the future are narrowly perceived by theegocentric student. Instruction as well as assessment must recog-nize where the student is and take developmental levels intoaccount. As students develop and accumulate experience, theirperformance in doing scientific investigations should begin tolook more and more like "real" science.

23 30

Central to the ways a scientist works is the concern for a fair test,a controlled experiment. Children seem to have an intuitive senseof what makes a fair test. What they lack is the ability to considerall the variables and the means to control the variables. It might bereasonable to consider a developmental continuum such as thefollowing when thinking about the control of variables.

This continuum begins with the simplest type of variablethenominal variable. Nominal variables have two or more unorderedvalues: "This plant was watered; that plant was not." "This seedwas placed in the sunlight and that one was placed in the dark." Thesecond level of controlling variables is the ordinal variables level.These variables have a sequential order and no determined intervals(e.g., the sequential ordering of objects by relative weight). Thethird level of variables is called the continuous variables level.These variables have sequence and equal intervals and are on acontinuous scale. "This object has a temperature of 50 degreesCelsius, and that object has a temperature of 57 degrees Celsius."The fourth level of variables is called the ratio variables level. Theyare similar to the continuous variables but have an absolute begin-ning point (e.g., Kelvin temperature scale with an absolute zeropoint).

As students are asked to demonstrate their ability to do scientificinvestigations, it is important to keep in mind this sort of develop-ment in understanding and performance, not just with respect to thecontrol of variables but also regarding the other elements of doingscience. The difficulty with the assessment may not be the contentbut the level of variable imbedded in the content.

Practical ReasoningPractical reasoning about matters with scientific content, the

ability to apply one's lcnowledge, thought, and action to realsituations (not "textbook problems") is influenced by the abilityto: (1) abstract and consider hypothetical experiences; (2) considerseveral factors simultaneously; (3) take a depersonalized view;and (4) realize the importance of practical reasoning and lifeexperience. These factors develop throughout life.

One of the characteristics of young children is that they havedifficulty dealing with multiple ideas simultaneously. With maturityand experience, they can consider several ideas at once and weighbenefits in relation to costs or risks. The ability to abstract and to

31. 24

consider hypothetical situations develops as students progressin science and learn to deal with more remote phenomena andgeneralizations.

As they mature, students also learn to take a depersonalized viewof a situation and to consider someone else's point of view. Oftenreal-life problems involve not only the theoretical and technicalelements, but also personal preferences. What will be the socialimpact of a new waste disposal system? What will neighbors sayif a traffic light is installed? How will other students react if thelunchroom noise is diminished by staggering the lunch hour? Toconsider these questions carefully, it is necessary to understanddifferent perspectives. Understanding the viewpoint of othersgrows with age and experience.

Young children also may not realize the need for scientificinformation in solving problems. For example, children below theage of 12 usually see no need to carry out measurements (Strang,1990). Also, because young children have little responsibilityfor decisions affecting their lives, they may not see the need forpractical reasoning. However, the more students have done or seen,the more likely it is that they can solve real-world problems. Withage and experience, the possibility increases that a new situationis analogous to a previous one and that the human, technical,and theoretical factors involved in a new situation have beenencountered before.

All these factors suggest that practical reasoning should becomea major factor in science assessment at grades 8 and 12 rather thanat grade 4. As students become eager to take control of their lives,wish to try out their understanding of the world, and progress indevelopment, practical situations related to their everyday life,school, and home provide excellent exemplars to demonstratescience-related practical reasoning. Thus, students might be askedto discuss problems such as noise abatement in the lunchroom, todesign a simple apparatus such as a flashlight or a burglar alarm,or plan a school garden.

By grade 12, students should be able to discuss larger science-and technology-linked problems not directly related to theirimmediate experience. Examples of these might be waste disposal,energy uses, air quality, water pollution, noise abatement, and thetrade-offs between the benefits and adverse consequences ofvarious technologies.

25 32

The Nature of Science

Knowledge of the nature of science is central to understanding ofthe scientific enterprise. Yet, oftentimes this category is relegated toa discussionor even rote memorizationof some version of the"scientific method." There is total agreement that the topic shouldplay a prominent role in the 1996 NAEP Science Assessment. Acontroversy did exist in the project committees concerning whether"The Nature of Science" is sufficient unto itself, or whether theFramework should include a separate section that deals with thenature of technology. The project committees were split on thisissue. However, all acknowledged that technology is integral tothe nature of science and ought to be included in the assessmentprovided it clearly does not exist as a separate subcategory withinthe assessment. Technology, then, will be measured as it relatesto science and the scientific enterprise.

ScienceBy grade 4, students should understand that science is tryingto find out what happens in the natural world. Through carefulobservation of objects and events, they should be able todevelop explanations for their observations. Students shouldalso understand that different people may notice differentthings, and therefore may explain things differently.

By grade 8, students should have acquired an understandingof the control of variables and the difference between showingthat conditions occur together and that they are causallyrelated. Students should grasp what makes for a good scien-tific explanation fitting all the relevant observations, suggest-ing what new observations to make, and explaining, as simplyas possible, a wide variety of observations.

By grade 12, students should demonstrate their knowledgeand understanding of the following:

Scientific conclusions are based on logic and evidence, butthere is no fixed series of steps that make up a "scientificmethod." Scientists try to invent explanations that arelogical and that fit observations, but these are subject tochange based on new evidence.

33 26

Explanations are most believable when they also accountfor observations that were not known to the explainer.

Scientists (like anyone else) tend to look for, pay attentionto, and cite evidence that supports what they alreadybelieve.

New conclusions require that scientists consider allpossible objections to their own findings.

Scientific organizations try to avoid bias and maintainquality by having scientists' reports of observations andexplanations judged by other scientists before they can bepublished.

Few human problems can be solved with scientificknowledge alone. Most are too complicated and involvevaluesabout which, science has little to say.

TechnologyStudents are surrounded by and interact with the man-made

world as much as with the natural world. Therefore, they mustdevelop an understanding of what shapes the design and develop-ment of the technologies that are a part of that world and their dailylives. Rather than being a content area, technology is embeddedwithin this section because of its close association with science.The following concepts are appropriate for assessment at thegiven levels.

By grade 4, students should understand that any designrequires making trade offs and that advantages anddisadvantages must be weighed.

By grade 8, students should understand that scientificknowledge often is useful in design and that much scientificinvestigation is done for the purpose of improving design.They should understand that there are often several ways tosolve a design problem and that possible solutions should beevaluated and justified in terms of their advantages anddisadvantages.

By grade 12, students should know that scientific knowledgemay help to predict consequences of one design or another,but that design decisions often depend upon human values that

273 4

are outside of science. They also should be able to applyscientific concepts to scientific, societal, and/or technicalconcerns. They should understand that every design has limitsand may fail if required to work outside of them.

Themes

Themes are the "big ideas" of science that transcend the variousscientific disciplines and enable students to consider problems withglobal implications. To understand the conceptual basis for thethemes that have been selected, students must begin to develop anunderstanding of major ideas by the 4th grade; they should continueto develop their understanding through the 8th grade, and by the12th grade should have the ability to integrate their knowledge andunderstanding.

The review of current state frameworks carried on in the courseof developing the new NAEP Assessment Framework revealed thatmany are based in part on crosscutting themes in science. Severalnational organizations, including the American Association for theAdvancement of Science, have issued reports that advocate theimportance of common themes. The number of themes defined inthese reports and state frameworks varies somewhat, but there isconsiderable agreement on what the common elements or big ideasof science are that should be understood by students as they com-plete their high school education. The decision by the NAEPScience Assessment committees to include themes underscorestheir emerging importance, as well as the necessity to integratethemes through programmatic threads in grades K-12. Three of thethemes are common to all the documents: Models, Systems, andPatterns of Change. These three themes will be included in the1996 NAEP Assessment because they constitute major, interdisci-plinary organizing principles of science. Further, they do notconflict or compete with the factual content of the various fieldsbut, rather, augment and help organize that information into acoherent intellectual framework.

Models of objects and events in nature are ways to understandcomplex or abstract phenomena. Models may be first attempts tohelp tease out the relevant variables in order to build ever moreuseful representations, or they may be highly refined for predictionsabout the actual phenomenon. Students need to understand thelimitations and simplifying assumptions that underlie the many

35 28

models used in the natural sciences. A model is likely to fit datawell only within a limited range of circumstances and to be mis-leading outside of that range.

Systems are complete, predictable cycles, structures, or pro-cesses occurring in natural phenomena, but students should under-stand that the idea of system is an artificial construction created bypeople for certain purposesto gain a better understanding of thenatural world or to design an effective technology. The construct ofa system entails identifying and defming its boundaries, identifyingits component parts and the interrelations and interconnectionsamong those parts, and the inputs and outputs of the system.

Regardless of the topic around which the Patterns of Changetheme is developed, students should be able to recognize patternsof similarity and difference, to recognize how these patterns changeover time, to have a store of common types of patterns, and totransfer their understanding of a familiar pattern of change to anew and unfamiliar situation. Appendix B contains more detail oneach of these three themes and the developmentally appropriateexpectations for students at grades 4, 8, and 12.

3 629

Chapter Three

Desired Attributes f the ssessment

Some Assessment Issues

NAEP functions to provide information about (1) the knowl-edge and scientific understanding of the nation's youth and(2) the features of science education programs that relate to

high levels of student achievement. Consequently, the design planfor the NAEP Science Assessment includes strategies for measure-ment in both of these areas. Research has produced practical andtheoretical knowledge that is important to the assessment designprocess.

The Nature of Testing and the Nature ofKnowledge and Learning

Except for early assessments, NAEP science tests have consistedprimarily of short-answer, paper-and-pencil questions that weremostly multiple choice. The previous assessments tended to focusondiscrete components of science, each of which was usuallylearned independently of the others. Hence, tests were made up ofindependent items, each comparable to the others and weighted thesame.

The Framework is based on a different view. It holds thatscientific knowledge should be organized to provide a structurethat connects and creates meaning for factual information, and thisorganization is influenced by the context in which the knowledge ispresented. Learning is perceived as an activity in which the learnerinteracts with the physical world, with peers and teachers, and withaccepted scientific knowledge. In this view, science proficiencydepends upon the ability to know and integrate facts into largerconstructs and to use the tools, procedures, and reasoning processesof science for an increased understanding of the natural world.

31 3

Rather than concentrate on facts in isolation, assessments willreflect the organization and structure of scientific knowledge andthe nature of learning in science. Because scientific knowledge isexpanding faster than can be accommodated by any curriculum,teachers and assessment designers must make choices about whattopics, concepts, and factual information to address. Consequently,the 1996 NAEP Science Assessment Framework concentrates onassessing students' ability to relate basic facts and concepts as wellas their ability to discuss and evaluate approaches to science-relatedproblems. The Framework also stresses that an assessment of whatstudents know and can do must employ techniques reflective of thenature of science.

Inferring Understanding From Student ResponsesTest items present students with tasks, which may require a

range of responses, from recall of factual information to perfor-mance of scientific investigations to complex reasoning. Based onanalysis of the responses, experts in the field make inferences aboutthe student's understandingthat is, the knowledge and reasoningskills that are assumed to have produced the response. The validityof these inferences is a central issue in assessment.

Responses to exercises designed to assess thinking or mentalprocessing are generally more difficult to interpret than responsesto items designed to assess factual knowledge. In practice, basingan assessment of the quality of mental processing on short re-sponses is problematic. Often the decision about the mentalprocesses applied is based only on the accuracy of the factualknowledge in the answer. When the answer is factually correct,the observer infers that the mental processes represent scientificreasoning (e.g., those mental processes that are necessary to under-stand information in the stem, to retrieve scientific knowledge frommemory, to reason from the stem to the correct response, or toeliminate incorrect responses). But this is not necessarily a validinference. An incorrect answer may be the result of misinformation,not flawed reasoning; a correct answer is not necessarily theproduct of sound reasoning. Illogical thinking or using a wrongassumption or incorrect information can produce a seeminglycorrect answer. Furthermore, a correct answer may not require anyhigher order thinking at all; it simply may have been recalled. Onlyif the student response includes some indication of how it was

38 32

obtained will those who score the assessment have informationfrom which to choose among alternative interpretations.

Science entails observing objects and phenomena in the naturalworld and collecting and interpreting information about them. Forthis reason, pencil-and-paper tests have been criticized as toolimited for assessing what students know and can do in science.Over the past several years, groups have developed assessmentexercises that engage students in "performance tasks" using scien-tific equipment and materials; student responses are recorded by anobserver or by the students themselves in written form. However,these exercises have limitations as well.

Indeed, in the course of developing the 1996 Framework,numerous examples of performance exercises were examined withrespect to science concepts and reasoning, but many did not standup to rigorous analysis. These exercises might have led scienceteachers and educators to draw faulty inferences.

The following is typical of a performance exercise that is coun-terproductive: The exercise requires a student to identify severalunknown substances by means of indicators, but the student isgiven minutely detailed directions for performing each step in theprocess of identification. Unfortunately, even if the answers arecorrect, the only inference to be drawn is that the student can followwritten instructions. A test item formulated with such detailed step-by-step directions reduces to zero the science understanding neededfor problem solving.

Many of the so-called performance assessment tasks that werereviewed turned out to be standard laboratory exercises which,again, were reduced to "follow-the-instructions" problems. Noinferences can be drawn from such exercises about a student'sknowledge of science or of its tools and procedures. To test higherorder thinking skillsa major goal of performance assessmentproblems need to be placed in new contexts, applied to new situa-tions, or have new elements introduced that preclude students'recalling what they have done before (Resnick, 1987).

Inferences in Assessment of Different Population Groups.Class, culture, ethnicity, gender, language ability, and access toquality instruction may influence the manner in which science islearned and the manner in which science attitudes and knowledgeare produced. Hence, individuals need opportunities to demonstrate

3339

knowledge or competencies in different contexts. NAEPshould investigate ways to address this issue in the next scienceassessment.

Assessment techniques that show group differences are morelikely to reveal problems with student learning and classroominstruction than with assessment, per se. However, this does noteliminate the assessment community's responsibility to the broadersociety. In addressing the issues of pluralism, multiple assessmentmethods may be more effective than any one methodno matterhow well it is developed.

Developments in Assessment

Currently, some state pilot-testing efforts are providing newideas about assessment exercise and task formats. These pilots alsoare aimed at assessing new types of information relevant to newcurriculum guides emerging in the states.

The experimental assessment work (much of it pioneered in theUnited Kingdom) uses new approaches, including performancetasks, open-ended tasks, and new types of multiple-choice questionsthat are thematic or conceptual or that ask students to explain theirchoices in short written answers. Moreover, state assessments areexperimenting with collecting information on other meaningfuloutcomes: sustained student work, proficiency in designing andconducting experiments, and fluency of ideas critical to the naturalsciences and related fields. They also are experimenting withinnovative reporting approaches. The performance measures arecreated through consensual judgment regarding what studentsshould know and be able to do at given grade levels or develop-mental stages.

Criteria for Assessing Learning andAchievement in Science

The Framework for the 1996 NAEP Science Assessment hasbeen developed according to the following broad guidelines:

By focusing on meaningful knowledge and skills, NAEPshould be a force in fostering progress as well as measuring it,enabling more students to learn more science.

3440

A range of assessment means must be used, including some inwhich the student is required to create and construct, not onlyto recognize and respond.

Assessment exercises should challenge students at develop-mentally appropriate levels to:

Explain commonplace natural phenomena using appropri-ate scientific theory, principles, and concepts.

Plan the investigation of a novel scientific problem.

Demonstrate understanding of the basic knowledgestructures of science by using the appropriate techniquesto connect concepts to each other and to the theory withinwhich they are embedded.

Demonstrate some understanding of pervasive crosscuttingthemes in science.

Solve practical problems by using the appropriate theories,principles, concepts, and techniques of science.

The assessment must be sensitive to the need and ability ofstudents to function in a variety of contexts.

Assessment exercises should use a variety of formats to allowstudents to display the wide range of competencies expectedas the outcomes of science education.

Assessment tasks that are larger than single items shouldbe analyzed in multiple ways, not restricted to providinginformation for single scales.

Test results should not be normalized; that is, students'outcomes should not be manipulated to fit a normaldistribution curve.

The assessment must have enough questions about enoughtopics to explore students' knowledge in depth.

Preliminary Achievement Levels in Science

Achievement levels describe how well students should performon the content and thinking levels required by the assessment.They evaluate the quality of the outcomes of students' educationin science at grades 4, 8, and 12 as measured by NAEP.

35 41

Three achievement levelsBasic, Proficient, and Advancedhave been defined for each grade level by the National AssessmentGoverning Board.

Basic denotes partial mastery of the knowledge and thinkingskills, but performance that is fundamental for adequate work ingrades 4, 8, and 12. Proficient represents solid academic perfor-mance and competency over challenging subject matter. If amajority of students performed at the Proficient level on thisassessment, the consensus committees believe they would havelearned enough science to be competent students and productivecitizens. Advanced performance on this assessment representsperformance that is equal to that expected of top students inother industrialized nationsthe ability to think critically aboutscientific issues and be able to integrate knowledge and skillsinto problem solving situations.

The following are preliminary achievement level descriptionsfor students participating in the 1996 NAEP Science Assessmentin grades 4, 8, and 12. Within each grade, it is assumed that eachhigher achievement level incorporates and builds upon the preced-ing levels. The assessment should be constructed to measureand report student performance according to the three levels ofachievementas required by NAEP policy. The followingdescriptions were developed by a panel of science experts, aftercompletion of the assessment framework. These panelists, manyof whom participated in the consensus process, are listed inappendix D.

4th-Grade BasicStudents at the basic level should be able to make simple measure-ments. For example, they should be able to use a thermometer, filla container to a specified point, and weigh an object using a bal-ance. Students at this level should be able to make comparativestatements about the physical properties of objects and be able toestimate length, weight, volume, and temperature. Students shouldbe able to describe and classify familiar objects and/or organismsand be able to describe their properties using the five senses. Theyshould be able to distinguish most living things from those that arenot living, and distinguish plants from animals. Students should beable to identify major features of the Earth's surface and about theEarth in space, and identify Earth resources used in everyday life.Given clear, sequential directions, students should be able toperform simple science tasks.

36

42

4th-Grade ProficientStudents at the proficient level should be able to conduct accuratemeasurements of temperature, volume, length, and weight. Whenstudents classify, they should be able to offer a reasonable justifica-tion for whatever system they use. Proficient skills and understand-ings should be demonstrated through written summaries of descrip-tive investigations of the natural world. While the proficient studentshould be able to conduct tests for properties that cannot be directlyobserved with the five senses (e.g., solubility, magnetic properties,conductivity), they should be able to apply concrete procedures

to solve abstract problems. Proficient students should be able tounderstand various concepts, such as the water cycle, the relation-ship of seasonal changes to weather conditions, and the motion of

the Earth in space.

4th-Grade AdvancedStudents at this level should be able to select appropriate measuresand measuring devices in the design of an investigation. Theyshould be able to propose multiple systems of classification thatrely on observable and appropriate criteria. Advanced studentsshould be able to appreciate the application of classification in theirdaily lives and recognize the advantages and disadvantages ofclassification systems (Which one is the best? Why?). When testingreasonable hypotheses, they should be able to justify their conclu-sions using observable evidence. Students should be able to relatestructure and function in living things and be able to cite advan-tages for the observed structure/function relationship. Advancedstudents should be able to describe ways to measure various naturalphenomena, such as weather changes, and to identify tools used togather information about space. They should be able to describesome relationships between and among the Sun, Moon, and Earthin the solar system.

8th-Grade BasicStudents should possess fundamental knowledge concerning boththe structure and function of human anatomy. They should knowthe main causes of common diseases. In addition, basic studentsshould be aware of their immediate environment including conceptsof the diversity of living things and food chains. In the physicalworld, they should be able to distinguish states of matter and

37 43

understand the basic properties and characteristics of matter. Theyshould be able to identify common energy sources and methods fortransforming energy. Basic students should be able to make accu-rate measurements and display the data. At the basic level, studentsshould be able to infer information from the simple tests they makeand apply facts and concepts to their everyday lives. They shouldbe able to identify forces that alter the Earth's surface; describe thecomposition of the Earth, its atmosphere, and climate; and describethe major features of the solar system and universe.

8th-Grade ProficientStudents should know and/or be able to collect basic informationand apply it to the physical, living, and social environments. Theyshould be able to link simple ideas in order to understand payoffsand tradeoffs. Proficient students should be able to understandcause and effect relationships such as predator/prey and growth/rainfall. Proficient students should be able to design experimentsto answer simple questions involving two variables, to isolatevariables, and to collect and display data and draw conclusionsfrom them. They should be able to draw relationships between twosimple concepts; they should be starting to understand relationships(such as force and motion and matter and energy); and they shouldbe beginning to understand the laws that apply to living andnonliving matter.

8th-Grade AdvancedIn addition to the ability to infer cause and effect relationships fromdata, advanced students should be beginning to visualize interactingsystems and subsystems at various levels. For instance, they shouldbe able to relate several factors (variables) to explain a phenom-enon. They should be able to describe many elements ofa system,select a particular example, and explain its limits. They should beable to understand more complex models and know that scientificmodels have limits. Advanced students should be beginning tounderstand the nature and limits of science, and that science issubject to change. Students at the advanced level should have abasic knowledge of genetics, cells and their communicationssystems; know basic laws of probability and be able to expressthem quantitatively. They should be able to describe basic chemicalprocesses and how chemicals and classes of chemicals interact andaccount for physical properties in terms of their physical state and

4 4 38

atomic structure. They should be able to understand more abstraOtconcepts/theories related to the Earth's climate, atmosphere, thesolar system, and the universe. The advanced student should be ableto manipulate variables and form hypotheses in the abstract as wellas in concrete settings.

12th-Grade BasicStudents should know and understand fundamental facts andconcepts concerning the everyday world, including basic character-istics of matter, diversity of life, heredity, cells, and interdepen-dence of life. They should be able to relate concepts such as therelationships between living things in a simple ecological system.They should be able to predict some short-term effects of changesin the systems. Students at the basic level should apply scientificprinciples to their everyday lives; for instance, they should be ableto investigate ways people manage water as a resource. Theyshould be able to interpret a weather map, applying what they knowto an abstract diagram to make reasonable short-term predictionsabout weather. Students should be able to recognize the variousforms of energy and know how one form can be changed intoanother. At this level, students should begin to develop an apprecia-tion for the nature of science and its limitations. They should beable to distinguish (from a newspaper, for instance) articles basedupon scientific principles from those based on pseudoscience.When presented with a problem, students should be able to isolateseveral variables, design a simple experiment, collect and displaydata, and formulate inferences from the data.

12th-Grade ProficientStudents at the proficient level should be able to grasp scientificprinciples on both a qualitative and quantitative basis. Studentsshould be able to understand that scientific knowledge is tentativeand subject to change. For example, they should be able to giveexamples of ways in which models of the atom have changed overtime and how our understanding of cell structure has evolved.Students should be able to integrate several abstract facts in orderto understand overarching scientific principles. They should beable to apply basic principles to human activitiesfor example,genetics to basic principles of genetic engineering in relation todrug and hormone production. Students at the proficient levelshould be able to relate properties of materials to simple models

39

45

of their molecular structure. They should be able to understand therelationships among various concepts, such as acceleration, mass,force, and kinetic energy, and have the ability to apply them in awide range of contexts. They should be able to describe conditionsunder which basic chemical reactions take place. These studentsshould be able to identify more than one way to solve a givenproblem and select the method with the most promise. At theproficient level, students should be able to manipulate data throughvarious mathematical models.

12th-Grade AdvancedStudents at this level should have a quantitative grasp of scientificprinciples, facility in relating them to one another and to phenom-ena, and awareness of their development and limitations. Studentsshould be able to formulate scientific questions, compare differentexperimental designs, devise experimentally valid protocols toanswer their questions, collect the relevant quantitative data usingappropriate instrumentation, and provide a scientifically validinterpretation of the data they have collected. Moreover, theyshould be self-critical, be able to discard unnecessary data, recog-nize gaps in information, and know where to locate the neededinformation in primary or secondary sources. Students at theadvanced level should have a well-developed knowledge baseconcerning the universe, the Earth and the forces that shape it, andthe basic functions of life at the cell and organism levels. Withinthese contexts advanced students should be able to express theirideas mathematicallyinterpolating, extrapolating, and interpretingpatterns of change in graphic and symbolic representations.

46 40

Chapter Four

Characteristics of AssessmentExercises

Types of Exercises

Innovative assessments in this and other countries use threemajor item types: performance exercises, open-ended pencil-and-paper exercises, and multiple-choice items probing understandingof conceptual and reasoning skills. A fourth type often added totime-limited tests is the collection and evaluation of portfolios ofstudent work done in the course of instruction. There is also anemerging assessment technique that involves two-phase testing.The following sections discuss these exercise types and providesome guidelines for the amount of assessment time to be devotedto each. Further details are provided in the "Specifications" and"Reporting Formats and Issues" documents.

Performance ExercisesIn performance exercises students actually manipulate selected

physical objects and try to solve a scientific problem about theobjects before them. Although various types of performance testshave been piloted extensively, their standardization and administra-tion differ widely. One method for ensuring uniform administrationis the use of standardized performance test kits, with each testproctored and scored by trained personnel. Depending on theobjectives established for the assessment, student answer sheetsalso can be used to provide responses for scoring. To measure thegoals outlined for the assessment adequately, performance itemsgenerally should make up at least 30 percent of the assessment, asmeasured by student response time. An extra period of time (20 or30 minutes) may be necessary for students assigned complexperformance tasks.

41 47

The shortcomings of many performance tasks currently beingused were discussed in the preceding chapter. How could a perfor-mance exercise be designed so that it meets criteria for assessingscience concepts and their relations? The problem should bemeaningful and not a context-free laboratory problem. Personalcontext, for example, is seen in the following problem: "You havejust been given this new drink. It is claimed to be sugar- andcalorie-free. What can you find out about the accuracy of theseclaims with these indicators?" Students can be given the namesand procedures for the safe use of each indicator, but without anyinformation as to their scientific use. The students would have toknow, for example, that iodine solution is a test for starch and whatthe negative and positive reactions are. They would also have toknow that if fats, protein, or carbohydrates are present, they willyield calories. They would have to plan how to conduct an investi-gation of the unknown in such a way as not to waste it, to be ableto repeat the investigation when they believe their procedure wasfaulty, and to have enough solution left to replicate for verification.They would also have to design their data-collecting procedures.Finally, they would need to interpret and justify their findings.The questions asked of students as part of a performance exerciseneed to enable the students to display understanding and to justifyinterpretations. Such questions as "What substance is in the un-known?" or "How far did the dye diffuse?" do not elicit suchresponses.

If students need additional information to carry out the task,they could be asked before they begin if they would like any othermaterials and why. If they request known substances for eachindicator to refresh their memories, those could be provided. Sucha request demonstrates one aspect of understanding the processesof scienceknowing what one doesn't know and how to find out.Scoring for such a problem could give points for science knowl-edge, for laboratory procedures, and (if using an observer) for asystematic approach to problem solving in contrast to a trial anderror or random approach.

Open-Ended Pencil-and-Paper ItemsOpen-ended items that require written responses provide parti-

cularly useful insights into students' levels of conceptual under-standing. They also can be used to assess students' ability tocommunicate in the sciences. In addition, open-ended items, if

48 42

carefully crafted, can be used to reflect students' ability to generaterather than recognize information related to scientific concepts andtheir interconnections. Open-ended items should make up at least50 percent of the assessment, as measured by student response time.About one-third of the open-ended questions should consist ofextended response items.

Multiple-Choice ItemsThe 1996 NAEP Science Assessment will send important

messages about the science curriculum and classroom instruction.The use of multiple-choice items should be considered carefullybecause they are often overused to test low-level recall. Balancedwith other item types, however, multiple-choice items are worth-while for measuring important facts, concepts, and deductivereasoning skills. Multiple-choice items should comprise no morethan 50 percent of the assessment, as measured by student responsetime.

Additional ConsiderationsMultiple-choice items, open-ended paper-and-pencil items, and

performance tasks could produce responses less subject to faultyinterpretation if students were given an opportunity to explain theirresponses, their reasoning processes, or their approach to a prob-lem. Hence, the assessment should afford this opportunity. But caremust be taken, particularly with 4th graders, that language ability isnot confounded with science ability. This caution also applies to themore complex multiple-choice items needed to probe conceptualunderstanding.

The 1996 NAEP Science Assessment, to be consonant withcurrent reform efforts in science education, needs to probe students'depth of knowledge and scientific understanding. For this reason,it is recommended that for at least half the students sampled, theassessment include an indepth examination involving a singleproblem or topic. The format could be a set of linked multiple-choice items, open-ended paper-and-pencil exercises, performancetasks, or a combination thereof. Pending modification after pilottesting, the suggested time to be spent by students on this type ofexercise is 10 minutes for grade 4; 20 minutes for grade 8; and 30minutes for grade 12.

43 49

Pilot Testing

Multiple approaches need to be tried in the pilot testing of theassessment exercises. It would be especially useful to test the sameconcept(s) and performance skill(s) in different ways to see whichmethod provides the richest, most reliable, and valid information.For example, if an open-ended question can easily be turned into amultiple-choice question without losing its intent and validity, itshould be multiple choice. Open-ended questions should tap skillsand knowledge that are truly "open"probing for the integratedapplication of relevant knowledge, not for recall of a series ofunconnected facts. The following additional issues need to beinvestigated during pilot testing:

Scoring rubrics. These should be developed a priori for open-ended questions and performance tasks, but modified on thebasis of pilot test results. Weights should be assigned withinscoring rubrics of complex items to reflect the quality of theresponses. Distinct rubrics must be developed to score formultiple aspects or for items contributing to more than onecomponent of the assessment.

Scaffolding. To what extent should scaffolding or providingadditional information be integrated into open-ended ques-tions? This could be done either in the form of "hints" (to begiven only after the student gets "stuck," for example, in acomputer-administered or individually timed exercise) or byinforming the students how their answers will be scored.How much should students be able to learn during the test, ascontrasted to what they learned in the classroom? Unfortu-nately, many science items in current tests require no scienceknowledge at all since everything is provided in the stem or inthe instructions. On pencil-and-paper tests, these questionsusually turn out to measure general reasoning or scale andgraph reading.

Student self-evaluation. Should students (perhaps in grades 8and 12 only) be able to evaluate their own test performance?Students might be given credit for what they know they don'tknow, particularly if they can articulate what steps they mighttake to find out. Alternatively, they might be asked to indicatehow well they think they did on some number of items(meta-cognitive questions).

44

Procedures for Development of the Assessment

The Planning Committee responsible for developing the 1996NAEP Science Assessment Framework was concerned that thenature and specifics of the Framework be faithfully mirrored in theactual assessment instrument. The Committee therefore recom-mended a detailed review of individual items and the proposedassessment as a whole be conducted at the conclusion of each ofthe following four stages:

Item development and selection for the pilot testing.

Analysis of the data from the pilot testing and review of theresults.

A priori development of the several scales at each grade level,together with review of the items used for the behavioralanchoring.

Selection of items and formulation of the assessment as awhole for the 1996 test administration.

The review committee(s) should be broadly constituted toinclude scientists, teachers, science educators and researchers,cognitive psychologists, psychometricians, and informed membersof the lay community, including representatives from business,higher education, education governing bodies, and parent groups.The several stages of review should be carried out by the samecommittee. If this is not feasible, the four groups need to haveoverlapping membership. Members of the Planning Committeeshould be included throughout the entire review process.

Special Study in Science

Special studies are often recommended as a part of the NationalAssessment process because new or emerging techniques offerpromise, and, if they yield useful information, will make a positivecontribution to the assessment process. Special studies are a part ofthe main NAEP process and usually are reported along with theresults from the national sample.

In 1996, a special study will be carried out to assess studentswith advanced training in science. Past NAEP science assessmentshave been criticized for having too low a ceiling; that is, not

45 51

including an adequate number of items at advanced levels ofdifficulty. As a result, NAEP tests are assumed not to have reflectedwhat the best prepared students know or can do in science. Thisissue has become more serious since the formulation of the Na-tional Education Goals, particularly Goal 4: "By the year 2000,U.S. students will be first in the world in science and mathematicsachievement." An international assessment being planned for 1995will include students who are concentrating on science in highschool. It would be highly desirable to have similar results fromNAEP so that reports on progress toward the NationalGoals willbe based on more than one study.

Suggestions on how to include the most competent students inthis special study include sampling students who are taking ad-vanced courses in science. The assessment must contain a sufficientnumber of challenging exercises to measure what these "best"students know and are able to do. It is recommended, therefore,that a special study be conducted in 1996 with a subsample of thenational NAEP sample to determine whether this is a useful ap-proach to establish the achievement and performance of the bestscience students. Results should be included in the national reportof NAEP results.

5246

References

Aldridge, B.G. (1989). Essential Changes in Secondary SchoolScience: Scope, Sequence and Coordination. Washington, DC:National Science Teachers Association.

American Association for the Advancement of Science (1989).Science for All Americans: A Project 2061 Report On LiteracyGoals in Science, Mathematics, and Technology. Washington,DC: American Association for the Advancement of Science.

American Geologic Institute (1991). Earth Science Education forthe 21st Century: A Planning Guide. Alexandria, VA: AmericanGeological Institute.

Bybee, Roger W., et al. (1989). Science and Technology Educationfor The Elementary Years: Frameworks for Curriculum andInstruction. Washington, DC: The National Center for Improv-ing Science Education.

California Department of Education (1990). Science Frameworkfor California Public Schools: Kindergarten Through GradeTwelve. Sacramento, CA: California Department of Education.

Davis, F. (1990). "Assessing Science Education: A Case forMultiple Perspectives." In G.E. Hein, ed., The Assessment ofHands-On Elementary Science Programs. Grand Forks, ND:North Dakota Study Group, 1990.

Frank, Phillip (1957). Philosophy of Science, the Link BetweenScience and Philosophy. A Spectrum Book. Englewood Cliffs,NJ: Prentice-Hall.

National Center for Improving Science Education (1990). Scienceand Technology Education for the Middle Years: Frameworksfor Curriculum and Instruction. Washington, DC: The NationalCenter for Improving Science Education.

National Center for Improving Science Education (1991). The HighStakes of High School Science. Washington, DC: The NationalCenter for Improving Science Education.

National Research Council (1990). Fulfilling the Promise: BiologyEducation in the Nation's Schools. Washington, DC: NationalAcademy Press.

47 53

National Science Board Commission on Precollege Education inMathematics, Science and Technology (1983). EducatingAmericans for the 21st Century. Washington, DC: NationalScience Foundation.

Resnick, L. (1987). Education and Learning to Think. Washington,DC: National Academy Press.

Shymansky, James A., W.C. Kyle, Jr., and J.M. Alport (1983)."The Effects of New Science Curricula on Student Perfor-mance." Journal of Research in Science Teaching.20(5): 387-404.

Stiggins, Richard J. (1987). "Design and Development ofPerformance Assessment." Educational Measurement: Issuesand Practices, 33-42, Fall).

Strang, J. (1990). Measurement in School Science. London:Assessment Performance Unit, School Examination andAssessment Council, Central Office of Information.

5448

Appendix AFields of Science

55

Fields of Science

The detailing of the fields of science below indicates themesthat could be explored within major topic areas as well asbrief examples of developmentally appropriate sample

questions that could form the basis for assessment exercises.

Earth Science

Earth science is the study of the planet Earth's composition,process, environments, and history, focusing on the solid Earth(lithosphere) and its interactions with air (atmosphere) and water(hydrosphere). The content to be assessed in Earth science centerson objects (that include bodies and materials, their composition,features, and structures), as well as processes and events that arerelatively accessible or visible. These include objects such as soil,minerals, rocks and rock outcrops, fossils, rain, clouds, the Sun andMoon; processes such as erosion and deposition and weather andclimate; and events like volcanic eruptions, earthquakes, andstorms.

The Solar System (subset)

Earth /Moon/Sun

Observable evidence (Patterns of Change)

Description, models (Systems, Models)

Students should understand that the apparent daily motions ofthe Sun, the Moon, the planets, and the stars are due to the rotationof the Earth about its axis every 24 hours. This rotation producesthe Earth's night-and-day cycle. They should also understand thatthe Earth's 1-year revolution around the Sun, because of the tilt ofthe Earth's axis, changes how directly sunlight falls on one part oranother of the Earth, thus producing seasonal variations in climate.They should know that the combination of the Earth's motion andthe Moon's own orbit around the Earth, once in about 28 days,results in the phases of the Moon.

51 56

Sample Questions:

Grade 4Draw a shadow, specifying direction and length, at different

times of day. Where does the sunshine fall in the classroom atdifferent times of the year? Where is the Moon?

Grade 8In which direction should an open-air sports field (e.g., football,

tennis, baseball) be oriented, considering the position of the Sunand season in which the sport is played? Where is the Sun when wesee the Moon at each of its various phases? Is the Moon full on thesame day everywhere on Earth?

Grade 12How long an overhang does one need on a house to admit

maximum sunshine in winter and provide appropriate shade insummer? Why don't we spin off the rotating Earth? Why doesn'tMercury fall into the Sun? Do the horns of the Moon's crescenthave the same orientation at different latitudes? Longitudes? Why?

The Earth and Forces That Shape ItWith respect to the Earth, the NAEP Science Assessment should

center on the following concepts:

Climate (Patterns of Change)

Water Cycle and Ground Water (Systems)

Pollution Capacity (Systems, Models)

Interior Effects (Models, Systems, Patterns of Change)

Crustal plates

Rock cycle and strata

Exterior Effects (Systems, Patterns of Change)

Weathering

Plants, animals, civilization

Students should understand that the Earth is a unique memberof the solar system but may be replicated in other galaxies in the

57 52

universe; it is at least 4.5 billion years old and is a complex planetwith several interacting systems; these systems have evolvedthrough time, and changes in them occur over periods of microsec-onds to millions of years and vary from subatomic to astronomical.Students also should understand that the Earth's systems contain avariety of renewable and nonrenewable resources that sustain life(American Geologic Institute, 1991).

Climate: Atmosphere and HydrosphereThe atmosphere is the gaseous envelope that surrounds the

Earth; it is continuously in motion, circulating in complex butregular patterns and driven by direct and stored solar energy. Thereare strong interactions between the atmosphere and the hydrospherethat determine weather and climate, profoundly influencing humanand all other life. Desired learning goals:

Grade 4Students are able to communicate what is special about air.What do our senses tell us about the air? What needs air?

Students will offer simple explanations for how the weatherchanges. How do we know when weather changes? How canwe measure changing weather conditions?

Grade 8Students know about basic weather-related phenomena:(e.g., tornadoes, hurricanes, drought, acid precipitation).

Students know that relatively small changes in globaltemperatures could have dramatic effects on the Earth.

Students can access climatological information bases viacomputers and other means to extract useful information.

Students can read a weather chart and can extract basicinformation.

Students understand and can use simple instruments measur-ing basic phenomena related to weather (e.g., barometershave made one, know how they work).

5853

Grade 12Generally, 12th-grade students are able to connect relationships

between atmospheric phenomena and long-term effects. Theyunderstand that much of what determines the details of the weatherdepends on phenomena such as sea breezes, thunderstorms, torna-does, and wind shear. Further, they understand how scientists canmonitor atmospheric events over time (e.g., products of pollutionand carbon dioxide deposition at the poles). Students are ableto discuss and relate causes and possible solutions with theirconsequences and trade offs.

Water Cycle and Ground WaterDesired learning goals:

Grade 4Students understand that water exists not only on the surfaceof the Earth, but beneath it as well.

Students understand that water is able to change the shape ofthe Earth. They can demonstrate their knowledge in two ways:by interpreting common local land features such as rivers,streams, mountain slopes, and delta deposits. And they candemonstrate their knowledge by designing simple modelsthat illustrate what they are attempting to show.

Students understand that most of the Earth's surface iscovered by water.

Students know about how waters of the Earth circulate. Theycan answer questions concerning where we find water; howwater enters and leaves the atmosphere; and how we can usewater more wisely.

Grade 8Students should know and demonstrate why water is special.What properties make water special? Where is water foundin the air, on Earth, and under the ground?

Students know that the oceans provide habitats for a widevariety of plant and animal life.

Students can discuss some common problems that concernwater; its availability in their areas, shortages, relationship to

59 54

supply and demand, and the effects of over-population on theavailability and quality of potable water.

Students are able to relate common interactive cycles such aswater cycle, nitrogen cycle, and carbon cycle.

Students are familiar with some of the ways scientists explorethe water environment.

Grade 12At grade 12, students are able to explain how water (or the lack

of it) relates to their immediate state, city, or area. They can discussnational issues related to water, such as acid precipitation and theeffects of global warming. They are able to understand how com-mon cycles affect the climate of the area where they live. Studentsshould know how to find answers to problems relating to how lawsaffect our use of water, and what hazards are associated with waterand how can we mitigate them.

Interaction of Earth's SystemsDesired learning goals:

Grade 4Students understand how the Earth relates to the Sun(e.g., periodicity, seasons, night/day).

Students know about tides with respect to the Earth, Moon,and Sun.

Students understand basic facts about volcanos and glaciers.

Students understand how we can investigate rocks andminerals; what they are made of and how they form.

Grade 8Students are able to understand how earthquake occurrencesare recorded and note some positional regularities.

Students are able to connect short-term changes in climatewith volcanic activity.

Students can discuss changes that have occurred in waterlevels, global temperature, and climate zones over the eons.

55

60

They can advance some hypotheses as to why these changeshave occurred.

Students can identify how and where we get energy from theEarth; Earth materials that we use, and where to find them;and the advantages and disadvantages of using the Earth'sresources.

Grade 12Students in grade 12 can connect "the zone of fire" with plate

tectonics. They understand how sliding plates cause sudden earthmovements. They can discuss issues related to Earth systems(e.g., why continents move, that they have not always been ar-ranged the way they are now, how human activity affects Earthsystems). Students can access data from more than one data sourceto develop an environmental impact statement on their region/town/block. Students can discuss problems associated with agricultureand the lithosphere. They can relate these problems to changingatmospheric conditions.

Physical Science

The physical science component of the 1996 NAEP ScienceAssessment should probe the following major topics: matter and itstransformations, energy and its transformations, and the motion ofthings. Each is detailed below, together with some example topicsappropriate for assessment at the different grade levels.

Matter and Its TransformationsWith respect to matter and its transformations, the 1996 NAEP

Assessment should center on the following concepts:

"Many From Few"

Diversity of materials (Models)

Classifications of and types of materials (Patterns of Change)

Particulate nature of matter (Models)

56

61

Temperature and States of Matter

Uses of Materials

Properties and uses

Modifying properties by mixing, processing, and reacting(Patterns of Change, Models)

Synthesis of materials with new properties(Patterns of Change, Models)

Resource Management

Resource depletion, substitute materials (Models)

Disposal and recycling (Systems)

Materials encountered in the physical world differ greatly inshape, density, flexibility, texture, toughness, and color; in theirability to give off, absorb, bend, or reflect light; in the form theytake at different temperatures; and in many other ways. Studentsneed to understand that these varied substances are made up ofrelatively few kinds of basic materialsthe atomic elementscombined in various ways. Only a few of these elements areabundant in the universe.

As they advance in science, students should come to under-stand that the basic premise of the modern theory of matter is thatmaterials consist of a limited number of different kinds of atoms(elements) that join together in different configurations to formsubstances. Thus, when substances react to form new substances,the elements composing them combine in new ways and theproperties of the substances created by the new combinations maybe very different from those of the old. Almost every substance canexist in a variety of statessolid, liquid, and gaseousdependingon temperature and pressure.

Patterns within the structure of matter have been elegantlydescribed in the Periodic Table, an outstanding example of a modelthat provides a systematic view of matter and its interactions.Changes in temperature and changes in state represent a categoryof physical change among substances, within the realm of matterand energy.

6257

An understanding of the particulate nature of matter can beassessed through the identification of types of materials; for ex-ample, "mystery powders" or the equivalent at the elementary level.The effects of temperature and heat energy on systems are exempli-fied in how refrigerators work, how and why ice cubes melt, or howmetallic fuses work. Some practical areas of human activity (e.g.,cooking and much of modern chemical industry) and processes(transport of materials in biological systems) are related both to thenature of matter and the effects of external factors on its behavior.All these topics can provide rich assessment exercises.

The properties of matter determine uses to which particularmaterials are put by manufacturers, engineers, and others involvedin technology. Materials can be physically combined or processedto serve human needs. Modern materials technology has focusedincreasingly on the synthesis of materials with entirely new proper-ties. Chemical changes are typically involved, and the properties ofthe new materialssuch as plastics and ceramicsmay be entirelydifferent from those of its constituents.

The growth of technology has led to the use of some materialsfrom the environment (e.g., forests, ore deposits, petroleum) muchmore rapidly than they can be replaced by natural processes. Thereis a continuing search for substitute materialsand in many casesthey have been found or invented.

Disposal of used materials has become an increasing problem.Some used materials such as food scraps and waste paper can bereturned safely to the environmentalthough as the populationgrows, the task becomes more difficult and expensive. Othermaterials, such as aluminum scrap and glass can be recycled, withresulting savings in energy and resources. Some materials such asplastics are not easily recycled nor do they degrade quickly whenreturned to the environment. Other used materialsradioactivewaste being the most dramatic but not the only exampleare sohazardous for such a long time that it is not clear how best todispose of them. This issue has become the subject of widespreaddebate and controversy. Solving these problems of disposal willrequire systematic efforts that include both social and technologicalinnovations. Assessment questions dealing with the scientificand technological issues involved in resource management areappropriate for grades 8 and 12.

6358

Energy and Its TransformationsWith respect to energy and its transformations, the 1996 NAEP

Assessment should center on the following concepts:

Forms of Energy

Energy Transformation (qualitative) and Audits (quantitative) in:

Living systems

PlantsAnimalsProtistaNatural physical systems

Artificial (human constructed) systems

Energy Sources and Use

Quantity/kind

Distribution (Patterns of Change)

Energy conversions, heat gain/loss, and efficiency(Patterns of Change, Models)

Slowing depletion of energy sources (conservation)

Costs, implications, advantage, risks, availability(Patterns of Change)

Students should understand that the concept of energy is centralto understanding changes observed in natural and artificial systems.Observable changes occur when energy is added to a system, whenenergy is removed from a system, or when energy is transformedfrom one form to another. Energy appears in many forms and iscategorized in different ways: Light, Heat, Sound, Kinetic andPotential (Electromagnetic, Electrical, Chemical, Gravitational,Elastic), Consumable and Renewable, Available and UnavailableEnergy. Although various forms of energy appear very different,each can be measured in a way that makes it possible to keep trackof how much of one form is transformed into another.

Students need to understand that energy is conserved. Within asystem, whenever the energy in one place or form changes, the

5964

quantity of energy in another place or form increases or decreasesby a similar quantity but the total energy remains the same. Thus,if no energy leaks in or out across the boundaries of a system, thetotal energy of all the different forms in the system will not change,no matter what kinds of changes occur within the system.

Energy transformations usually result in producing somethermal energy (heat), which "leaks away" by radiation or conduc-tion (i.e., from engines, electrical wires, hot-water tanks, humanbodies, stereo systems) and becomes unavailable for furthertransformations. Thus, the total quantity of energy available fortransformation usually decreases.

Students need to develop an understanding of the more generalprinciple that natural processes occur in the direction of increasingthe total disorder of the system and its surroundings. Althoughsome subsystems do increase in orderliness (such as the freezingof water to form ice), another part of the system or a connectedsystem becomes more disordered. The cells of a human organism,for example, are always busy increasing order (i.e., buildingcomplex molecules and body structure). But this occurs at thecost of increasing disorder even more (i.e., breaking down themolecular structure and order of food we eat and in warming upour surroundings).

Energy transformations occur naturally and in devicesconstructed by humans.

Naturally occurring transformations:

Solar energy into stored energy such as starches, fats, andproteins

Solar energy into heat

Potential energy into kinetic energy (e.g., the potentialenergy of roller coasters at the top converting to kineticenergy on the way to the bottom)

Transformations occurring in human artifacts:

Electric mixer converts electrical energy into mechanicalenergy

Hair dryer converts electrical energy into heat energy

Automobile converts chemical energy into mechanical energy

6 560

In the operation of these devices, as in all phenomena, the usefulenergy outputthat is, what is available for further changeisalways less than the energy input, with the difference usuallyappearing as heat. One goal in the design of such devices is to makethem as efficient as possiblethat is, to maximize the useful outputfor a given input and to minimize wasted heat energy.

Radiant energy from the Sun is the ultimate source of most ofthe energy we use. It becomes available to us in several ways: Theenergy of sunlight is captured directly in plants, which then may beeaten; it also heats the air, land, and water, causing wind and rain.For much of history, burning wood was the most common source ofintense energy for cooking, for heating dwellings, and for runningmachines. Most of the energy used today is derived from burningfossil fuels, which contain stored solar energy that plants collectedover millions of years. A new source of energy is the fission of thenuclei of heavy elements, whichcompared to the burning of fossilfuelsreleases an immense quantity of energy in relation to themass of material used. In nuclear reactors, the energy generatedis used mostly to heat water into steam, which drives electricgenerators.

Humans use energy for technological processes: transporting,manufacturing, communicating, and getting raw materials, thenworking them and recycling them. Students need to appreciate thatdifferent sources of energy and ways of using them involve differ-ent costs, implications, and risks. Some resources will continue tobe available indefinitely; some can be made self-renewing, but onlyat a limited rate. Fuels like coal, oil, natural gas, and uranium willbecome more difficult to obtain as the most readily availablesources become depleted. New technology may make it possibleto use the remaining sources better; the ultimate limitation may beprohibitive cost rather than complete disappearance.

Students should know that depletion of nonrenewable energysources can be slowed both by technical and social means. "Techni-cal means" includes maximizing the advantage realized from agiven input of energy through good design of the transformationdevice, through insulation to restrict heat flow (e.g., insulatinghot-water tanks), or through additional work with the heat as itdissipates. "Social means" includes government, which may re-strict low-priority uses of energy or establish requirements forefficiency (as in automobile engines) or for insulation (as in house

61 66

construction). Individuals also may make energy efficiency aconsideration in their own choice and use of technology (e.g.,turning out lights and driving high-efficiency cars), either toconserve energy as a matter of principle or to reduce their personallong-term expenses. Students need to appreciate that there willalways be trade offs. For example, better insulated houses restrictventilation and thus may increase the indoor accumulation ofpollutants.

The bases for these energy-related concepts should be laid inelementary school science. The following examples illustrateappropriate activities that can be used to formulate assessmentexercises for 4th grade:

1. The student gives examples from his/her own experience ofheat energy and light energy changing a system.

2. The student identifies the source of energy for a familiarsystem (animal, plant, car, electric appliance) and describessome of the energy conversions that take place in each system.

3. The student identifies the energy stored in a stretchedrubberband and in a compressed or stretched spring as poten-tial energy. The student explains that to store potential energyin a rubberband or in a spring he/she must exert a force tostretch the rubberband, or stretch or compress a spring.

4. Given pictures of several situations, some of which depict aforce being exerted or work being done and some of which donot, the student identifies those pictures in which a force isbeing exerted.

5. The student explains, using the words fuel and energy incontext, why a candle goes out when the wax is used up.

6. The student writes a short essay on how his/her life would bedifferent if all the coal and petroleum on Earth were used up.

6 7 62

The Motion of ThingsWith respect to this topic area, the 1996 NAEP Framework

should center on the following concepts:

Reference Frames

Motion

Force and changes in position and motion

Action and reaction

Waves

Vibrations and waves as motion summaries

General wave behavior

Electromagnetic radiation

Effects of wavelength

Interactions of electromagnetic radiation with matter

What students at the three grade levels should understand aboutthese concepts is summarized below.

Reference Frames

Grade 4Everything movesbicycles, cars, trains; the stars, planets, andmoons; the Earth and its surface, and everything on its surface; allliving things and every part of living things. Positions of thingsmay be described; positions may change. Monitoring changes intime yields information about speed.

Grade 8No special point in space can serve as a reference for all othermotion. All motion is relative to whatever point or object wechoose.

6863

Forces and Motion

Grade 4Changes in motionthat is, changes in speed or in directionare due to the effects of forces.

Grade 8Any object maintains a constant speed and direction of motion(including being at rest) unless an unbalanced outside force acts onit. When an unbalanced force does act on an object, the object'smotion changes. Depending on the direction of the force relative tothe direction of motion, the object may change its speed (a fallingapple), or its direction of motion (the Moon in its curved orbit), orboth (a fly ball). The greater the extent of the unbalanced force, themore rapidly a given object's speed or direction of motion changes.In most familiar situations, friction between surfaces brings forcesinto play that complicate the description of motion, although thebasic principles still apply.

Grade 12The more massive an object is, the less rapidly its speed or direc-tion changes in response to any given force. Whenever somethingA exerts a force on something B, B exerts an equal force back on A,but in the direction opposite of the force exerted by A.

Vibrations and Waves

Grade 4Some motions can be described most conveniently in summarydescriptions of the pattern of motion, such as vibrations and waves.Vibrations may set up a traveling disturbance that spreads awayfrom its source.

Grade 8Vibration involves parts of a system moving back and forth inmuch the same place, so the motion can be summarized by howfrequently it is repeated and by how far the parts of a system aredisplaced during the cycle. Vibration may move through a systemas a wave. Wave behavior can be described in terms of speed,

69 64

wavelength, and frequency. Wavelength can help determine how awave interacts with thingshow well it is transmitted, absorbed,reflected, or diffracted.

Grade 12Apparent change in wavelength can provide information aboutrelative motion. The ways in which shock waves of differentwavelengths travel through and reflect from layers of rock areimportant clues to the structure of the Earth's interior.

Light as Waves

Grade 4White light is made up of all different colors of light. Things appearto have different colors because they reflect or scatter light of somecolors more than others.

Grade 8Light behaves in many ways like waveschanging direction,bouncing off surfaces, spreading out, speeding up, slowing down,changing wavelength.

Grade 12The interaction of electromagnetic waves with matter varies greatlywith wavelength. Thus, different but somewhat overlapping electro-magnetic ranges have been given distinctive names: radio waves,microwaves, radiant heat or infrared radiation, visible light, ultra-violet light, X rays, and gamma rays. Materials that allow one rangeof wavelengths to pass through them may completely absorb others.For example, some gases in the atmosphereincluding carbondioxide and water vaporare transparent to much of the incomingsunlight but not to the infrared radiation emitted by the warmedsurface of the Earth. Consequently, heat energy is trapped in theatmosphere. The temperature of the Earth rises until its totalradiation output reaches a state of balance with total radiation inputfrom the Sun.

7065

Life Science

The fundamental goal of life science is to attempt to understandand explain the nature of life. During the 20th century, the thrust ofbiological research has changed its focus from descriptive naturalhistory to experimental science, with most biological investigationsconducted within the theory of evolution. The major concepts to beassessed in the life sciences, with evolution as the central, unifyingtheory, are listed below and developed further in the grade-leveldescriptions that follow:

EvolutionThe diversity of life on EarthGenetic variation within a speciesAdaption and natural selectionChanges in diversity over time

CellsInformation transferEnergy transferCellular communication

OrganismsReproduction, growth, and developmentLife cyclesFunctions and interactions of systems within organisms

EcologyThe interdependence of life: populations, communities,and ecosystems

The three themes in the Science Framework (Patterns of Change,Systems, and Models) can be interwoven with these major conceptsin the life sciences. Because evolution is the major pattern ofchange that occurs in the life sciences, the Patterns of Changetheme can enhance understanding of all of the life science conceptslisted above. Because the Systems theme can pertain to systems atthe cellular, organismal, population, community, and ecosystemlevels, this theme also can enhance understanding of most, if notall, of the life science concepts. Although Models are used in the

71 66

life sciences, this theme receives less emphasis in life science thanin the physical and Earth sciences, particularly at grade 4 andgrade 8.

Students' understanding of life science concepts develops graduallyas the students proceed from grade 4 to grade 8 to grade 12. Adescription of the developmentally appropriate concepts that shouldbe understood at each of these grades follows:

Grade 4Organisms

As some animals grow, they look pretty much the sametheyjust increase in size. As other animals grow, they change fromone form to another form that looks very different. They maychange form several times before they become adults.

Only adults can reproduce, but not all young animals survivelong enough to become adults.

Many activities go on inside the body that cannot be seenwhen something happens in one part of the body, it affectswhat goes on in other parts of the body.

Ecology

Plants make their own food with sunlight, water, and air.

Some animals eat plants; some of these animals are eaten byother animals.

Plants and animals get energy and building materials fromtheir food.

Evolution

There are different kinds of plants and animals on Earthand in the sea.

There are differences among individuals of the same kindof plant or animal.

Children of the same parents are somewhat alike andsomewhat different.

67 72

Grade 8Organisms

Different systems of the body have different functions;however, the functioning of each system affects othersystems.

Interactions among systems are complexthese interactionsmaintain a fairly stable operation of the entire system that canresist disturbance from within or without.

Interaction with other organisms (especially microorganisms)are important to maintain health or cause disease. Avoiding orkilling microorganisms can prevent disease.

Ecology

Plants use energy in sunlight to assemble food molecules fromwater and carbon dioxide.

Plants and animals break down food molecules to obtain foodenergy.

The source of energy and materials for all animals is plants.

The pattern of what eats what in a community can becomplex.

EvolutionThe organisms that survive long enough to reproduce may bedifferent in some ways from others in a population that do notsurvive long enough to reproduce; their offspring may inheritthe anatomical, chemical, and/or behavioral characteristicsthat enabled the parents to survive.

Gradually, over many generations, organisms with thefavorable characteristics may crowd out other organisms inthe population that do not have these characteristics.

Scientists believe that these processes, operating oververy long periods of time, have resulted in the diversity oforganisms that can be seen on Earth today.

Adaptation may be to either the living or nonlivingcomponents of the environment.

73 68

Grade 12

CellsEvery cell contains a recipe for running the cell, coded inDNA molecules; the code mainly specifies how to putproteins together.

During cell reproduction, the information in the DNA codeis passed on to the next generation of cells.

Proteins control most of what goes on within cells and withinthe body.

There are interactions among the cells of an organismmolecules from one cell affect what goes on inside other cells.

In plant cells, energy from sunlight is transformed intochemical energy during photosynthesis; in plant and animalcells, the chemical energy stored in food molecules is releasedduring digestion and produces heat. Some of this releasedenergy is used to build new molecules.

OrganismsSeparate parts of the body can communicate with each otherusing electrical or chemical signals.

Complex interacting systems include feedback that tends toproduce cycles of activities within the body.

In organisms that reproduce sexually, each parent passes onone-half of its DNA information to each of its offspring;therefore, half of the DNA in each cell of an organism camefrom one parent, and half from the other parent.

EcologyInteractions between living and nonliving components affecthow ecosystems function as a whole.

A change in one component of an ecosystem affects othercomponents of an ecosystem. These components in turnreact in a way that will restore the ecosystem to its originalcondition.

7469

Often changes in one component of an ecosystem will haveeffects on the entire system that are difficult to predict.

The size of a population and the rate of growth of a populationare determined largely by the survival rate, the reproductiverate, and the death rate of the organisms in the population.Predictions about changes in the size or rate of growth of apopulation can be described using mathematical models.

EvolutionRecombination and mutation are the raw materials for newtraits upon which natural selection acts.

When the environment changes, different characteristics maybe important for survivaldifferent adaptations are importantfor survival in different environments.

Some descendants are so different from other descendantsthat they can no longer breed with each other.

Particularly at grade 12, students should be able to integrate infor-mation from different concepts within the life, physical, and Earthsciences. They should understand how key concepts apply atdifferent levels of biological organization (molecular, cellular,organism, population, community, ecosystem, and biome), and howthese concepts apply to current societal problems and are signifi-cant to the development of a variety of biotechnologies. Theyshould be able to describe common misconceptions about naturalphenomena and describe how these explanations are contrary tocontemporary scientific explanations. Students also should under-stand the effects of technologies created by humans on the lifecycles of organisms and their effects on communities of plants andanimals, including humans. In addition, students should havedeveloped some familiarity with the historical development ofkey concepts in the life sciences.

7570

Appendix BExamples of Themes

7 6

Examples of Themes by Grade Level

Systems

tudents should understand that systems are artificial construc-tions created by people for certain purposesto gain a betterunderstanding of the natural world or to design an effective

technology.

The construct of a system entails identifying and defining itsboundaries, identifying its component parts, identifying theinterrelations and interconnections among the component parts,and identifying inputs and outputs of the system.

Systems should be embedded in life science learning at the threegrade levels in the following ways:

Grade 4Systems should be approached at the level of organisms. Studentsshould have broad and rich acquaintance with structure/functionrelationships as a precursor to a more thorough knowledge of organsystems by grade 8. Examples of food chains and interdependenciesamong organisms, say, within an aquarium, are precursors tounderstanding complex systems.

Grade 8Students should understand that the organism is made up oforgan systems that have structure/function adaptations andinterconnections among organ systems.

Interdependence of plants and animals in communities should beunderstood by grade 8: plants consumers decomposers.Students should be able to explain specific examples such as purpleloose strife replacing cattails and the effects of the introduction ofrabbits into Australia.

Disease and health should be understood in systems terms; if a partof a system is put out of kilter by disease, for example, the whole isaffected. Taking drugs or smoking by an individual may have animpact on another system (organism); for example, secondarysmoking effects on children of smoking parents or fetal damage

73 77

from drugs. Measles vaccine taken by an individual, or not, affectsthe whole population of a region or further, depending on migrationpatterns. If a specific animal or plant population becomes unhealthy(fish poisoned, raccoons diseased, species of grass infected byvirus), the food chain and, therefore, the rest of the community,are affected.

Grade 12Ecosystems should be understood in their full complexity, includ-ing interrelationships of plants and animals with each other as wellas with the physical components of a system. Students also need torecognize the effects of human activity on ecosystems and thelimitations on human activity imposed by natural systems.

At this level, the cell should be understood both as a system in itselfand as a component of a system.

Patterns of Change

"Patterns of Change" is a particularly valuable theme in the lifesciences because a conceptual understanding of patterns of changecan be developed in the context of several different levels in thehierarchy of biological organization. At the cellular and organismallevel, the primary patterns of change are the growth and develop-ment that occur throughout the life of organisms. At the populationlevel, the primary patterns of change are the changes in populationgrowth over relatively short periods of time and the evolutionarychanges that occur over longer periods of time. At the community!ecosystem level, the primary patterns of change are those thatinvolve the nonliving and living components of ecosystems duringthe process of succession. Patterns of change may be linear, or theymay be cyclical; for example, many of the patterns of change thatoccur within cells are related to homeostasis, in which a changeleads to feedback reactions that result in a return to conditions thatexisted before the change. An understanding of cyclical patterns ofchange can also be developed in the context of ecosystems (nutrientcycles) and in the context of organisms (life cycles).

Regardless of the context in which an understanding of the "pat-terns of change" theme is developed, students should be able torecognize patterns of similarity and difference, to recognize how

7S74

these patterns change over time, and to transfer their understandingof a familiar pattern of change to a new, unfamiliar situation.

To understand the conceptual basis for the "patterns of change"theme, students must begin to develop an understanding of majorideas by the 4th grade, continue to develop their understandingthrough the 8th grade, and integrate their knowledge at the 12thgrade.

Grade 4Understanding Patterns of Change at the Organismal Level

Life cycles (including growth and metamorphosis)

Understanding Patterns of Change at the Population Level

Concept of biotic potential, birth rates, survival rates

Diversity of many types of plants and animals (an importantpreconcept for the understanding of evolution)

Variation within species (focus on humans, dogs, cats)

Understanding Patterns of Change at the Community/Ecosystem Level

Food chains (also important for the "systems" theme)

A more general understanding involves the notion that everythingchanges, sometimes quickly and sometimes slowly, and thatchanges may be too rapid or too slow to observe directly.

Grade 8Understanding Patterns of Change at the Organismal Level

Growth, development, and reproduction of the humanorganism

Homeostasis of body systems

Understanding Patterns of Change at the Population Level

Adaptation and natural selection, including learned andinstinctive behavior

Variation and similarity among many different organisms,including humans

75

79

Understanding Patterns of Change at the Community/Ecosystem Level

Food webs (also part of "systems" theme)

Environmental effects of human activity (also part of"systems" theme)

More general understandings involve the following knowledge:Changes in quantity usually have natural limits, but changes inform in which each form arises from a previous one can produce anunlimited variety. The rate of change may be as interesting as thechange itself. Trends can be steady, accelerated, approach a limitgradually, or have a highest or lowest value.

Grade 12Students should have acquired an understanding of these additionalconcepts and developed the ability to integrate them into the"patterns of change" theme.

Understanding Patterns of Change at the Cellular/Organismal Level

Growth and development of cells, including an understandingof the importance of mitosis and meiosis

Understanding Patterns of Change at the Population Level

Patterns of evolution, mechanisms for evolution, the conse-quences of evolution (such as speciation and diversity throughtime), and evidence for evolution

Understanding Patterns of Change at the Community/Ecosystem Level

Nutrient cycles and the impact of human activity on thosecycles

Succession, both natural and as a result of human disturbance

More general understandings entail knowing that trends, cycles, andrandomness can occur at the same time; randomness may make ithard to see trends or cycles; randomness sometimes may look like atrend or cycle. Feedback in systemsoften an influence that reactsagainst changetends to produce cycles; changes that followprecise rules from one moment to the next may still be unpredict-

76

able in the long run; the environment in which any one changeoccurs is usually changing also, and they affect each other.

Models

Models of objects and events in nature are approaches to under-standing. As such, they have limits, and involve simplifyingassumptions but also possess generalizability and, sometimes,predictive power. Models are composed of groups of interrelatedconcepts selected in an attempt to represent the interrelations ofobjects or events in nature or in the laboratory. Models need not bedeemed correct to be useful but may represent attempts to helptease out the relevant variables in order to build ever more usefulrepresentations.

Models may be conceptual and consist of word descriptions ordrawings. Models can also be mathematical, consisting of equationsor other formal representations. Finally, physical models consist ofphysical objects that possess or represent some characteristics ofthe real thing.

The solar system often is modeled conceptually in the classroom bydescribing the planets as huge balls moving about an even largerSun. A mathematical model of the solar system should includequantitative descriptions of the gravitational forces between theplanets and the Sun as determined by their masses and distancesfrom each other, and might include the shape of a planet's orbit asbeing elliptical. And finally, a physical model of the solar systemmight consist of a series of scale-sized balls placed at appropriatedistances throughout the room or hallway.

Other examples from the Earth and physical sciences includemodels of beaches and continental plates, and stick and ball modelsof molecules. Physical models, such as those of the eye, leaf, andhuman torso, have been used in the life sciences for decades.Experiments with animals serving as models of human beings havebeen used to understand the effects of medical treatments that mightbe useful in preventing human diseases; bacteria have been used tomodel population growth and decay.

Similarly, conceptual models are common in both the biologicaland physical sciences. The simplified treatment of photosynthesis;the stages of meiosis and mitosis, accounting for an electricalcurrent in terms of a "water flow" analogy; and the characterization

778.1

of gas molecules as bouncing balls are examples of commonly usedconceptual models.

Mathematical models such as the gas laws and Newton's laws ofmotion are major components of the physical sciences. Somemathematical models, such as Mendel's laws, have been part ofthe biological sciences for most of this century, whereas the Hardy-Weinberg formulation for describing ecosystems mathematicallyhas become part of introductory biological knowledge morerecently.

Models often serve as prototypes in technology and in that casemay be a full-sized representation of the final product. However,models can be used to test the workings of technology withoutcostly investments in full-scale objects. Small boats and airplanesare tested in tanks and wind tunnels before their full-sized counter-parts are built. In this way, many experiments can be tested inex-pensively to optimize the design.

"Models" can be easily developed as a theme and can be linked tothe immediate experiences of children since they have grown upwith a variety of toys. Children readily understand that most toysare models that look like the real objects, such as cars, airplanes,babies, and animals, but do not possess all the attributes of thoseobjects. Many of these toys are models, sometimes scale models, ofobjects from the natural world. For example, models of dinosaursenable children to develop ideas about what these creatures werelike.

"Models" has been selected as a theme in this assessment becauseof the importance of enabling students to distinguish the idealiza-tions of models from the phenomena themselves. Students need tounderstand that the model of the human eye does not represent allaspects of human eyes as they occur in human organisms. Themodel is a simplification, leaving unrepresented the many impor-tant variations in human eye structure, yet the simplification hasutility in illuminating some features of the eye and enables newquestions about the eye to be generated.

Students need to understand the limitations and simplifying as-sumptions that underlie the varied models used in the naturalsciences. For example, beliefs that models are replicas of "real"objects or events can negate the critical concept of variation that

78

82

many models do not take into account. While generalized models,such as a generalized graph of growth in populations, are useful,they are not to be confused with a graph of the growth of a particu-lar organism or population or of data from a single experiment.

Grade 4At this level, models should be identified by students as representa-tions of objects or events. Students can examine both conceptualand physical models in terms of how they are like and not like theobject or event being represented. Examples could be models ofinsects, seeds, leaves, and other physical objects. These models andothers in the sciences can be linked to children's experiences withscale models of cars, dinosaurs, and doll furniture.

Grade 8Students should have knowledge of both conceptual and physicalmodels and their uses and limitations. For example, when asked toillustrate their understanding of vertebrate structure and functionwith models of skeletons of different vertebrates, students need tobe aware of variations in real skeletons and the generalized natureof the replicas.

Grade 12Mathematical, physical, and conceptual models should be familiarto students beyond grade 8. It is appropriate to assess students'ability to formalize the concept of models and their uses andlimitations in the natural sciences and in technology.

83

79

Appendix CScience Content OutlinesGrades 4, 8, and 12(Excerpts)

(The complete content outlines appear in the Specifications Document)

84

Grade 4Earth Science

A. Solid Earth (lithosphere)

1. Composition of the Earth

Students can classify substances such as soil, sand, or rock.

Students can identify common geographic features oflandscapes.

2. Forces that alter the Earth's surface

Students can describe/explain basic facts about major featuresof the Earth's surface and natural changes in those features(e.g., volcanos, glaciers).

Students can predict the effects of weathering (e.g., rainand wind on sand piles, mud piles, or rock).

Students can describe the relative difference in time ittakes to erode a sand pile, a mud pile, and a rock pile(Conceptual Understanding; Patterns of Change).

Given a picture, topographical map or globe, or worddescription of a major Earth feature (e.g., canyon, mountainrange, Great Lake, cavern, or island), students can identifya geologic force that contributed to producing that feature(Conceptual Understanding; Models).

3. Rocks: their formation, characteristics, and uses

Students can identify common rocks and minerals and canexplain how we can investigate what they are made of andhow they form.

Students can classify rock samples according to color,texture, or other identifying properties (ScientificInvestigation; Nature of Science).

Students can explain that molten rock comes out ofvolcanos, hardens, and becomes part of the landscape(Conceptual Understanding; Patterns of Change).

8583

4. Soil: its changes and uses

Students know some facts about the composition of soil.

Students can separate soil samples into component parts(Scientific Investigation; Nature of Science; Systems).

Students recognize that plants grow in soil and that soilprovides both nutrients and support for the plant.

Students can classify and relate major solid types (e.g.,clay, sand, loam, subsoil) to their ability to support plantgrowth; that is, can identify/predict the major plant typeslikely to grow in those soils (Conceptual Understanding;Nature of Science).

5. Resources from the Earth used by humankind

Students can identify Earth resources used in everyday life.

Students can identify common uses of rock in the humanenvironment (e.g., buildings, roads, walls) (PracticalReasoning; Nature of Technology).

Students can explain/identify that gasoline is processed fromoil, which is pumped from the Earth (Practical Reasoning;Nature of Technology).

Grade 8Life Science

A. Cells and their functions

1. Cells

Students can describe their observations of cells under themicroscope.

Students can demonstrate the use of a microscope toexamine a tissue, plant, or animal and to differentiatebetween plant and animal cells (e.g., students can look atan animal cell and a plant cell and notice that an animalcell is flexible and a plant cell is not) (Scientific Investi-gation; Systems).

Students can look at pond water through a microscope anddescribe outstanding features/activities of the protista theysee (e.g., locomotion, nutrition, excretion) (ScientificInvestigation).

86 84

Students can observe diatoms and try to distinguish asmany features as possible (Scientific Investigation).

Students can explain, in a general way, the advantages of cellularinterdependence versus independence (e.g., multicellular animalsversus single-celled animals).

Students can describe, in general terms, the difference betweenasexual and sexual reproduction in cells and the advantages anddisadvantages of each [The stages of mitosis are not to be tested].

B. Organisms

1. Reproduction, growth, and development

Students can describe growth, development, and reproductionof the human organism.

Students can identify the age ranges at which human beingsgo through common stages of development (e.g., canrecognize their parents; can learn to walk, talk, socialize;can conceive or give birth) (Conceptual Understanding;Patterns of Change).

Students can identify the changes human beings undergoat puberty and can explain their functions (ConceptualUnderstanding; Patterns of Change).

Students can, in simple terms, describe changes in humanembryo development and the effects of environmentalinfluences such as smoking, drugs, disease, and themother's diet on the development of the embryo(Conceptual Understanding; Patterns of Change).

2. Life cyclesStudents can identify some major influences on the human lifecycle (e.g., diet, disease).

Students can discuss the influence of diet and foodavailability on human life cycles worldwide (PracticalReasoning; Patterns of Change).

Students can explain that microorganisms can cause diseaseand can identify some common diseases caused by micro-organisms (e.g., bacteria, viruses, protista) [Differencesbetween viruses and bacteria are not to be tested](Conceptual Understanding).

8587

Students can describe the immune system of animals ashelping the animal fight disease and as controlled, inpart, by the white blood cells in the body (ConceptualUnderstanding).

3. Functions and interactions of systems within organisms

Students are aware that, while different systems of the bodyhave different functions, the functioning of each systemaffects other systems (e.g., students can describe/identifymajor organ systems of the human body, state their majorfunctions, describe some of their interactions).

Students can describe the primary tissues of the bodyand relate the special characteristics of each to its function(e.g., blood, lymph, muscle) (Conceptual Understanding;Systems).

Students can distinguish cells from other structures underthe microscope (e.g., can distinguish between an onion celland a salt crystal) (Scientific Investigation; Systems).

Students can describe how two or more organs of thebody work together to perform a function (e.g., the heartand lungs working together in respiration) (ConceptualUnderstanding; Systems).

Students demonstrate an understanding of the functions andinteractions of organ systems to maintain a stable internalenvironment that can resist disturbances from within orwithout (homeostasis).

Grade 12Physical Science

A. Matter and its transformations

1. Diversity of matter (materials): Classification and types,particulate nature of matter, conservation of matter

Students can distinguish/classify objects, both regular andirregular; pure substances, both elements and compounds; andmixtures, both homogeneous (solutions, liquids, gases) andnonhomogeneous.

Students can describe, measure, and compare substances interms of mass, volume, and density/specific gravity.

88 86

Given a substance of unknown volume or weight andappropriate laboratory equipment, students can determineits specific gravity (Scientific Investigation).

Given an irregular solid and the appropriate laboratoryequipment, students can determine the density of the object(Scientific Investigation).

Students can offer a simplified distinction between weightand density (Conceptual Understanding).

Students can identify evidence that matter is composed of tinyparticles (e.g., atoms, molecules) and that the particles are inmotion (kinetic molecular theory).

Students can define, describe, and contrast physical, chemical,and nuclear changes in molecular terms.

Given various examples of changes in materials, studentscan distinguish among chemical, physical, and nuclearchanges (Conceptual Understanding; Patterns of Change).

Students can discuss the conservation of matter in physical,chemical, and nuclear changes [Can also be tested undertemperature and states of matter, or energy and itstransformations].

2. Temperature and states of matter (physical changes)

Students can discuss/identify the relationship of physicalstates of matter to molecular energy.

Students can associate energy states with molecular motion(Conceptual Understanding).

Students can discuss/identify the energy transfers involvedin the change of phase from solid to liquid to gas and thereverse [Also tested under energy and its transforma-tions] (Conceptual Understanding; Patterns of Change).

Students can discuss/identify the relationship of physicalchanges in substances (i.e., melting, boiling, thermal expan-sion and contraction, compression and expansion underpressure, increase or decrease in density) to changes in thestructural organization of the atoms or molecules of whichthey are composed.

8987

Students can explain how antifreeze solutions work toprevent freezing of the water in car radiators (PracticalReasoning; Systems).

Students can explain why NaC1 (sodium chloride/salt) isadded to ice when making ice cream (Practical Reasoning).

3. Properties and uses of materials: modifying properties,synthesis of materials with new properties

Students can relate the physical properties (e.g., compressibil-ity, structural rigidity) of pure substances in solid, liquid, andgaseous states to the structural organization of particles in thesubstance and their freedom of motion.

Students can explain/identify that the molecules in a crystalare arranged in a regular pattern that gives the crystalrigidity and causes it to take a simple geometric shape(Conceptual Understanding; Models).

Students can examine/utilize useful properties of materials.

Given an unknown liquid and a universal indicatorwith chart, students can determine the pH of the liquid(Scientific Investigation).

Given an unknown marking pen, students can use paperchromatography to identify the brand of marking pen fromamong several others (Scientific Investigation; Nature ofScience).

Students can describe how common artificial materials aremade, recognizing that substances can be designed to havecertain properties, and that the addition of relatively smallamounts of some substances can significantly alter theproperties.

Students can describe how common artificial materials aredisposed of or recycled and can discuss the technological andenvironmental issues involved in these processes.

4. Resource management

Students can discuss scientific, technological, and socialissues involved in resource management.

Students can discuss the issue of worker safety inmanufacturing processes that involve poisonous chemicals(Practical Reasoning; Nature of Technology).

90 88

Appendix DConsensus Committees and Project Staff

91

NAEP ScienceConsensus Project Team

Staff

Ramsay W. Selden, DirectorState Education Assessment

CenterCouncil of Chief State School

Officers

Richard C. ClarkConsensus CoordinatorCouncil of Chief State School

Officers

Senta A. RaizenDirectorNational Center for Improving

Science Education

Julia H. MitchellAssociate Research ScientistAmerican Institutes for

Research

Bonnie L. VerricoAdministrative AssistantCouncil of Chief State School

Officers

Steering CommitteeMembers

William 0. Baker, RetiredAT&T Bell LaboratoriesMurray Hill, New Jersey

Mary Louise BellamyEducation DirectorNational Association of Biology

TeachersReston, Virginia

Frank BettsDirectorCurriculum Technology

CenterAssociation for Supervision

and CurriculumDevelopment

Alexandria, Virginia

William B. CampbellExecutive DirectorNational Industry Council

for Science EducationCollege Park, Maryland

Glenn A. CrosbyProfessorChemistry DepartmentWashington State UniversityPullman, Washington

Gerald DiffordExecutive DirectorColorado Association of

SchoolsEnglewood, Colorado

Janice EarleDirectorCenter on Educational EquityNational Association of

State Boards of EducationAlexandria, Virginia

John FowlerDirectorTriangle Coalition for

Science and TechnologyCollege Park, Maryland

912

Johnnie HamiltonPrincipalFranklin Intermediate SchoolChantilly, Virginia

Elam Hertz ler, RetiredSecretary's Commission on

Achieving Necessary SkillsWashington, D.C.

Ann Kahn, Past PresidentParent Teachers AssociationFairfax, Virginia

Douglas LappDirectorNational Science Resources

CenterSmithsonian InstitutionWashington, D.C.

John LaymanDirectorScience Teaching CenterUniversity of MarylandCollege Park, Maryland

Harold PrattExecutive DirectorScience and Technology

ManagementJefferson County SchoolsGolden, Colorado

Judith Torney-PurtaDepartment of Human

DevelopmentUniversity of MarylandCollege Park, Maryland

Douglas ReynoldsChiefBureau of Science EducationState Department of EducationAlbany, New York

93 92

Bella RosenbergAssistant to the PresidentAmerican Federation of

TeachersWashington, D.C.

Jane SiskBiology TeacherCalloway County High SchoolMurray, Kentucky

Planning CommitteeMembers

Andrew AhlgrenAssociate DirectorProject 2061American Association for

the Advancement ofScience

Washington, D.C.

Bill AldridgeExecutive DirectorNational Science Teachers'

AssociationWashington, D.C.

J. Myron AtkinProfessorSchool of EducationStanford UniversityStanford, California

Joan Boykoff BaronAssessment CoordinatorConnecticut Common Core

of LearningBureau of Evaluation

and Student AssessmentConnecticut Department of

EducationHartford, Connecticut

Audrey ChampagneProfessorSchool of EducationState University of New YorkAlbany, New York

Sally CrissmanLower School Head/

Science TeacherShady Hill SchoolCambridge, Massachusetts

Edmund W. GordonProfessorDepartment of PsychologyYale UniversityNew Haven, Connecticut

Henry HeikkinenDirector, M.A.S.T. CenterUniversity of Northern

ColoradoGreeley, Colorado

George HeinProfessorLesley CollegeCambridge, Massachusetts

Joseph L. PremoScience ConsultantMinneapolis Public SchoolsMinneapolis, Minnesota

Senta A. RaizenDirectorNational Center for

Improving ScienceEducation

Washington, D.C.

James Robinson, RetiredCurriculum and EvaluationBoulder Valley SchoolsBoulder, Colorado

Thomas P. SachseManager, Math and Science

UnitCalifornia Department of

EducationSacramento, California

Gary E. SkaggsTest Development AnalystOffice of Research and

EvaluationFairfax County Public SchoolsFalls Church, Virginia

Science AchievementLevels Panel

Andrew AhlgrenAssociate DirectorProject 2061American Association for

the Advancement ofScience

Washington, D.C.

Audrey ChampagneProfessorSchool of EducationState University of New YorkAlbany, New York

Richard C. ClarkConsensus Coordinator

and Science CoordinatorMinnesota Department of

EducationSt. Paul, Minnesota

Yvonne CurbeamScience Department ChairDunbar High SchoolBaltimore, Maryland

93 94

Joseph PremoScience ConsultantNew Hope, Minnesota

Thomas PrestonScience TeacherFrick International Studies

AcademyPittsburgh, Pennsylvania

Senta RaizenDirector, National Center

for Improving ScienceEducation

Washington, D.C.

Dwight SieggreenScience TeacherCooke Middle SchoolNorthville, Michigan

95

94

William SpoonerChief Consultant, Science

EducationNorth Carolina Department

of Public InstructionRaleigh, North Carolina

Douglas WagnerScience TeacherEmmanuel Lutheran SchoolSt. Charles, Missouri

Sylvia WareDirector, Education

DepartmentAmerican Chemical SocietyWashington, D.C.

9

ISBN 0 16 048903-2

1 1

7801601 1

9 0 0 0 0

489037