Summary Writing as a Critical School Improvement Strategy

by John Collins, EdD

Introduction

For more than 30 years, Collins Education Associates has been providing professional development in writing across the curriculum. While we have refined our techniques and strategies over the years, the foundation of our approach, the Five Types of Writing, briefly described below, has not changed.

Type One is a structured process to capture ideas. Type Two is concerned with writing about the correct content. Type Three focuses on writing about content and attending to some aspects of writing craft. Type Four is peer editing Type Three assignments. Type Five is error-free writing.1

We refer to Type One and Type Two as writing to learn and Type Three, Type Four, and Type Five as learning to write. Until recently, when we worked in schools, we explained the power of the Five Types of Writing and encouraged teachers to use the Types at their discretion to suit the needs of their students. But, based on the examination of the Common Core State Standards (CCSS), we have refined our approach. In light of these standards and the tests that will be used to assess them, we are making three specific recommendations:

1. Give frequent Type Two constructed response quizzes2. Assign Ten Percent Summaries of complex nonfiction text3. Require students to write multi-paragraph argument essays based on source materials.

This document provides an in-depth explanation of our second recommendation, assign ten percent summaries of complex nonfiction text.

With all the strategies available to improve schools both electronic (flipped classroom, etc.) and traditional (cooperative learning, etc.), why emphasize summary writing? Quite simply, our students need to read more to expand their subject specific background knowledge and their academic vocabularies, and the best way to encourage the careful reading of non-fiction texts that will add content and vocabulary knowledge is to ask students to summarize what they have read in writing, not orally. Marilyn Jager Adams describes the decline of reading skills, especially in the upper grades, in her

1 See Appendix A for a one-page description of the Five Types of Writing, and for a complete description, see The Collins Writing Program: Improving Student Performance Through Writing and Thinking Across the Curriculum by John Collins.

5/19/2023 Page 1

Why the Emphasis on Summary Writing

extensively researched article “Advancing Our Students’ Language and Literacy” in the American Educator. This article carefully reviews the literacy achievement data from 1960-2010 and concludes, “The literacy level of our secondary students is languishing because the kids are not reading what they need to be reading” (p.3). This is primarily because school text books have become easier over the last fifty years while newspapers and other texts have not changed or, in the case of scientific magazines, “have increased dramatically from 1930 to 1990” (p.5)! For example, American text books average sentence length has decreased from an average of 20 words in 1962 to 14 today. In light of this information, one might conclude that teachers can lecture about content to help close the information and complexity gap, but then we are faced with another difficult fact: “The richness and complexity of the words used in oral language samples paled in comparison with written texts” (p.5). Students need the experience of reading complex texts because written language is more difficult: The words are more precise, the concepts are more elaborated, and syntax is more complex.

Reading complex, subject related text helps students begin to overcome what has been popularly called the “Matthew Effect” which describes why good readers become better and poor readers become poorer as each year goes by. Our non-reading students are constantly falling behind because their store of background knowledge and academic vocabulary is not growing. Once students get beyond basic decoding, we need to help them expand what E.D. Hirsch (2003) calls students “word and world knowledge” (p. 20) which brings us back to summary writing.

Writing summaries comes in as the second most impactful strategy to improve student achievement according to the meta-analysis by Marzano, Pickering, and Pollock (2001), only topped by identifying similarities and differences. And I would argue that identifying similarities and differences is impossible without the knowledge gained by reading. In another exhaustive meta-analytical study, Graham and Perin (2007) examined the research to determine the most effective strategies to improve writing skills. After explicitly teaching writing skills (e.g. brainstorming, editing) summarization has the most powerful, positive impact (p.16).

The national standards known as the Common Core arose as a result of the growing evidence that our students are not able to comprehend texts that are necessary for success in careers or college. The Common Core has created ten Anchor Standards for reading. The second of the ten states that students need to “Determine central ideas or themes of a text and analyze their development; summarize the key supporting details and ideas” (CCSS, 2010, p.10). The Anchor Standards for writing almost (with the possible exception of narrative writing) all depend upon the students’ ability to first understand a text before they analyze it or use evidence from text(s) to support their own ideas.

After a careful analysis of the CCSS, a group dedicated to helping schools meet the standards, Achieve the Core, determined that schools need to make three shifts in their practice. Schools must encourage students to:

1. Build knowledge through content-rich nonfiction2. Read, write, and speak grounded in evidence from text3. Regularly practice with complex text and its academic language

What I propose here, frequent summary writing of content-related nonfiction articles, will meet the requirements set by these three shifts. The next section describes exactly what the proposed summaries should look like and how we can teach students to write them.

5/19/2023 Page 2

The previous section made the case for summary writing in general. This section will argue for a specialized form of summary writing, the Ten Percent Summary. The Ten Percent Summary requires students to write a summary that meets the following five characteristics:

Are approximately 10 percent of the length of the original article Summarize content-related nonfiction articles that are from 700 to 3,000 words long, the typical

length of most magazine or newspaper articles. (Articles shorter than 700 words are hard to summarize using the Ten Percent formula, and they do not build the reading stamina that students need.)

Begin with a sentence that includes the source of the article, the title, the author or authors, a verb that describes the author’s purpose (e.g., describes, argues, explains) and a statement of the topic, all correctly punctuated. I use the acronym S "T" A r t (source, title, author, right verb, and topic) as a reminder to students. (The quotes around the T and the underlining of the S indicate to students to put the title of the article in quotes and underline the source. The uppercase S T A indicate that the title, source, and authors should have initial capitalization.

Include the main ideas in the order the article presents them, without introducing personal opinion

Be written in the summarizer’s own words but may include a few short quotes, especially if these contain original expressions that convey the tone of the article

The advantages of the Ten Percent Summary are many. Here are a few:

1. Can be used in all subjects: One of the major advantages of the Ten Percent Summary is that it provides a consistent approach for all teachers at all grade levels in all subjects. Since we are discussing school improvement, not individual teacher improvement, we should have an approach all teachers can use. But if you were to poll twenty teachers and ask them to define summary writing you would hear twenty different approaches. (For example, Rich Wormeli’s book Summarization in Any Subject gives over 50 techniques.) The structure of the Ten Percent Summary helps teachers and students know what to expect, and once it is taught it can be an easy-to implement assignment in all subjects not just ELA or reading classes.

2. Builds writing skills: There are a great many ways to teach summarization, and summaries can take many forms; for example two column notes, web graphic organizers, and wheel-and-spoke graphic organizers. But what I am advocating here is a consistently formatted summary that requires students to write in sentence and paragraph form. While this type of writing will never be construed as creative nor does it help develop an original voice, it does give students practice putting ideas into their own words, creating transitions between these ideas, and expressing these ideas succinctly. Additionally, if you add an aspect of conventions as criteria, students can practice the conventions of writing—without the burden of having to create their own content.

3. Provides reasonable test prep: Most educators hate teaching to the test, but the ACT, SAT, and most middle and high school state tests have reading sections of about 500 to 1,500 words with either multiple choice or open response questions that require students to identify—you guessed it—the central idea. Students who have read 50, 100, or 150 articles before taking

5/19/2023 Page 3

Why the Ten Percent Summary

these high stakes tests have four advantages: lots of background knowledge, lots of practice reading nonfiction, lots of practice finding main ideas, and lots of exposure to academic vocabulary.

4. Provides opportunity to use technical/academic vocabulary in context: In the section below, I recommend focusing on a few criteria, and one of the criteria is using academic vocabulary from the article in the summary. For example, before distributing an article to summarize, select and circle 4 to 8 content or general academic vocabulary words that you feel are critical. Review these words with the students and require that they use a certain number of the words in the summary. In the Collins Writing Program this requirement would become a focus correction area (FCA) for the summary; for example, correctly use 4 of the 7 circled vocabulary words in your summary. Circle these words and number them so I can find them and give you credit for your work.

5. Develops research skills: For a student to be able to write an effective research paper, they must be able to attribute sources (the S "T" A r t topic sentence), summarize, and paraphrase. This assignment provides practice in all these skills.

6. Easy to evaluate: If Ten Percent Summary writing is to be one of the key areas of school improvement and if every teacher is to assign one summary per month in every subject and if the articles to be summarized were to average 1000 words, a typical student in grades 6-12 who has five subjects would do 45 summaries (9 months x 5 subjects = 45) per year. That typical student will have read 45,000 nonfiction, content related articles and written 4,500 words in sentence and paragraph format with proper research citations - the quality and quantity of this work would make students career and college ready. The down side is that teachers of those students will have to evaluate all their work. For example, if a teacher has 120 students and does one summary per month, that teacher will have to evaluate 1,080 summaries; therefore, the evaluation system must be efficient and easy to use.

The evaluation system I recommend is called focus correcting. When using this system, a teacher selects three areas and only grades those areas. The teacher assigns points to each element—such as the opening S "T" A r t topic sentence, 3-4 central ideas in their own words, and correctly used vocabulary words—and the student then lists each element with its corresponding points at the top of the paper. This technique benefits everyone: Students know up front how the teacher will evaluate their writing and will typically try harder to master these areas, and the teacher doesn’t need to correct every single error on every student’s summary. For example, I might assign 20 points to the topic sentence (S "T" A r t); 60 points to the 3-4 main ideas written in the student’s own words; and 20 points to 4 vocabulary words that are used correctly and underlined. Or the teacher might assign 20 points to the topic sentence, 50 points to the 3-4 main ideas, and 30 points to 2 well-selected and correctly punctuated quotations from the text. I can quickly assess all these areas. Although I might wish to give extensive feedback on the summary, time doesn’t permit it, so I do not comment on all the errors.

With this system, after the first few papers that usually require more time to establish a focus and pace, I can evaluate each paper in less than a minute. One time saving practice I use is to grade a few volunteer’s papers in each class using a document camera. If students volunteer, I grade their papers using focus correcting with the promise that they can redo the paper for a

5/19/2023 Page 4

higher grade when they see my evaluation. By the time I have done a number of these volunteer papers, I have found my “grading groove” and my students have seen the process. As the year goes by, and if other teachers are using the same system, the grading is valid and reliable and no one teacher is overworked with too many papers to grade. Of course the hope is with this amount of practice, the students will become efficient and accurate summary writers, and as we know, it is a lot quicker grading papers with 100’s than with lower grades.

Works Cited

Adams, Marilyn Jager. (2010-2011, Winter). "Advancing our students' language and literacy." American Educator, 3-53.

Graham, S., Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools. Washington, DC: Alliance for Excellent Education.

Hirsch, Jr, E.D. (2003). "Reading comprehension requires knowledge—of words and the world." American Educator, 10-29.

Marzano, R., Pickering, D., & Pollock, J. (2001). Classroom instruction that works. Alexandria, VA: ASCD.

National Governors Association Center for Best Practices (NGA Center) and Council of Chief State School Officers (CCSSO). 2010a. Common Core State Standards for English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects. Washington, DC:NGA Center and CCSSO.

5/19/2023 Page 5

Appendices

Appendix A Collins Writing Program’s Five Types of Writing ------------------------------------------- 7

Appendix B General Academic Vocabulary for Summary Writing ------------------------------------- 8

Appendix C Annotated List of Resources Available on the Web by Subject Area and Grade ---- 10

Appendix D Tips on Reading Informational Text for Central Ideas ------------------------------------ 15

Appendix E Creating the Ten Percent Summary ----------------------------------------------------------- 16

Appendix F Template to Structure the Ten Percent Summary ----------------------------------------- 17

Appendix G Template to Structure the Ten Percent Summary - Spanish version ------------------ 18

Appendix H Template to Structure the Ten Percent Summary - Two Page version --------------- 19

Appendix I Unedited Sample Student Summary: grade __ --------------------------------------------- 21

Appendix J Edited Sample Student Summary: grade __ ------------------------------------------------- 22

Appendix K Student's Guide to Ten Percent Summary --------------------------------------------------- 23

Appendix L Article: "Summarize to Get the Gist" (Educational Leadership) ------------------------ 25

Appendix M Articles to Practice Summarization

1. "How Tests Make Us Smarter" (822 words) ------------------------------------------------- 29

2. "Why Flunking Exams is Actually a Good Thing" (2,884 words) ------------------------ 31

3. "Studying for the Test by Taking It" (857 words) ------------------------------------------- 37

4. "Studying: You're Doing it Wrong" (1,289 words) ----------------------------------------- 39

5/19/2023 Page 6

Appendix A

5/19/2023 Page 7

Appendix B

5/19/2023 Page 8

General Academic Vocabulary for Summary Writing

analyze to consider more than one opinion on a topic or question before drawing your own conclusions or to consider a topic or question carefully by breaking your response to it into parts

appropriate something that is acceptable for a particular situationargue to state something is true and give reasons and evidence why you think it is

trueargument a way to convince another person of your opinion by using evidence from a

text or another relevant sourceaudience the person or group of people, in addition to your classmates and teacher,

who may read your writingcaption the words printed under a picture or cartoon that explain what it is about

central idea the chief point or main idea (expressed in a sentence or two, not just a few words) that an author is making about a topic

claim an idea you believe to be true that you can support with evidenceclarify to make something easier to understand by explaining it in more detail

clear/clearly easy to understand; or, to say something in a way that is obvious and not easily mistaken

coherent clear, logical; well planned so that all parts go together wellcomplex/complexity something with many different and related parts that can be difficult to

understandcomplex ideas ideas with many different or related parts and, therefore, often difficult to

understandcomponent a part of a whole

conclude an endingconcluding statement a way to wrap up a piece of writing; it comes at the end of a composition

and reminds the reader what the central ideas of the writing arecontext the information surrounding a word or phrase that determines exactly how

it was meantcredible believable, trustworthy; based on data or experience

demonstrate to make clear by explaining or showing with wordse.g. a Latin abbreviation that means “for example”

establish to set something up, start; to build in a secure, solid wayevaluate to think about something carefully in order to make a good judgmentevidence information that you give to prove a point

explain to give details or describe somethingfeasible something that is possible or can be done

gist the general meaning of a piece of writing, usually expressed in a sentence or two (similar to central idea)

hypothesis an educated guessidentify to say what something is

illustration labeled drawings or pictures that help explain ideasinterpret to describe something's meaning

metaphor a figure of speech suggesting a similarity between two unrelated things; an imaginative comparison without using like or as

paraphrase to express what is said or written using your own words and in about the

5/19/2023 Page 9

same number of words as the original speech or piece of writingperspective a point of view or opinion on something

plagiarize to use or copy another person’s idea or work and present it as your ownquotation a phrase, line, or paragraph put in quotation marks that you include, word-

for-word, in your own writing while attributing the authorquote to include the exact words of another writer or speaker in your own writing

to support a particular point or claim while attributing the authorrelevant closely relates to and supports the point you are making

simile a comparison using like or assource the book, magazine, or internet reference in which the story, argument, or

piece of research you are summarizing, paraphrasing, or quoting first appeared

structure the way in which something is organizedsubtitle the second title, usually under the main title, that is often longer and

explains the titlesummary a brief statement of the central ideas of a speech, event, process, or piece

of writingten percent summary 10% of a written work that sums up the whole work in a specific format

text any written materialtext structure how a text is organized and if it has any graphics, headings, or illustrations

that make it easier to understandtextual evidence support for an idea taken from a text, not made up

topic the subject being written abouttopic sentence the sentence in a paragraph that contains the central idea of that paragraph

well-structured organized; logical

5/19/2023 Page 10

Appendix CAnnotated List of Resources Available on the Web by Grade and Subject Area

Language Arts

Poetry Foundation: This website, created by Poetry magazine publisher Poetry Foundation, features articles about poetry in the form of interviews and essays. The website has a search box so educators can search for specific topics, such as elliptical poetry or certain poets. The website offers several RSS feeds, such as Poem of the Day and Glossary Term of the Day, which educators may find useful for sparking ideas for future lessons. Link: http://www.poetryfoundation.org/features/articles Word count: Usually 2,000–3,000 words per articleHigh School

Ploughshares: This website, maintained by Emerson College, allows visitors to sample pieces of the Ploughshares magazine, which publishes fiction, poetry, and essays. Each issue has a guest editor; past editors have included Seamus Heaney and Tobias Wolff. The website also hosts a blog, which offers more articles on such literary themes as publisher and book reviews, providing more material for educators to use. Link: http://www.pshares.org/index.cfmWord count: Usually 1,500–2,000 words per literary pieceHigh School

Cicada: This website, a companion to Cicada magazine (part of the Cricket Magazine Group), has content for children aged 14 and older. It publishes short fiction, poetry, essays, comic reviews, and interviews with writers. Site visitors can access excerpts from the current magazine issue, along with excerpts from a few past issues. Cicada also features short interviews with artists that provide content for the magazine; educators can also ask students to summarize these interviews.Link: http://www.cicadamag.com/Word count: Usually 500–800 words per literary pieceMiddle, High School

Science

HighlightsKids: Produced by Highlights for Children, this website contains content for children aged 2 through 12. The articles are mostly on biology and nature, with some additional topics, such as space exploration. The website has a limited search feature, but it does have recordings of the articles, which may be helpful for educators who are assigning the articles to younger students.Link: http://www.highlightskids.com/articles Word count: Usually 350–500 words per articlePrimary, Upper Elementary School

Science News for Students: This publication, offered by the Society for Science and the Public, is designed for students aged 9 through 14 and holds an easily navigable database of free articles divided into topics such as atoms and forces, tech and math, and humans and health. The site also has a search

5/19/2023 Page 11

box that allows educators to search for articles on specific topics. Most articles also contain a Power Words section at the end, which is a glossary of keywords used in the article.Link: https://student.societyforscience.org/sciencenews-students Word count: Usually 500–2,000 words per articleUpper Elementary, Middle School

Ranger Rick: Hosted by the National Wildlife Federation, Ranger Rick contains articles designed for students aged 7 through 12, accessed on the homepage by clicking on the links in the Ranger Rick magazine box in the middle of the page. Educators can also use the search box at the top of the site’s web pages to find articles on particular topics. This website is limited to articles pertaining to animals and nature.Link: http://www.nwf.org/Kids/Ranger-Rick.aspx Word count: Usually 400–1,000 words per articleUpper Elementary, Middle School

Scientific American: Maintained by Scientific American Inc., this website publishes articles on such topics as fuel cells, dinosaurs, neurology, and quantum physics. Educators can subscribe to a RSS feed for each of these topics, allowing them to see when relevant articles become available. The site also offers a search box on its pages.Link: http://www.scientificamerican.com/ Word count: Usually 600–900 words per articleMiddle, High School

ChemMatters: Produced by the American Chemical Society, this website contains articles aimed at high school students from ChemMatters magazine . From the homepage, educators can click on links to the free articles in the current issue, as well as view selected articles from past issues through the Past Issues link. Topics, focused on solely chemistry, include metals and nonmetals, food, and bonding. The site even provides Spanish translation for selected articles.Link: http://www.acs.org/content/acs/en/education/resources/highschool/chemmatters.html Word count: Usually 1,000–2,000 words per articleHigh School

The National High School Journal of Science: This website is offered by and contains articles from The National High School Journal of Science, which is a free, peer-reviewed journal written by and for high school students. The journal also has a Scientist Advisory Board of professionals from across the health sciences and engineering fields who help review the articles before publication. Educators can browse the most recent edition on the homepage or browse by subject using the Subjects tab. Subjects covered include physics, biology, and environment.Link: http://nhsjs.com/ Word count: Usually 2,000–2,500 words per articleHigh School

Discover: Owned by Kalmbach Publishing, this site is companion to Discover magazine. Articles are categorized by topics, such as health and medicine, space and physics, and environment, and are further divided into subtopics, such as climate change and pollution. A select number of articles from the current issue are available for free, as are some articles in the issue archive, which goes back to 1987. Educators can also sign up for RSS feeds to receive new articles.Link: http://discovermagazine.com/

5/19/2023 Page 12

Word count: Usually 600–1,500 words per articleHigh School

Social Studies

KidsPost: This website is owned by The Washington Post and contains articles specifically for students in grades 2 through 7. Its search box and its Latest KidsPost Stories section help educators navigate its collection of articles on current events (both national and international), politics, sports, and art, among others. The Feature Stories section offers longer articles for students at higher reading levels, while the rest of the website has shorter articles for those at the primary school level.Link: http://www.washingtonpost.com/lifestyle/kidspost/ Word count: Usually 300–700 words per articlePrimary, Upper Elementary, Middle School

Discovery News—History: Published by Discovery Channel’s owner, Discovery Communications, the History section of Discovery News offers articles on a range of historical topics, including archeology, US history, and religion. Each article features a More Like This section at the top of the page, enabling educators to find more articles on the same themes. Educators can also explore the related tags at the bottom of the article to find more on a particular topic. Link: http://news.discovery.com/history Word count: Usually 700–1,000 words per articleMiddle, High School

Science and Social Studies

National Geographic Kids: This website, run by the National Geographic Society, contains an online database of articles that allows educators to limit their search by category, including geography, history, mythology, nature, science, and space, and by content type, such as articles and videos. At the end of each article, Dive Deeper section links to articles on a similar topic, helping educators select more than one relevant article.Link: http://kids.nationalgeographic.com/explore.html# Word count: Usually 400–500 words per articlePrimary, Upper Elementary, Middle School

TIME for Kids: Maintained by TIME Inc., TIME for Kids contains an archive of full-text articles sorted by topic, including world, science, health, and sports. In the Polls section, educators can read the findings from online survey questions, such as “Should national parks allow the use of cell phones?” which could provide source material for argument writing. The website also has a search box for educators looking for a specific topic.Link: http://www.timeforkids.com/news Word count: Usually 500–600 words per articlePrimary, Upper Elementary School

DOGOnews: This website, hosted by DOGO Media, publishes articles on social studies and science for students aged 9 to 13. It tags each article with a suggested grade level or span and provides suggested lesson plans based on Common Core Standards. The homepage features both a search box for keyword

5/19/2023 Page 13

searches and a Most Popular Articles section of articles deemed popular based on the number of comments written about the article.Link: http://www.dogonews.com/ Word count: Usually 400–600 words per articlePrimary, Upper Elementary, Middle School

Newsweek: The companion site of Newsweek magazine contains articles about current events occurring both nationally and internationally. Articles are categorized by topic, such as US, world, business, and culture. The site also contains science material in the Tech and Science section. Educators can utilize the search box at the top of the page and an archive of past issues in the page footer to further research articles. Articles are tagged with keywords to ease a search for more articles on the topic.Link: http://www.newsweek.com/us Word count: Usually 500–1,000 words per articleHigh School

Math

Plus: Online magazine Plus publishes articles about the practical applications of mathematics across a span of topics, such as art, sports, and cosmology. The search feature is limited, and the articles are mainly organized in chronological order. However, tying applied mathematics to topics such as the movie Jurassic Park and space travel should help hold students’ interest.Link: http://plus.maths.org/content/Article Word count: Usually 600–1,000 words per articleHigh School

The New York Times—Mathematics: This section of The New York Times currently features over 1,000 articles about mathematics and the instruction of it in school systems. NYT’s tagging system includes such subjects as education, Nobel Prizes, and National Council of Teachers of Mathematics. A search box in the page header allows for keyword searching as well.Link: http://topics.nytimes.com/top/news/science/topics/mathematics/index.html Word count: Usually 500–1,000 words per articleHigh School

Biographies of Women Mathematicians: Hosted and maintained by the Agnes Scott College, this website is a collection of biographies of women mathematicians spanning time and the globe. Although the resource format is limited to biographies, educators may find it to be a good resource for introducing role models to their classes. The website’s material is sorted by an alphabetical index of the women’s names, a chronological index, and a geographical index of the women’s birth locations.Link: http://www.agnesscott.edu/lriddle/women/women.htm Word count: Usually 600–800 words per articleMiddle, High School

Math and Science

NOVA scienceNOW: A companion site to the PBS television series NOVA, this website contains a variety of videos and articles on several topics, such as space and flight, body and brain, physics and math, and tech and engineering. After selecting a topic, an educator can narrow results to just articles. The search

5/19/2023 Page 14

results provide a one-sentence summary of the article, which will help educators select relevant articles quicker.Link: http://www.pbs.org/wgbh/nova/sciencenow/ Word count: Usually 1,500–2,500 words per articleMiddle, High School

Scholastic News: This website, a companion to Scholastic News magazines, contains an archive of news articles on a variety of topics, including uses of mathematics and scientific discoveries. For example, “A Deeper Mystery at Stonehenge” describes the ancient site in England, while “Movie Matchmaker” explores how Netflix uses algorithms. However, the lack of a search box means educators must use an article’s one-sentence description to find the content they need or use a separate search engine, such as Google.Link: http://magazines.scholastic.com/ Word count: Usually 500–600 words per articlePrimary, Upper Elementary School

General

Newsela: This website offers an archive of more than 500 non-fiction articles organized by category and reading standard, something for every teacher's subject area. Newsela has licensing agreements with a variety of the nation's most trusted media sources like the Associated Press and Scientific American. The articles are always current and each is available at five different reading levels which allows all students to read the same article at a reading level that is appropriate. Students can take quizzes and view their progress related to critical thinking and close reading—questions are Common Core aligned. Newsela offers a Newsela Pro account (fee based option) that is useful to administrators because it includes the ability to view student and class-wide dashboards, track student, class, and school progress against the Common Core standards, and the ability to sort and filter student performance data and print reports.Link: https://newsela.com/about/Word Count: Usually 500-1000 words per articleMiddle, High School

ReadWorks: This website provides research-based units, lessons, and authentic, leveled non-fiction and literary passages to educators online. ReadWorks curriculum is aligned to the Common Core State Standards and the standards of all 50 states. The information is free and can be shared broadly. The educator can create an account by entering an email and password.Link: https://www.readworks.org/Word Count: Usually 400-900 words per articleElementary, Middle, High School

ProCon.org: ProCon.org is a non-profit public charity founded in 2004. The charity's purpose is to provide resources for critical thinking without bias. Controversial issues are highlighted in a balanced, transparent pro-con format that allows the reader to better understand important social issues and to make more informed decisions about them. Date and source information for all research is cited in multiple style protocols and the available resources are for readers ranging from elementary to university levels.Link: http://www.procon.org

5/19/2023 Page 15

Word Count: VariesElementary, Middle, High, University

5/19/2023 Page 16

Appendix DTips on Reading Informational Text for Central Ideas: The Way You Usually

Read May Be Slowing You Down and Confusing You

When most of us have something to read for a school assignment or for pleasure, we just start reading. This habit is great, especially for works of fiction. But when we have to read informational text, especially nonfiction, about a difficult subject or a subject that is new to us, it is best to break some of our old habits and try some new practices. This is especially true if there is time pressure. The practices listed below are hard to do at first but will make us faster, smarter readers if we stick with them.

1. Read the full title and look for the source (where the article was published) and the author. Do you know this magazine or newspaper? Is it a reliable source? Do you know the author? Is he or she respected? Does the author have a title or degree that would cause you to respect them?

2. Read the subtitle if the article has one (newspaper stories usually do not, magazines usually do). The subtitle can be the single best summary of the article.

3. Look for subheadings in the article. Sometimes you can turn the subheading into a sentence to create a great central idea statement.

4. Look at any graphics and the captions under the graphics which explain what the information in the graphics means.

5. Read the first 10% or 20%. The introduction will usually give you a preview of what the article will be about, but be careful. Sometimes the author will use an attention-getting beginning, and you’ll have to read past it to find what the article is really about.

6. Read the first sentence in each paragraph of the body of the article. The body is the middle 60 to 80 percent. These first sentences usually, but not always, tell you what each paragraph is about—a great way to find central ideas.

7. Read the last 10% of the article carefully. Most writers sum up their main points at the end.

The above practices are not the way most of us read. Reading a novel this way would ruin it; but for informational articles, these practices are the best way to begin and will help you quickly understand the central ideas without getting confused by the details. When you are done, you may decide to go back and read sections of the body of the article more carefully, or you may decide that you understand the article—especially the central ideas—without reading every word.

5/19/2023 Page 17

Appendix ECreating the Ten Percent Summary

Before Students Read the Article

1. Select a 700 – 2,500 word nonfiction article related to your curriculum.

2. Select 5 – 10 key vocabulary words from the article and record them on the Vocabulary Focus

Sheet.

3. Have students complete the middle column of the Vocabulary Focus Sheet as a Type One

assignment. If time permits, have them share and revise answers. While students are

completing the middle column, use this time to determine your students' prior knowledge of the

subject; knowledge of a subject's vocabulary is the best single indicator of knowledge of the

subject.

4. Give students the correct meaning of the words and have them record the meaning in the last

column of the Vocabulary Focus Sheet.

5. As a scaffolding activity, consider “boxing” the article to show sections containing each central

idea.

6. Select the third FCA for a Type Three Ten Percent Summary. Two good alternatives include:

1. Use a specified number of vocabulary words from the Focus Sheet.

2. Correct use of specific writing convention (no comma errors, correct capitals, etc.).

Students Read Article

1. Have students read and mark up the article.

2. Have students do the Ten Percent Summary Focus Sheet.

After Students Read Article

1. Have students write the Ten Percent Summary to be evaluated based on the following FCAs:

Topic sentence with S “T” A r t

2 – 4 central ideas in order

To be determined by teacher

2. Require that students edit by reading summary out loud to themselves.

3. Grade students based on FCAs and return.

4. End with a Type One assignment: "In three to four lines, explain why you received your

5/19/2023 Page 18

grade."

5/19/2023 Page 19

5/19/2023 Page 20

Appendix FTemplate to Structure the Ten Percent Summary

After you have read and highlighted the article, this focus sheet may help you get started.

STEP 1: In your own words, briefly list the two (2) to four (4) central ideas in the order they appear in the article. Then write a complete sentence describing each central idea. Remember, central ideas should not include your personal opinions about the article or the topic.

Central Idea One: __________________________________________________________________________

Sentence: ____________________________________________________________________________

_____________________________________________________________________________________

Central Idea Two: ___________________________________________________________________________

Sentence: _____________________________________________________________________________

______________________________________________________________________________________

Central Idea Three: __________________________________________________________________________

Sentence: _____________________________________________________________________________

_____________________________________________________________________________________

Central Idea Four: ___________________________________________________________________________

Sentence: _____________________________________________________________________________

______________________________________________________________________________________

STEP 2: Create a topic sentence with S “T” A r t

In ________________________________________________________________________________________Source: magazine, newspaper (If your paper is handwritten, underline the source. If typed, italicize.)

article ____________________________________________________________________________________ ,Title of the article in quotes; capitalize all “big” words, over five letters long, and all nouns and verbs

______________________________________Author: In general, you can use just the last name of the author(s). If there is more than one, list all authors. If no author is named, write “the author(s)”.

_________________________________________right verb, for example: argue(s), explain(s), propose(s), discuss(es), any verb describing the purpose of the reading.Avoid: says, tells, or writes.

topic: In a few words, describe what the article is about. If possible, do not include exact words from the title in the description of the topic.

__________________________________________________________________________________________

__________________________________________________________________________________________.

5/19/2023 Page 21

Appendix GTemplate to Structure the Ten Percent Summary - Spanish Version

Después de leer y subrayar el artículo, esta hoja de enfoque puede ayudarte a empezar

Paso 1: En tus propias palabras, brevemente enumera las dos (2) o cuatro (4) ideas centrales en el orden en que aparecen en el artículo. Luego escribe una oración completa describiendo cada idea central. Recuerda, las ideas centrales no deben incluir tu opinión personal acerca del artículo o el tema.Idea Central Uno: ______________________________________________________________________

Oración: _______________________________________________________________________

_____________________________________________________________________________________

Idea Central Dos: _______________________________________________________________________

Oración: _______________________________________________________________________

_____________________________________________________________________________________

Idea Central Tres: ______________________________________________________________________

Oración: _______________________________________________________________________

_____________________________________________________________________________________

Idea Central Cuatro: ____________________________________________________________________

Oración: _______________________________________________________________________

_____________________________________________________________________________________

Paso 2: Crea una oración central con F “T” A v t

En __________________________________________________________________________________Fuente: revista, periódico (si tu papel está escrito a mano, subraya la fuente. Si está escrito a máquina, escribe en letras itálicas.)

Artículo ______________________________________________________________________________Titulo del articulo debe estar entre comillas; todas las primeras letras de palabras “grandes” en mayúscula (si tienen más de cinco letras), y todos los nombres y verbos.

__________________________________________ ______________________________Autor: en general, puedes usar solamente el verbo correcto, por ejemploApellido del autor(es). Si hay más de un autor, argumenta, explica, discute. Cualquier haz una lista de todos ellos. Si no hay autores, verbo describiendo el propósito deescribe “el (los) autor (es)”. la lectura. Evita: dice, cuenta o escribe.

_____________________________________________________________________________________Tema: En pocas palabras, describe sobre que trata el artículo. Si es posible, no incluya las palabras exactas del artículo en la descripción del tema.

5/19/2023 Page 22

Appendix HTemplate to Structure the Ten Percent Summary - Two Page Version

5/19/2023 Page 23

5/19/2023 Page 24

Appendix IUnedited Sample Student Summary: grade __

Here is a Ten Percent Summary a student wrote of a 900 word article on team mascot names. It has

mistakes that are specific to the Ten Percent Summary assignment. See if you can find and fix them.

1. In the New York Times Upfront article “Insult or honor,” Veronica 2. Majerol talks about the debate over using Native American names or 3. images for team mascots. In this composition, I’ll tell you what the article 4. was about. Majerol describes how approximately 6,500 schools use Indian-5. themed names. Some feel the names honor Native Americans, others feel 6. they insult them. The NCAA “identified 19 universities with names it 7. considered ‘hostile or abusive’ to Native Americans. The governing body in 8. college sports said teams could keep their names if they got permission 9. from tribal leaders; otherwise, they’d have to find a new name or be 10. excluded from hosting championship games.” Another central idea in the 11. article is that it is expensive to change a team name, mascot, or image 12. because fans want that image. For example, fans at the Atlanta Braves 13. game do the “tomahawk chop” to build spirit, but at each game there are 14. Native Americans protesting this action. I think schools should not use 15. Native American names or images even if they get permission because 16. there will also be some people who are offended, and we should not offend 17. anyone if we can help it!

5/19/2023 Page 25

Appendix JEdited Sample Student Summary: grade __

1. In the New York Times Upfront article “Insult or Honor,” Veronica

2. Majerol reports on the debate over using Native American names or

3. images for team mascots. Majerol describes how approximately 6,500

4. schools use Indian-themed names. Some feel the names honor Native

5. Americans, others feel they insult them. It is expensive to change a

6. team name, mascot, or image because fans want that image. For

7. example, fans at the Atlanta Braves game do the “tomahawk chop” to

8. build spirit, but at each game there are Native Americans protesting this

9. action.

# of words 89

5/19/2023 Page 26

Appendix K

Student's Guide to Ten Percent Summary

The Ten Percent Summary from the Collins Writing Program requires you to summarize nonfiction articles in a very specific format. Completing these assignments will increase your knowledge and improve your reading, writing, and research skills. Many teachers ask students to summarize information, but often students do not know how little or how much to write. The Ten Percent Summary solves this problem by letting you know that your summary should have the following three characteristics:

1. The summary begins with a specific type of topic sentence called S “T” A r t; that is a sentence with the source underlined, title in quotes, the authors, the right verb, and the topic.

2. The summary describes two to four of the central ideas in the order they appear in the original article, in your own words, and without your personal opinion.

3. The summary is approximately 10 percent of the words of the original article.

The more of these summaries you do, the more you’ll read, remember, write, and research—all skills critical to your success.

TIPS FOR WRITING THE TEN PERCENT SUMMARY

Length: Before you begin, make sure you know the length of the article, because this will determine the length of your summary. Usually, your teachers will tell you the expected length of the summary. If they do not, it’s easy to determine how much you should write—approximately 10 percent of the words of the original article.

If you use Collins Writing paper, you can estimate how much you should write or how much you’ve written. Collins paper has a heading at the top for focus correction areas (FCAs), points, name and date, and 10 lines to write on. The average student writes 7 to 8 words per line, so if you fill the first page, you’ll have written about 75 words. The second page, with no heading, has 13 lines. If you fill both pages, you’ll have approximately 175 words. Since most articles you will summarize are between 750 and 2,000 words, you can make a good estimate of how much to write and how much detail to add. If you type, remember that you can check your word count in your word-processing software. Often, a double-spaced page has 250 words or more.

Central Ideas: Begin by listing the two to four central ideas in the order they appear in the original source, and then rewrite these ideas into your own sentences. Check your word count; this will give you a sense of how much more detail you need to include. If you want to add more details, you’ll know how much you can add before exceeding the word or space limit. Do not begin with “The first central idea is” or “I’ll tell you the second central idea now.” In fact, do not use I, me, or mine in the summary.

5/19/2023 Page 27

No Opinions or Feelings: Do not include your own opinions or feelings about the article. Although important, your opinions and feelings do not belong in an objective summary; therefore, avoid sentences that start with “I feel that” or “In my opinion.”

S “T” A r t Topic Sentence: Create the S “T” A r t (Source, “Title,” Authors, right verb, topic) topic sentence. The S “T” A r t topic sentence can be up to 15 to 30 words, depending on the length of the title, the name of the source, and your description of the topic. When you describe the topic, try to use different words from those in the title.

→ Example: In an article entitled “My Life as a Nurse” from the Journal of Autobiography by Florence Nightingale, you could write an S “T” A r t topic sentence like this: In the Journal of Autobiography’s article “My Life as a Nurse,” Florence Nightingale describes how she dedicated her life to helping the sick and suffering.

You do not want to write, “… Florence Nightingale describes her life as a nurse.” (It is too much like the title.)

No Plagiarism: To avoid plagiarism, take notes as you read, and write your summary from your notes.

Short Quotes: Consider using a few short quotes of three to five words, especially if they help the reader get a feel for the writer’s voice, style, or tone. Remember, though, that the summary should be your words, with only a few short quotes from the article.

5/19/2023 Page 28

Appendix LArticle: "Summarize to Get the Gist" (Educational Leadership)

June 2012| Volume 69 | Number 9Strong Readers AllSummarize to Get the Gist by John Collins

The 10 percent summary strategy costs little in teacher time,and it prepares students for the Common Core State Standardsin literacy.

As schools prepare for the Common Core State Standards in literacy, they'll be confronted with two challenges: first, helping students comprehend complex texts, and, second, training students to write arguments supported by factual evidence. A teacher's response to these challenges might be to lead class discussions about complex reading or assign regular in-class argument essays. Yet the reality is that after discussing a difficult article with a class of 20 or more students, even the most engaging teacher cannot guarantee that every student will understand it. Meanwhile, one would be hard pressed to find an English teacher who has not inwardly cringed at the thought of having to routinely grade stacks of in-class essays. Some teachers may even neglect to assign such essays, wanting to avoid the work that follows.

I would argue that frequent written summaries of complex texts are a great way to develop students' reading comprehension and argument-writing skills, while minimizing the time the teacher spends correcting. Let's look at the benefits of this strategy as well as how the process works.

The Plusses of Summary Writing

It Requires Active ReadingIf a teacher asks students to not only read but also summarize an article, the students will have to give that article more than a cursory glance. Summary writing hinges on the student's ability to parse main ideas from supporting details and paraphrase those main ideas—two essential skills for reading comprehension and argument writing.

It Builds Background KnowledgeNo longer will students be able to rely on personal experience alone to support their positions. The Common Core State Standards explicitly state that students will be required to write "arguments focused on discipline-specific content." When students routinely summarize content-rich texts, they absorb content that they can subsequently use as evidence in their written arguments.

For example, take Charles C. Mann's Before Columbus: The Americas of 1491 (Atheneum Books, 2009), a nonfiction text recommended by the Common Core for grades 9–10. When students read a section about the importance of maize, having them write a summary of the section will help them attend to the author's point: The development of maize was a great achievement. Later, if called on to write an argument about notable global achievements, the students will most likely recall this point about maize because of the effort they expended in producing their summaries.

5/19/2023 Page 29

It Shows Students How to Craft an ArgumentStudents need to know how to read arguments before they can learn how to write them. Summarizing well-written arguments familiarizes students with rhetorical strategies—such as problem and solution, claim and counterclaim, and reference to authority rather than personal opinion—that are common to persuasive essays. Further, if students routinely summarize articles in a variety of subjects, they'll learn how to construct arguments in each discipline. For example, arguments in science are often based on controlled experiments, whereas arguments in the humanities are often based on expert opinion and appeal to authority.

It's Easy to GradeBecause a summary is shorter and requires less analysis than a fully developed essay, the assignment is faster and simpler to grade, especially with the focus-correcting strategy I describe below.

For all these reasons, summary writing is an essential building block for both reading comprehension and argument writing. Moreover, three meta-analytical studies (Graham & Hebert, 2010; Graham & Perin, 2007; Marzano, Pickering, & Pollock, 2001) all single out summary writing as an effective instructional strategy.

What a Good Summary EntailsAsk 20 teachers how they'd define or assign a summary, and you'll hear 20 different approaches. Summaries can take many forms, from graphic organizers to two-column notes. Teachers who want to experiment with different kinds of summaries might refer to Rick Wormeli's book, Summarizing in Any Subject (ASCD, 2005) or Emily Kissner's Summarizing, Paraphrasing, and Retelling (Heinemann, 2006) for more ideas.

The 10 Percent SummaryMy personal recommendation is a model I call the 10 percent summary, which requires students to write in complete sentences and in paragraph form. Unlike a summary presented in a graphic organizer, this approach pushes students to improve their writing and research skills. For example, students must cite their sources in the proper form; decide which lines (if any) from the article to quote; know how to use relevant vocabulary words in context; and correctly use specific writing conventions—such as proper use of the comma with quotes or an ellipsis.

The 10 percent summary should Be approximately 10 percent of the length of the original article. Summarize content-related nonfiction articles that are from 700 to 3,000 words long, the

typical length of most magazine or newspaper articles. Begin with a sentence that includes the title of the article, the source, the author or authors,

and a summary of the topic, all correctly punctuated. I use the acronym "T"SAT (article title, source, author, and topic) as a reminder to students. (The quotes around the T and the underlining of the S indicate to students to put the title of the article in quotes and underline the source.)

Include the main ideas in the order the article presents them, without introducing personal opinion.

Be written in the summarizer's own words but may include a few short quotes, especially if these contain original expressions that convey the tone of the article.

5/19/2023 Page 30

Although the 10 percent summary could never be construed as creative, it does give students practice putting complex ideas into their own words, creating transitions between ideas, and expressing these ideas succinctly.

How to Evaluate a SummaryTo evaluate a 10 percent summary, I recommend a technique called focus correcting, in which a teacher selects three areas and only grades those areas. The teacher assigns points to each element—such as the opening topic sentence, the main ideas, and correctly used vocabulary words; the student then lists each element with its corresponding points at the top of his or her paper (see "Sample 10 Percent Summary"). This technique benefits everyone: Students know up front how the teacher will evaluate their writing and will typically try harder to master these skills, and the teacher doesn't need to correct every single error on every student's summary.

For example, a teacher might assign 20 points to the topic sentence ("T"SAT); 60 points to the three or four main ideas written in the student's own words; and 20 points to four vocabulary words that the student has used correctly and underlined. Or the teacher might assign 20 points to the topic sentence, 50 points to the three or four main ideas, and 30 points to two well-selected and correctly punctuated quotations from the text.

I can quickly assess the topic sentence, the summary of main ideas, and the usage of the underlined vocabulary words. I don't comment on all the errors, and although I might wish to give extensive feedback on the summary, time doesn't permit it.

A note of caution regarding focus correcting: If I receive a paper that meets the standards set by the focus correction areas but has egregious errors (terrible spelling, sloppy presentation, basic capitalization mistakes, and so on), I'll send it back as an incomplete with a time limit for the student to redo it.1

In Summary …Assigning a 10-point summary gives me an opportunity to determine whether students can understand and put into their own words the main ideas of a complex, content-related article. It also enables me to reinforce basic documentation rules, domain-specific vocabulary, and writing conventions—and do it all quickly and reliably. It may not be the perfect writing assignment, but when I'm trying to teach 100-plus students how to be better readers and writers, I think it comes pretty close.

5/19/2023 Page 31

Sample 10 Percent Summary

Here's a sample 10 percent summary (98 words) of Elizabeth Cady Stanton's Declaration of Sentiments (969 words). Written in 1848 at Seneca Falls, New York, in support of women's rights, the declaration is included in the Common Core State Standards Appendix B as an exemplar of informational text inhistory/social studies at the 11th and higher grade levels.

Title: Women's MovementIn the Declaration of Sentiments from An American Primer, the members of the Seneca Falls Conference created a document that lists 15 women's grievances in 1848. Modelled after the Declaration of Independence and keeping much of the original language, the declaration gives examples of "a history of repeated injuries and usurpations" including unfair divorce, marriage, and property laws with no franchise to change them. Women, the document asserts, are denied career and educational opportunities and must live under a different code of morals. The declaration concludes with the demand for immediate change, granting women the rights of men.

Summary ScoreTotal possible points / points earned:

"T" S A T (title, source, author, topic) 20 / 162–4 main ideas 60 / 605 underlined vocabulary words 20 / 16

100 / 92

The student lost four points for not putting quotation marks around the title and four points for not using a fifth key vocabulary word in context from the original document. Note that the student was not penalized for mistakes outside the focus correction areas, such as the misspelling of the wordmodeled in the second sentence.

Endnote1 For more on focus correcting, see The Collins Writing Program: Improving Student Performance Through Writing and Thinking Across the Curriculum (Collins Education Associates, 2007).

ReferencesGraham, S., & Hebert, M. (2010). Writing to read: Evidence for how writing can improve reading. New

York: Alliance for Excellent Education.

Graham, S., & Perin, D. (2007). Writing next: Effective strategies to improve writing of adolescents in middle and high schools. New York: Alliance for Excellent Education.

Marzano, R., Pickering, D., & Pollock, J. (2001). Classroom instruction that works. Alexandria, VA: ASCD.

John Collins is director of Collins Education Associates, West Newbury, Massachusetts, and author of The Collins Writing Program: Improving Student Performance Through Writing and Thinking Across the Curriculum (Collins Education Associates, 2007).

5/19/2023 Page 32

Copyright © 2012 by ASCD

Appendix MArticles to Practice Summarization

The New York Times Gray Matter - By HENRY L. ROEDIGER III

How Tests Make Us SmarterJULY 20, 2014

TESTS have a bad reputation in education circles these days: They take time, the critics say, put students under pressure and, in the case of standardized testing, crowd out other educational priorities. But the truth is that, used properly, testing as part of an educational routine provides an important tool not just to measure learning, but to promote it.

In one study I published with Jeffrey D. Karpicke, a psychologist at Purdue, we assessed how well students remembered material they had read. After an initial reading, students were tested on some passages by being given a blank sheet of paper and asked to recall as much as possible. They recalled about 70 percent of the ideas.

Other passages were not tested but were reread, and thus 100 percent of the ideas were re-exposed. In final tests given either two days or a week later, the passages that had been tested just after reading were remembered much better than those that had been reread.

What’s at work here? When students are tested, they are required to retrieve knowledge from memory. Much educational activity, such as lectures and textbook readings, is aimed at helping students acquire and store knowledge. Various kinds of testing, though, when used appropriately, encourage students to practice the valuable skill of retrieving and using knowledge. The fact of improved retention after a quiz — called the testing effect or the retrieval practice effect — makes the learning stronger and embeds it more securely in memory.

This is vital, because many studies reveal that much of what we learn is quickly forgotten. Thus a central challenge to learning is finding a way to stem forgetting.

The question is how to structure and use tests effectively. One insight that we and other researchers have uncovered is that tests serve students best when they’re integrated into the regular business of learning and the stakes are not make-or-break, as in standardized testing. That means, among other things, testing new learning within the context of regular classes and study routines.

Students in classes with a regimen of regular low- or no-stakes quizzing carry their learning forward through the term, like compounded interest, and they come to embrace the regimen, even if they are skeptical at first. A little studying suffices at exam time — no cramming required.

Moreover, retrieving knowledge from memory is more beneficial when practice sessions are spaced out so that some forgetting occurs before you try to retrieve again. The added effort required to recall the information makes learning stronger. It also helps when retrieval practice is mixed up — whether you’re practicing hitting different kinds of baseball pitches or solving different solid geometry problems in a

5/19/2023 Page 33

random sequence, you are better able later to discriminate what kind of pitch or geometry problem you’re facing and find the correct solution.

Surprisingly, researchers have also found that the most common study strategies — like underlining, highlighting and rereading — create illusions of mastery but are largely wasted effort, because they do not involve practice in accessing or applying what the students know.

When my colleagues and I took our research out of the lab and into a Columbia, Ill., middle school class, we found that students earned an average grade of A- on material that had been presented in class once and subsequently quizzed three times, compared with a C+ on material that had been presented in the same way and reviewed three times but not quizzed. The benefit of quizzing remained in a follow-up test eight months later.

Notably, Mary Pat Wenderoth, a biology professor at the University of Washington, has found that this benefit holds for women and underrepresented minorities, two groups that sometimes experience a high washout rate in fields like the sciences.

This isn’t just a matter of teaching students to be better test takers. As learners encounter increasingly complex ideas, a regimen of retrieval practice helps them to form more sophisticated mental structures that can be applied later in different circumstances. Think of the jet pilot in the flight simulator, training to handle midair emergencies. Just as it is with the multiplication tables, so it is with complex concepts and skills: effortful, varied practice builds mastery.

We need to change the way we think about testing. It shouldn’t be a white-knuckle finale to a semester’s work, but the means by which students progress from the start of a semester to its finish, locking in learning along the way and redirecting their effort to areas of weakness where more work is needed to achieve proficiency.

Standardized testing is in some respects a quest for more rigor in public education. We can achieve rigor in a different way. We can instruct teachers on the use of low-stakes quizzing in class. We can teach students the benefits of retrieval practice and how to use it in their studying outside class. These steps cost little and cultivate habits of successful learning that will serve students throughout their lives.

Word Count: 822

5/19/2023 Page 34

Magazine | The Education Issue

Why Flunking Exams Is Actually a Good Thing

By BENEDICT CAREY SEPT. 4, 2014

Imagine that on Day 1 of a difficult course, before you studied a single thing, you got hold of the final exam. The motherlode itself, full text, right there in your email inbox — attached mistakenly by the teacher, perhaps, or poached by a campus hacker. No answer key, no notes or guidelines. Just the questions.

Would that help you study more effectively? Of course it would. You would read the questions carefully. You would know exactly what to focus on in your notes. Your ears would perk up anytime the teacher mentioned something relevant to a specific question. You would search the textbook for its discussion of each question. If you were thorough, you would have memorized the answer to every item before the course ended. On the day of that final, you would be the first to finish, sauntering out with an A+ in your pocket. And you would be cheating.

But what if, instead, you took a test on Day 1 that was just as comprehensive as the final but not a replica? You would bomb the thing, for sure. You might not understand a single question. And yet as disorienting as that experience might feel, it would alter how you subsequently tuned into the course itself — and could sharply improve your overall performance.

This is the idea behind pretesting, one of the most exciting developments in learning-science. Across a variety of experiments, psychologists have found that, in some circumstances, wrong answers on a pretest aren’t merely useless guesses. Rather, the attempts themselves change how we think about and store the information contained in the questions. On some kinds of tests, particularly multiple-choice, we benefit from answering incorrectly by, in effect, priming our brain for what’s coming later.

That is: The (bombed) pretest drives home the information in a way that studying as usual does not. We fail, but we fail forward.

The excitement around prefinals is rooted in the fact that the tests appear to improve subsequent performance in topics that are not already familiar, whether geography, sociology or psychology. At least they do so in experiments in controlled laboratory conditions. A just-completed study — the first of its kind, carried out by the U.C.L.A. psychologist Elizabeth Ligon Bjork — found that in a live classroom of Bjork’s own students, pretesting raised performance on final-exam questions by an average of 10 percent compared with a control group.

The basic insight is as powerful as it is surprising: Testing might be the key to studying, rather than the other way around. As it turns out, a test is not only a measurement tool. It’s a way of enriching and altering memory.

5/19/2023 Page 35

Many of us dread tests because we’ve been wounded by a few over the years, and sometimes severely. Almost everyone has had at least one lost-in-space experience, opening an exam to find a long list of questions that seem to hail from another course altogether. Vision narrows, the mind seizes; all feeling drains from the extremities. We would crawl into a hole if we weren’t already in one.

Yet another species of exam collapse is far more common. These are the cases in which we open the test and see familiar questions on material we’ve studied, perhaps even stuff we’ve highlighted with yellow marker: names, ideas, formulas we could recite easily only yesterday. And still we lay an egg, scoring average or worse.

Why does this happen? Psychologists have studied learning long enough to have an answer, and typically it’s not a lack of effort (or of some elusive test-taking gene). The problem is that we have misjudged the depth of what we know. We are duped by a misperception of “fluency,” believing that because facts or formulas or arguments are easy to remember right now, they will remain that way tomorrow or the next day. This fluency illusion is so strong that, once we feel we have some topic or assignment down, we assume that further study won’t strengthen our memory of the material. We move on, forgetting that we forget.

Often our study “aids” simply create fluency illusions — including, yes, highlighting — as do chapter outlines provided by a teacher or a textbook. Such fluency misperceptions are automatic; they form subconsciously and render us extremely poor judges of what we need to restudy or practice again. “We know that if you study something twice, in spaced sessions, it’s harder to process the material the second time, and so people think it’s counterproductive,” Nate Kornell, a psychologist at Williams College, said. “But the opposite is true: You learn more, even though it feels harder. Fluency is playing a trick on judgment.”

The best way to overcome this illusion is testing, which also happens to be an effective study technique in its own right. This is not exactly a recent discovery; people have understood it since the dawn of formal education, probably longer. In 1620, the philosopher Francis Bacon wrote, “If you read a piece of text through twenty times, you will not learn it by heart so easily as if you read it ten times while attempting to recite it from time to time and consulting the text when your memory fails.”

Scientific confirmation of this principle began in 1916, when Arthur Gates, a psychologist at Columbia University, created an ingenious study to further Bacon’s insight. If someone is trying to learn a piece of text from memory, Gates wondered, what would be the ideal ratio of study to recitation (without looking)? To interrogate this question, he had more than 100 schoolchildren try to memorize text from Who’s Who entries. He broke them into groups and gave each child nine minutes to prepare, along with specific instructions on how to use that time. One group spent 1 minute 48 seconds memorizing and the remaining time rehearsing (reciting); another split its time roughly in half, equal parts memorizing and rehearsing; a third studied for a third and recited for two-thirds; and so on.

5/19/2023 Page 36

After a sufficient break, Gates sat through sputtered details of the lives of great Americans and found his ratio. “In general,” he concluded, “best results are obtained by introducing recitation after devoting about 40 percent of the time to reading. Introducing recitation too early or too late leads to poorer results.” The quickest way to master that Shakespearean sonnet, in other words, is to spend the first third of your time memorizing it and the remaining two-thirds of the time trying to recite it from memory.

In the 1930s, a doctoral student at the State University of Iowa, Herman F. Spitzer, recognized the broader implications of this insight. Gates’s emphasis on recitation was, Spitzer realized, not merely a study tip for memorization; it was nothing less than a form of self-examination. It was testing as study, and Spitzer wanted to extend the finding, asking a question that would apply more broadly in education: If testing is so helpful, when is the best time to do it?

He mounted an enormous experiment, enlisting more than 3,500 sixth graders at 91 elementary schools in nine Iowa cities. He had them study an age-appropriate article of roughly 600 words in length, similar to what they might analyze for homework. Spitzer divided the students into groups and had each take tests on the passages over the next two months, according to different schedules. For instance, Group 1 received one quiz immediately after studying, then another a day later and a third three weeks later. Group 6, by contrast, didn’t take one until three weeks after reading the passage. Again, the time the students had to study was identical. So were the quizzes. Yet the groups’ scores varied widely, and a clear pattern emerged.

The groups that took pop quizzes soon after reading the passage — once or twice within the first week — did the best on a final exam given at the end of two months, marking about 50 percent of the questions correct. (Remember, they had studied their peanut or bamboo article only once.) By contrast, the groups who took their first pop quiz two weeks or more after studying scored much lower, below 30 percent on the final. Spitzer’s study showed that not only is testing a powerful study technique, but it’s also one that should be deployed sooner rather than later. “Achievement tests or examinations are learning devices and should not be considered only as tools for measuring achievement of pupils,” he concluded.

The testing effect, as it’s known, is now well established, and it opens a window on the alchemy of memory itself. “Retrieving a fact is not like opening a computer file,” says Henry Roediger III, a psychologist at Washington University in St. Louis, who, with Jeffrey Karpicke, now at Purdue University, has established the effect’s lasting power. “It alters what we remember and changes how we subsequently organize that knowledge in our brain.”

If tests are most effective when given sooner rather than later, then why not go the distance? Why not give the final on the first day, as well as on the last? This is the radical question that Bjork, the U.C.L.A. psychologist, has set out to investigate.

She did not actually give a comprehensive prefinal on the first day of class, in order to avoid overwhelming her students. She also decided to start with fairly basic material, conducting the study on her Psychology 100B class at U.C.L.A., which covers research methods.

She and Nicholas Soderstrom, a postdoc, gave the entire class of more than 300 students a short pretest, all multiple-choice questions, immediately before the start of some lectures but not others. “We

5/19/2023 Page 37

wanted to see whether students would better remember and understand material from lectures preceded by a pretest than from lectures not preceded by a pretest,” Soderstrom said.

To answer that, Bjork and Soderstrom did something clever on a cumulative final exam, which was given at the end of the course. Namely, they included on it questions that were related to the pretest ones as well as questions that were not. “If pretesting helps, then students should do better on related questions during a later exam than on questions about material we covered in the lectures but was not pretested,” Bjork said. She and Soderstrom would compare students’ scores on pretest-related questions with their scores on nonpretested ones, to see if there was any difference.

For example, here’s a question from one of the pretests:

Which of the following is true of scientific explanations?

a. They are less likely to be verified by empirical observation than other types of explanations.

b. They are accepted because they come from a trusted source or authority figure.

c. They are accepted only provisionally.

d. In the face of evidence that is inconsistent with a scientific explanation, the evidence will be questioned.

e. All of the above are true about scientific explanations.

And here’s a related question, from the cumulative test given after the lectures:

Which of the following is true of explanations based on belief?

a. They are more likely to be verified by empirical observation than other types of explanations.

b. They are accepted because they come from a trusted source or authority figure.

c. They are assumed to be true absolutely.

d. In the face of evidence that is inconsistent with an explanation based on belief, the belief will be questioned.

e. b and c above.

5/19/2023 Page 38

The students tanked all three pretests, performing no better than if they had guessed at random. Bjork and Soderstrom had expected as much. But the class received prompt feedback, attending the relevant lecture shortly after they took each of the three pretests. Those lectures in effect supplied them with correct answers to questions that had just been posed on the pretest. In previous experiments, such immediacy seemed to be a critical component: Pretests led to the most improvement when students received the correct answers reasonably soon after their guessing.

In order to gauge the effect of the testing, Bjork and Soderstrom gave a cumulative exam at the end of the 10-week course. It was the same format as the others: multiple-choice questions, each with five possible answers. The result? Bjork’s Psych 100B class scored about 10 percent higher on the related questions than on the unrelated ones. It’s far from a magic memory pill — but 10 percent, as we all know, can often translate to a letter grade. “On the basis of this significant difference,” Bjork said, “giving students a pretest on topics to be covered in a lecture improves their ability to answer related questions about those topics on a later final exam.” Even when students bomb, she said, pretests provide them an opportunity to see what vocabulary will be used in the coming lectures, what kinds of questions will be posed and which distinctions between concepts will be crucial.

Bjork’s experiment suggests that pretesting serves to prime the brain, predisposing it to absorb new information. Scientists have several theories as to how this happens. One is fairly obvious: Students get a glimpse from a pretest of the teacher’s hand, of what they’ll be up against. That’s in the interest of not just students but of teachers, too. You can teach facts and concepts all you want, but what’s most important in the end is how students think about that material: How they incorporate all those definitions into a working narrative about a topic that gives them confidence in judging what’s important and what’s less so. These are not easy things to communicate, even for the best teachers. You can’t download such critical thinking quickly, hard as you might try. But you can easily give a test with questions that themselves force that kind of hierarchical thinking. “Taking a practice test and getting wrong answers seems to improve subsequent study, because the test adjusts our thinking in some way to the kind of material we need to know,” Bjork said.

A second possibility has to do with the concept of fluency. Wrong guesses expose our fluency illusions, our false impression that we “know” the capital of Eritrea because we just saw it or once studied it. A test, if multiple-choice, forces us to select the correct answer from a number of possibilities that also look plausible. “Let’s say you’re pretty sure that Australia’s capital is Canberra,” Robert A. Bjork, Elizabeth Ligon Bjork’s husband and a leading learning scientist, said. “O.K., that seems easy enough. But when the exam question appears, you see all sorts of other possibilities — Sydney, Melbourne, Adelaide — and suddenly you’re not so sure. If you’re studying just the correct answer, you don’t appreciate all the other possible answers that could come to mind or appear on the test.” Pretesting operates as a sort of fluency vaccine.

Biologically, too, there may be something deeper at work. To review, memory builds on itself in ways we don’t usually notice. Retrieval — i.e. remembering — is a different mental act than straight studying; the brain is digging out a fact, together with a network of associations, which alters and enriches how that network is subsequently re-stored. But guessing is distinct from both study and retrieval. It too will reshape our mental networks by embedding unfamiliar concepts (the lend-lease program, the confirmation bias, the superego) into questions we at least partly comprehend (“Name one psychological phenomenon that skews our evaluation of evidence”). Even if the question is not entirely clear and its solution unknown, a guess will in itself begin to link the questions to possible answers. And those networks light up like Christmas lights when we hear the concepts again.

5/19/2023 Page 39

And here is where pretesting shows its likely limitations: A prefinal for an intro class in Arabic or Chinese could be a wash, because the notations and characters are entirely alien. There’s no scaffolding of familiar language to work with — no existing network in which to situate the new symbols — before we make a guess. We are truly lost, with no recognizable landmark. The research thus far suggests that prefinals will be much more useful in humanities courses and social-science disciplines in which unfamiliar concepts are at least embedded in language we can parse.

The word “testing” is still loaded, of course, in ways that have nothing to do with learning science. Educators and experts have debated the value of standardized testing for decades, and reforms like the No Child Left Behind law, which increased the use of such exams, have only inflamed the argument. Many teachers complain that a focus on testing limits their ability to fully explore subjects with their students. Others attack tests as woefully incomplete measures of learning, blind to all varieties of creative thinking.

But the emerging study of pretesting flips that logic on its head. “Teaching to the test” becomes “learning to understand the pretest,” whichever one the teacher chooses to devise. The test, that is, becomes an introduction to what students should learn, rather than a final judgment on what they did not.

Word Count: 2,884

__________

Benedict Carey is a science reporter for The Times and the author of How We Learn: The Surprising Truth About When, Where and Why It Happens from which this article is adapted.

Editor: Ilena Silverman

5/19/2023 Page 40

The New York TimesSunday Review

Studying for the Test by Taking It

By BENEDICT CAREY NOV. 22, 2014

PROTESTS are flaring up in pockets of the country against the proliferation of standardized tests. For many parents and teachers, school has become little more than a series of workout sessions for the assessment du jour.

And that is exactly backward, research shows. Tests should work for the student, not the other way around.

In an experiment published late last year, two University of Texas psychologists threw out the final exam for the 900 students in their intro psych course and replaced it with a series of short quizzes that students took on their laptops at the beginning of each class.

“They didn’t like it, at least at first,” said one of the professors, James W. Pennebaker. The other professor, Samuel D. Gosling, added, “For the first few weeks, every time their friends went out drinking, they couldn’t go — they had yet another quiz the next day.”

But they did significantly better than a comparison intro psych class, both in their grades and on a larger quiz that included 17 of the same questions that appeared both in the quizzes and on the other class’s midterm. The quizzes were especially beneficial for the type of students — many from low-performing high schools — who don’t realize how far behind they are until it’s too late.

One leading researcher in this field, Henry L. Roediger III of Washington University in St. Louis, argues that tests of varying scale and intensity can deepen learning. “We now know that testing, including self-testing, is an especially powerful form of study,” said Dr. Roediger, co-author of the book “Make It Stick.”

How so? Because retrieving facts, formulas or concepts is a threefold mental act: finding the sought-after information in the vast catacombs of the brain; bringing it consciously to mind; and finally, storing it. That newly stored memory will be embedded in a host of additional associations and connections and be much easier to recall later than if you’d merely read it again.

Testing has so many dimensions that it can often be easily disguised. In a study published last month, the cognitive scientist Doug Rohrer of the University of South Florida made a clever change in the math homework of middle school students in Tampa, Fla. Working with teachers, he essentially gave each student two types of assignments. To test some skills they got so-called blocked practice — concentrated drills on a single thing, like solving for X. On other skills they got mixed practice, that is, blending certain skills — say graphing — with other types of problems from the course.

5/19/2023 Page 41

The rationale was straightforward. Practicing skills in isolation leads to noticeable improvement, but students do not have to shift gears. They know how to proceed because the assignment says “graphs” or “solving for X” at the top of the page.

Yet at the end of the term, on a surprise test, the students solved 72 percent of the problems that they’d studied in mixed sets, compared with 38 percent of those in blocked, homework-as-usual sets. The problems and teachers were the same, and the classes were taught as usual. The only change was in the homework.

In the jargon of the field, this strategy of mixed problem-sets is called interleaving. It is distinct from the end-of-unit reviews that teachers commonly give, because it’s self-guided and continual. It mimics a test in a crucial way, in that it forces students to distinguish between types of problems and decide which strategy is appropriate.

The beauty of broadening testing beyond basic assessment is that the approach can be applied at home, easily, for students of all ages. Most young children squirm through their homework, but many love to play teacher. By cutting short “study time” and asking them to be the teacher, parents can make the session more fun, more interactive and a richer learning experience. Teaching is self-testing of an especially potent kind.

Ditto for making an outline of a chapter (with the book closed), or discussing the material with a friend or roommate. One reason scientists suspect that studying in pairs or groups can be helpful is that students are forced to talk to one another about the material — or better yet, argue about it. These are all forms of self-examination, and as such deepen learning more than passively rereading or reviewing the material.

The brain is an exotic learning machine, to put it mildly. It does not take orders well. You can tell it to remember the major players in the settling of Manhattan, stress how crucially important that is, and on the test a week later very little comes back. And yet you might remember nearly every play in the San Francisco Giants’ Game 7 World Series victory. Why? Because the brain doesn’t listen to what you say; it watches what you do. And thinking often about Madison Bumgarner pitching, talking about the game, arguing about it: These are mental actions, as well as subtle forms of testing knowledge.

Testing in all its permutations, subtle and otherwise, convinces the brain that the knowledge is useful, and important. And by varying one’s testing strategies, the actual final exam — the dreaded assessment — isn’t nearly as scary.

Word Count: 857

5/19/2023 Page 42

Boston Globe Magazine

Studying: You're Doing it WrongBy Peter C. Brown, Henry L. Roediger III and Mark A. McDaniel March 09, 2014

PEOPLE COMMONLY BELIEVE that if you expose yourself to something enough times — say, a textbook passage or a set of terms from biology class — you can burn it into memory. Not so. Many teachers believe that if they can make learning easier and faster, the learning will be better. Much research turns this belief on its head: When learning is harder, it’s stronger and lasts longer. It’s widely believed by teachers, trainers, and coaches that the most effective way to master a new skill is to give it dogged, single-minded focus, practicing over and over until you’ve got it down. What’s apparent from research is that gains achieved during such practice are transitory and melt away quickly.

In fact, what students are advised to do is often plain wrong. For instance, study tips published on a website at George Mason University include this advice: “The key to learning something well is repetition; the more times you go over the material, the better chance you have of storing it permanently.” Another, from a Dartmouth College website, suggests: “If you intend to remember something, you probably will.” Belief in the power of rereading, intentionality, and repetition is pervasive, but the truth is, you usually cannot embed something in memory simply by repeating it over and over.

Consider a simple example called the “penny memory test,” which presents a dozen different images of a common penny, only one of which is correct. As many times as you’ve seen a penny, you’re hard pressed to say with confidence which one it is.

The finding that rereading textbooks is often labor in vain ought to send a chill up the spines of educators and learners, because it’s the number one study strategy of most people — including more than 80 percent of college students in some surveys — and is central in what we tell ourselves to do during the hours we dedicate to learning. Rereading has three strikes against it: It is time-consuming; it doesn’t result in durable memory; and it often involves a kind of unwitting self-deception, as growing familiarity with the text comes to feel like mastery of the content. The hours immersed in rereading can seem like due diligence, but the amount of study time is no measure of mastery.

It turns out that much of what we’ve been doing as teachers and students isn’t serving us well. But some comparatively simple changes in how we study could make a big difference, regardless of age. Here are some of the principal insights that we and other cognitive scientists have gathered from our research into effective learning:

Learning is deeper and more durable when it costs effort. Learning that’s easy is like writing in sand, here today and gone tomorrow.

We are poor judges of when we are learning well and when we are not. When the going is harder and slower and it doesn’t feel productive, we are drawn to strategies that feel more fruitful, unaware that the gains from these strategies are often temporary.

5/19/2023 Page 43

Rereading text and massed practice of a skill or review of new knowledge are common study strategies of learners of all stripes, but they’re also among the least productive. By massed practice, we mean the single-minded, rapid-fire repetition of something you’re trying to burn into memory, the “practice-practice-practice” of conventional wisdom. Cramming for exams is an example. Rereading and massed practice give rise to feelings of fluency that are taken to be signs of mastery, but for true mastery or durability these strategies are largely a waste of time.

Retrieval practice — recalling facts or concepts from memory — is a more effective learning strategy than review by rereading. Flashcards are a simple example. Retrieval strengthens the memory and interrupts forgetting. A single, simple quiz after reading a text or hearing a lecture produces better learning and remembering than rereading the text or reviewing lecture notes. While the brain is not a muscle that gets stronger with exercise, the neural pathways that make up a body of learning do get stronger. Periodic practice arrests forgetting, strengthens retrieval routes, and is essential for hanging onto the knowledge you want to gain.

When you space out practice at a task and get a little rusty between sessions, or you interleave — that is, alternate between — the practice of two or more subjects, retrieval is harder and feels less productive, but the effort produces longer-lasting learning and enables more versatile application of it in later settings.

Trying to solve a problem before being taught the solution leads to better learning, even when errors are made in the attempt.

The popular notion that you learn better when you receive instruction in your preferred learning style — for example, as an auditory or visual learner — is not supported by the empirical research. People do have multiple forms of intelligence, and you learn better when you “go wide,” drawing on all of your aptitudes and resourcefulness, than when you limit instruction or experience to the style you find most amenable.

When you’re adept at extracting the underlying principles or “rules” that differentiate types of problems, you’re more successful at picking the right solutions in unfamiliar situations. This skill is better acquired through interleaved and varied practice than massed practice. For instance, interleaving the identification of different types of birds or the works of different oil painters improves your ability both to learn the unifying attributes within a type and to differentiate between types, improving your skill at categorizing new specimens you encounter later.

We’re all susceptible to illusions that can hijack our judgment of what we know and can do. Testing helps calibrate our judgments of what we’ve learned. In virtually all areas of learning, you build better mastery when you use testing as a tool to identify and improve your areas of weakness.

In a “Far Side” cartoon by Gary Larson, a bug-eyed kid asks his teacher: “Mr. Osborne, can I be excused? My brain is full!” If you’re just engaging in mechanical repetition, it’s true, you quickly hit the limit of what you can retain. However, if you practice elaboration, there’s no known limit to how much you can learn. Elaboration is the process of giving new material meaning by expressing it in your own words and connecting it with what you already know. The more you can explain about the way your new learning relates to your prior knowledge, the stronger your grasp of the new learning will be and the more connections you create that will help you remember it later.

5/19/2023 Page 44

Putting new knowledge into a larger context helps learning. For example, the more of the unfolding story of history you know, the more of it you can learn. And the more ways you give that story meaning, say by connecting it to your understanding of human ambition and the untidiness of fate, the better the story stays with you. Likewise, if you’re trying to learn an abstraction, like the principle of angular momentum, it’s easier when you ground it in something concrete that you already know, like the way a figure skater’s rotation speeds up as she draws her arms to her chest.

Many people believe that their intellectual ability is hard-wired from birth and that failure to meet a learning challenge is an indictment of their native ability. But every time you learn something new, you change the brain — the residue of your experiences is stored. It’s true that we start life with the gift of our genes, but it’s also true that we become capable through the learning and development of mental models that enable us to reason, solve, and create.

In other words, the elements that shape your intellectual abilities lie to a surprising extent within your own control.

Word Count: 1,289

5/19/2023 Page 45