Are Readability Scores Passé and too “Old-fashioned”

for Today’s World?

Fay TessWSRA Reading & Technology Committee

http://www.nconnect.net/~faytess/faytess@nconnect.net

Two important elements in writing…

1. The reading skills of the audience

2. The readability of the text

Some golden rules of documentation– Use short, simple, familiar words– Avoid jargon.– Use correct grammar, punctuation, and spelling.– Use simple sentences, active voice, and present tense.– Use simple graphic elements such as bulleted lists and

numbered steps to make information visually accessible.

How Users Read on the Web: http://www.useit.com/alertbox/9710a.html

The Relationship Between Reading & Writing

• WSRA Position Statement– http://www.wsra.org/readwrite.html

• “Reading and writing to be understood and appreciated fully, should be viewed together, learned together, and used together." Tierney and Shanahan (1991) – http://wilearns.state.wi.us/apps/default.asp?ci

d=20

Do we use them today?

• By the 1980s—– IRA: actually resolved against using the formulas– NCTE: Advise against “uncritical” use of readability

formulas.• They do not help us predict how well a reader will understand

the text!– 200 formulas– Over a thousand studies published on the readability

formulas attesting to their strong theoretical and statistical validity.

• Today they are used in journalism, research, health care, law, insurance, and industry.

• Today—Lexile scores “replacing” readability formulas

Has the readability level of information changed through the ages?

• In 1880, a professor of English Literature at the University of Nebraska, Lucius Adelno Sherman, compared how sentence length averages shortened over time:

– Pre-Elizabethan times: 50 words per sentence

– Elizabethan times: 45 words per sentence

– Victorian times: 29 words per sentence

– Sherman’s time: 23 words per sentence

• Today: 20 words per sentence.

• Future: recommendation 16 words per sentence

What is readability?

• Readability is what makes some texts easier to read than others.

• It is often confused with legibility, which concerns typeface and layout– http://www.jaddesignsolutions.com/legibility01.html – (Throw the rules out!)

• Resources:– http://www.lighthouse.org/print_leg.htm– http://www.lighthouse.org/about_browser.htm– http://desktoppub.about.com/od/typelegibility/– http://www.rationalchristianity.net/usability/legibility.html

• (Text on web pages)

• http://www.raedata.com/readlev.html

What effect does handwriting have on reading?

• Girls tend to perform better in free response writing

• The quality of handwriting does have an impact on the scores awarded to essays

• Increased legibility results in higher ratings

• Essays with better handwriting or typed scripts received higher scores– http://www.ielts.org/teachersandresearchers/research/art

icle84.aspx– Study done by the International English Language Testing System

What does this mean?

• 40 to 44 million adults in the U.S. are in Level 1 : “functionally illiterate, not having enough reading skills for daily life.”

• Some 50 million are in Level 2—do not read much better.

• The percentage of adults who struggle at Levels 1 and 2 (below the 5th-grade level) in the U.S. reaches 48 percent.

How does this effect them…and us?

• The report confirmed that – other skills increase with reading skills.

• Adults of different reading skills have different life experiences. – 43% of adults with low-literacy skills live in poverty, – 17% receive food stamps, and – 70% have no job or only part time jobs.– 20% + of adults read below the sixth-grade level – Adults at Level 1 earned a median income of $240 a

week, while those at Level 5 earned $681– 70% of prisoners are in the lowest two levels.

Readability formulas have limitations

• A low style of writing can result from slavish use of readability formulas– A monotonous succession of short sentences

and simple words can make writing dull which won’t hold the reader’s attention

– Readability formulas were designed for ‘rating, not for writing,’ i.e. they are useful for analyzing text AFTER it has been written

• Formulas do not measure comprehension, grammar, or the use of technical or abstract words

What do we mean by grade level?

• The reading level predicted by a readability test is the ‘break-off’ point for a reader of that reading age. ie. a reading level measured as 14 years predicts that an average 14-year-old would be at the limit of his/her reading comprehension ability with that book !

• This is because most readability formulae are based on a 50 per cent correct answer score in a comprehension test.

• So if a book has reading level of 14 years, an average 14-year-old pupil would score only 50 per cent on a test of comprehension of that text !

• . . . and 50 per cent is a long way from full comprehension !

Source: http://www.timetabler.com/

Readability Tests

• Gunning ‘FOG’ Readability Test– Suitable for secondary and older primary age groups

• Fry Readability Graph– Suitable for all ages, from infant to upper secondary

(estimates reading age in years)

• Flesh-Kincaid (Grade Level) Formula– US Government of Defense standard test– Grade level

• Flesch (Reading Ease) Formula– Reading Ease – Works well for upper elementary and secondary texts

• McLaughlin ‘SMOG’ formula– Gives higher values than other formulae– Intended to predict the level necessary for 100%

comprehension

Flesch: Grade Level and Reading Ease

Rudolf Flesch writes, “Here's how the scores translate into school grades. Reading matter with the score shown on the left side will be easy for students on the level shown on the right.” (Flesch, n.d.)Flesch

For Readability Computational Chart and list of magazines scores —click on link below

Flesch Reading Ease Score Flesch-Kincaid Grade Level Score

Score School Level

90 to 100 5th grade

80 to 90 6th grade

70 to 80 7th grade

60 to 70 8th and 9th grade

50 to 60 10th to 12th grade (high school)

30 to 50 college

0 to 30 college graduate

*Adapted from an article by Rudolf Flesch, How to Write Plain English: Chapter 2: Let's Start With the Formula” which was retrieved from the Internet November 4, 2004)

http://www.mang.canterbury.ac.nz/writing_guide/writing/flesch.shtml Description of formulas starts at page 13.

More readability formulas

• Forcast Formula– Devised for assessing US army technical manuals– Not suitable for primary age materials– Does not need whole sentences so it is suitable for assessing notes

and multiple choice questions– Tested against a comprehension level of 35%

• Powers-Sumner-Kearl Formula– Not suitable for secondary age books; most suitable for material in the

7-10 age range

• Cloze Procedure– Used to measure the suitability of text– Scoring objective– Easy to use and analyze– Yiels correlations to other formulas– Does not actually tell if the materials are comprehensible– http://www.adams12.org/literacy/secondary/cloze.htm

• Source: http://www.ed.psu.edu/paliteracycorps/hied/course_materials/Teaching_Strategies.pdf

…and still a few more.• Spache Readability Formula

– http://www.interventioncentral.org/htmdocs/tools/okapi/okapimanual/spacheWorksheet.PDF

– The average number of words per sentence– The percent of words in the passage NOT found on the Spache Revised World List

(fewer than 1000 words– http://www.interventioncentral.org/htmdocs/tools/okapi/okapimanual/spachelist.shtml

• Dale-Chall Readability Formula– http://www.interventioncentral.org/htmdocs/tools/okapi/okapimanual/dalechall1.shtml – The average number of words per sentence (3000 words)– The percent of words in the passage NOT found on the Dale Word List– http://www.interventioncentral.org/htmdocs/tools/okapi/okapimanual/dalechalllist.shtm

l

• Will give different results, but they are designed to complement each other: – Spache: texts up and through 3rd grade– Dale-Chall: texts beyond 3rd grade

– http://www.interventioncentral.org/htmdocs/tools/okapi/okapimanual/okapimanual.shtml

Lexile Framework for Reading—what is it?

From Lexile.com• A scientific approach to reading measurement that matches readers to text. • measures both reader ability and text difficulty on the same scale, called the

Lexile scale.

• allows educators and families to manage reading comprehension and encourage reader progress using Lexile measures and a broad range of Lexile products, tools and services.

• More than 14 million students are estimated to have received a Lexile measure in the 2003-04 school year.

• All major standardized tests can report student reading scores in Lexiles. • More than 450 publishers Lexile their materials to meet this demand,

applying Lexile measures to tens of thousands of books and tens of millions of articles.

• http://technology.usd259.com/resources/SRI/documents/SRI-Overview.ppt#19

– SRI : Scholastic Reading Inventory– Uses lexiles– This gives some introductory information about lexiles

Lexiles—how does it work?• Check the lexile score for almost any book.

– http://www.lexile.com/DesktopDefault.aspx?view=ed&tabindex=5&tabid=67

• Book Search– http://www.lexile.com/DesktopDefault.aspx?view=pa&tabindex=3&tabid=69

• Vocabulary– http://www.lexile.com/DesktopDefault.aspx?view=pa&tabindex

=2&tabid=26&tabpageid=190

• Calculator: determine expected comprehension level– http://www.lexile.com/DesktopDefault.aspx?view=ed&tabindex=2&tabid=16&ta

bpageid=56

• Series books• http://www.lexile.com/DesktopDefault.aspx?view=ed&tabindex=2&tabid=16&tabpageid

=182

• FAQ– http://www.lexile.com/DesktopDefault.aspx?view=ed&tabindex=6&tabid=18

• Spanish: find books and lexile levels in Spanish– http://www.lexile.com/DesktopDefault.aspx?

view=ed&tabindex=13&tabid=67&tabpageid=312

The Lexile Scale• Based on two factors:

semantic difficulty and syntactic complexity

•Word frequency

•Sentence length

• Also considers log freqency

•An logarithm which takes into account the number of times a word appears in each 5-million words of nearly 600 million words—used to determine the syntactic difficulty

•Text such as recipes, poetry, and song lyrics are not measured because they do not contain standard punctuation

• The measurement unit has less potential stigma than grade-equivalent leveling

http://teacher.scholastic.com/products/sri/pdfs/Lexile_Map.pdf

TRIO from Scholastic compares three different readability scores for the same materials

• http://teacher.scholastic.com/products/trio/TRIOLevelingGuide.pdf – Lexile score– Guided Reading Levels– DRA (Directed Reading Assessment

Commentary & comparisons of readability results• http://www.leapfrogschoolhouse.com/content/brochures/

LRSguide.pdf

• Created a list of books and scored each using the various readability formulas

• Assessing the Lexile Framework: Results of a Panel Meeting

• http://nces.ed.gov/pubs2001/200108.pdf

– How schools view the Lexile Framework• http://www.mvcsc.k12.in.us/nwea/datacoach.ppt#43 • http://www.ode.state.or.us/teachlearn/specialty/pre-post/

012004/jan2004slides.pdf

Subtleties in writing styleConsider the following sentences:          

– The cat sat on your mat.            – The cat sat on the mat. – On the mat the cat sat.– Sat, on the mat: the cat.The cat on the mat sat. – On the mat sat the cat. – Sat: the cat on the mat. – Sat the cat on the mat?

Readability formulae will give the same value to each of these sentences even though the first example is probably the most readable.  

It has a personal touch and a clear sequence of subject-verb-object which names the topic and then describes it.

However readability formulae do distinguish very clearly between crisp and extended styles of writing.

They will easily distinguish between:        

‘The cat sat on the mat’  and     ‘The feline reclined on the axminster’.

Both of these would compute the sameCat the mat the sat on.The cat sat on the mat.

Practical Example: (Re-)writing for ReadabilityQuoted from• http://www.pfizerhealthliteracy.com/improving_e

mpathy.html

• Original text - Flesch Reading Ease 14.5: – A living will is a written declaration directing your doctor

to provide, withhold or withdraw lifesaving procedures should you be diagnosed as having a terminal condition or exist in a vegetative state and you lack the capacity to make the decision.

• Revised text in an easy-to-read format - Flesch Reading Ease 68.1 (^ = better): – A living will is a paper you sign telling your doctor

whether you want treatments that could keep you alive if you have a terminal disease or are in a coma. This lets your doctor know how you want to be treated if you are not able to make the decision yourself.

How readability levels differ

The following sentences give examples of how reading levels differ:

College Reading LevelWith the onset of nausea, diarrhea or other gastrointestinal disturbances, consult your physician immediately.

• 12th Grade Reading LevelIf you experience nausea, diarrhea or other stomach or bowel problems, call your physician immediately.

• 8th Grade Reading LevelIf you start having nausea, loose bowel movements or other stomach problems, call your doctor immediately.

• 4th Grade Reading LevelIf you start having an upset stomach, loose bowel movements, or other problems, call your doctor right away.

• Source: http://www.ucdmc.ucdavis.edu/cne/HealthEd/Guide/Read.htm

Directory--Plain Talk• Because of the high cost of aviation fuel, a private pilot once wrote

to the aviation administration asking if he could mix kerosene in his aircraft fuel. He received this reply:– Utilization of kerosene involves major uncertainties/probabilities

respecting shaft output and metal longevity where application pertains to aeronautical internal combustion power plants.

• The pilot sent the following cable: – "Thanks for the information. Will start using kerosene next week.“

• He received the following urgent letter:– Regrettable decision involves uncertainties. Kerosene utilization

consequences questionable, with respect to metalloferrous components and power production.

• This prompted another cable from the pilot: – "Thanks again. It will sure cut my fuel bill.“

• The same day he finally received a clear message:

• DON'T USE KEROSENE. IT COULD KILL THE ENGINE AND YOU TOO!

Change to more acceptable words, phrases and language

Examples of acceptable words, phrases and language:

• Depending on grade level, you may use:"doctor" instead of "physician""shot" instead of "injection""pill" instead of "medication""high blood pressure" instead of "hypertension""making" instead of "manufacturing""cream" instead of "ointment""use" instead of "utilize"

• Address one concept at a time and use uncomplicated sentences:

Use "Your doctor knows how to help you." instead of "Naturally, your physician knows best what is likely to help your particular case."

Readability formulas are popular because they

• Reduce to simple formulas the complex work of writing

• Provide a convenient check and measure of one’s writing

• Possess the glitter of mathematical exactness

• Can be calculated using word processing software

Things they cannot tell you

• How complex the ideas are• Whether or not the content is in a logical order• Whether the vocabulary is appropriate for the

audience• Whether there is a gender, class or culture bias• Whether the design is attractive and helps or

hinders the reader• Whether the material appears in a form and type

style that is easy to read.• Reader resources such as word recognition skills,

interest in the subject, prior knowledge of the topic• It cannot measure the circumstances in which the

reader is using the text—psychological or physical

Formulas give conflicting results

The following example was rated using various formulas• Flesch-Kincaid formula: 6th grade reading level • Coleman-Liau; 7.9 grade level• Bormuth formula: 8.9 grade level. 

The study is good. People will find it important. 

The scientists write that exercise will make people happy. That is because it makes people feel good. They feel good inside and out.

 If more people with disabilities exercise, our nation will be a lot happier.

 

Rules of thumb--readabililty

• Use the formulas as a guide• Formulas will NOT replace the clear

and logical thinking that is the foundation of all writing (and the evaluation of reading materials).

• Readability indexes are helpful, but they do not consider the best motivator of all—interest in the material.

Check the readability of your own writing using Microsoft Word

1. Tools menu2. Click Options3. Click the Spelling & Grammar tab4. Select the Check grammar with spelling check box. 5. Select the Show readability statistics check box, and 6. Click OK. 7. Click Spelling.

When Word finishes checking spelling and grammar, it can display information about the reading level of the document, including the following readability scores. Each readability score bases its rating on the average number of syllables per word and words per sentence.

FleschReading ScoreFlesch-Kincaid Grade Level

http://www.nconnect.net/~faytess/

If your computer cannot check for readability• StyleWriter

– http://www.stylewriter-usa.com/index.html

• Calculations– http://www.micropowerandlight.com

• SpellCatcherPlus– http://www.rainmakerinc.com

• GrammarExpertPlus– http://www.wintertreesoftware.com

• WordPerfect

• Recommendation: Make sure your readability software accurately measures readability. Before you put your trust—and your reputation—into any readability software, analyze a few documents by hand and compare your results to the software.

Checking readability on the web

• http://www.ilovejackdaniels.com/resources/readability-score/– Paste text in and obtain a score.

• http://www.juicystudio.com/fog/

– (Computes the readability of a document, text, or a website.)

– http://www.peacehartford.org

– http://www.reuters.com/newsArticle.jhtml?type=sportsNews&storyID=7333330

– http://seattletimes.nwsource.com/html/sports/2002151567_kell15.html

– http://www.brox1.demon.co.uk/lightning/index.html

• http://www.readability.info/ (Article on flash drives.)

ESL

• http://towerofenglish.com/ – Most of this has many sites and activites particularly useful and

intersting for ESL students. – You may want to check this out the HUMOR category first. While

not pornographic or anything, the humor it is not always “perfectly” politically correct

Babelfish…

• Translate from one language to anotherhttp://babelfish.altavista.com/babelfish/tr

• Translate web pages to another languagehttp://babelfish.altavista.com/babelfish/tr

Translate web pagesEnglish to Japanese

http://www.peacehartford.org

http://www.peacehartford.org