Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Top Lang DisordersVol. 32, No. 4, pp. 297–318Copyright c© 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins
The Developmental WritingScaleA New Progress Monitoring Toolfor Beginning Writers
Janet M. Sturm, Kathleen Cali, Nickola W. Nelson,and Maureen Staskowski
Developing writers make qualitative changes in their written products as they progress fromscribbling and drawing to conventional, paragraph level writing. As yet, a comprehensive mea-surement tool does not exist that captures the linguistic and communicative changes (not justemergent spelling) in the early stages of this progression. The Developmental Writing Scale (DWS)for beginning writers was developed as a tool that can capture evidence of refined changes ingrowth over time. This measure is a 14-point ordinal scale that defines qualitative advances inlevels of a learning progression for beginning writing from scribbling to cohesive (linguisticallyconnected) and coherent (on an identifiable topic) paragraph-level writing. The measure can beused with young typically developing children and children with disabilities at all ages who arefunctioning at beginning levels of writing. Limitations of current writing measures, in contrast tothe DWS, are described. The development of the DWS and techniques for using the measure aredescribed with regard to construct and content validity. Preliminary research on reliability of DWSscoring and validity for 5 purposes support usefulness of the DWS for educational and researchpurposes, including monitoring the progress of beginning writers with significant disabilities.Key words: beginning writers, learning progression, writing assessment, writing scale
Author Affiliations: Central Michigan University,Mount Pleasant, Michigan (Dr Sturm); University ofNorth Carolina at Chapel Hill (Ms Cali); WesternMichigan University, Kalamazoo, Michigan (DrNelson); and Macomb Intermediate School District,Clinton Township, Michigan (Dr Staskowski).
Portions of the Developmental Writing Scale were devel-oped when the first author, as PI, was receiving supportfrom NIH Grant R41HD059238-01 to Don Johnston,Inc. for development of software to support early writ-ing development. The authors would like to thank themany teachers, undergraduate, and graduate studentsfor their ongoing support during the development ofthe Developmental Writing Scale.
The authors disclose that they may receive royalties inthe future for products based on the work described inthis article.
Supplemental digital content is available for thisarticle. Direct URL citations appear in the printedtext and are provided in the HTML and PDF ver-sions of this article on the journal’s Web site (www.topicsinlanguagedisorders.com).
SROD UF DEF POT 1. Hir I whs sentn din rind.SROD UF DEF POT 1. (Grade 5 student)
My role model is my dad because he waledon tools, moshens, and matel baerols. I thinkit’s verey intuorasting to do because I like to fixthing’s, and it’s fun to do waleding. (Grade 8 stu-dent)
Pow Pow Pow, I think it would be a good ideafor teachers to have a gun permit. It would reduceviolence in schools and outside of schools. It willalso protect themselves as well as the students.I think it would be a great idea for it. (Grade 11student)
Corresponding Author: Janet M. Sturm, PhD, CentralMichigan University, 2167 Health Professions Building,Mount Pleasant, MI 48859 ([email protected]).
DOI: 10.1097/TLD.0b013e318272159e
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
297
298 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
EACH OF THE students whose work isquoted previously (shared with permissionfrom the Tennessee Department of Educa-tion, 2010) faces significant challenges for de-veloping sophisticated, conventional writingskills. Although the students are at differentgrade levels, each presents as a “beginningwriter.” These students have something elsein common—they all received a score of a1 out of 6 on their most recent end-of-gradewriting tests. Despite differences that can bereadily observed across the three samples, theholistic scoring method used to evaluate themwas not sensitive enough to detect the differ-ences, nor would it be likely to reflect positivechanges that would represent progress fromone evaluation period to the next.
NEED FOR WRITING ASSESSMENTTOOLS FOR BEGINNING WRITERS
The Developmental Writing Scale (DWS)described in this article was developed in re-sponse to the need for better tools to assessbeginning writing. Development of the DWSbegan in response to an immediate need onthe part of researchers to track the progressin writing quality of beginning writers withdevelopmental disabilities (DD) who wereparticipating in an investigation of EnrichedWriters’ Workshop intervention (see Sturm,2012). The research team needed a tool sensi-tive enough to detect small advances in writ-ing by students with various diagnoses involv-ing complex cognitive, linguistic, and neuro-motor impairments. Each of the students inthe target population was a beginning writer,and, subjectively, each student participatingin the early research appeared to have madegains in intervention; however, the availablewriting measures were not sensitive enoughto detect the changes. This illuminated thebroader need for a writing measure that couldbe used to capture differences in beginner-level writing samples for children with disabil-ities across grade levels as well as for typicallydeveloping children in the early grades. Sucha tool should be sensitive enough to measureadvancing emergent writing abilities, descrip-tive enough to inform intervention planning,
stable enough to measure progress reliably,and valid for addressing purposes related bothto educational and research concerns.
Despite an emphasis on improving writinginstruction for typically developing beginningwriters, students with disabilities continueto lag behind their peers on statewide andnational writing assessments. Data from theNational Assessment of Educational Progresswriting assessment showed nearly half (46%)of eighth-grade students with disabilities scor-ing below the basic level of proficiency(Salahu-Din, Persky, & Miller, 2008). Childrenwith the most severe and complex disabilitiesmay not even be represented in these nationaldata because they were not considered to bewriters. Tools that would be more sensitiveto features of early writing than the holisticwriting rubrics common to state writing as-sessments could help change this picture. Forstudents with complex disabilities, who oth-erwise might never receive a score of morethan 1 (below basic) out of 6 (proficient/exemplary) on current summative measuresthroughout their primary and secondary ex-perience, such tools could improve access toachieving components of core curricular stan-dards.
Quality writing instruction has been shownto improve the writing of students with dis-abilities (e.g., Graham & Harris, 2005; Joseph& Konrad, 2009), and ongoing progress mon-itoring plays a critical role in the develop-ment of quality writing instruction. Althoughcurrently available writing assessments mightprovide educators with sufficient informationfor summative assessments, they generally donot provide sufficient or appropriate informa-tion to guide the formative assessment pro-cess (Heritage, 2010).
An extensive review of the literature didnot reveal any existing measures that wouldbe sensitive to fine-grained differences in sam-ples produced by beginning and atypical writ-ers, with or without disabilities. Althoughseveral descriptions of beginning writingdevelopment were identified for componentskills (e.g., Clay, 2006; Sulzby, Barnhart, &Hieshima, 1989), comprehensive scales ofdevelopmental writing skills appropriate for
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 299
this purpose were not found. As Coker andRitchie (2010) concluded, “ . . . no measurecurrently exists to bridge single-letter writ-ing and spelling and beginning compositionabilities” (p. 178). Thus, plans were set in mo-tion to develop one. This is a report of theprocedures used to develop a DWS for use inassessing samples of students’ early writing at-tempts and the results of preliminary researchon the scale’s effectiveness.
WHO IS A BEGINNING WRITER?
For the purpose of this article, a beginningwriter is one who is learning to use writtenlanguage to express communicative intent,and beginning writing is defined as startingwith emergent writing (drawing, scribbling,and writing letters) and ending with conven-tional writing abilities, usually acquired bysecond or third grade for typically developingchildren. Sulzby and her colleagues (Sulzbyet al., 1989; Sulzby & Teale, 1991) defined be-ginning writers somewhat differently—as in-dividuals who are in the early stages of learn-ing to compose texts that can be read byother literate persons and that can be readconventionally by the writers themselves.This emphasis led Sulzby and her colleaguesto focus on stages in the development ofconventional spelling as characterizing earlywritten language development. Although in-telligible spelling clearly is a component ofemergent written language, the boundary be-tween emergent and beginning writing is notclear when spelling metrics are used alone.Consistent with the inclusion of drawing andscribbling in the scale of early writing de-velopment by Sulzby et al. (1989), begin-ning writing involves skills at language lev-els beyond emergent spelling. In this article,we consider students to be beginning writ-ers who have not yet acquired rudimentaryspelling but who have demonstrated otheremergent written communication skills. Thisis consistent with a more inclusive view ofcandidacy for early literacy instruction pro-posed by Kaderavek and Rabidoux (2004).
Children as young as 18 months have beennoted to make intentional marks on a page
(Tolchinsky, 2006). By 3 years of age, chil-dren without disabilities may engage in emer-gent writing activities that include drawing,scribbling, and writing letters. As childrencontinue to develop, they begin to differen-tiate between drawing and writing (Dyson,1985, 1986), to form letter shapes, and to de-velop concepts about print, such as linear-ity and directionality (Clay, 1975; Ferreiro &Teberosky, 1982). Children then use alpha-betic and syllabic principles to match lettersto sounds, first randomly, then using inventedspelling to represent initial and final sounds inwords (Sulzby et al., 1989; Tolchinsky, 2006).
During this period, most children begin tounderstand that print can be used to com-municate a message to an audience that isnot present (Clay, 1975; Scott, 2012; Sulzby& Teale, 1991). By the end of second grade,most beginning writers are becoming con-ventional writers who can compose wordsand sentences that are intelligible to a reader(Kress, 1982/1994; Tolchinsky, 2006). Theyare also beginning to produce cohesive, co-herent, elaborated texts consisting of mul-tiple sentences (Fitzgerald & Spiegel, 1986,1990; Halliday & Hasan, 1976; Langer, 1986;McCabe & Bliss, 2003; Newkirk, 1987, 1989;Peterson & McCabe, 1983). Definitions of thekey concepts that are associated with learningto write are presented in Table 1.
BEGINNING WRITERS WITHDISABILITIES
Among students with disabilities, the writ-ing abilities of students with learning dis-abilities (LD) have been investigated moreextensively than perhaps any other group.Research examining the writing productsof students with LD has shown that thesestudents demonstrate difficulty with hand-writing, spelling, vocabulary, complex sen-tence constructions, fluency, text struc-ture, cohesion, and coherence (Ehren, 1994;Graham, MacArthur, Schwartz, & Page-Voth,1992; MacArthur, Schwartz, & Graham, 1991;Newcomer & Barenbaum, 1991; Scott, 1989).
A reasonable body of research also has high-lighted the writing challenges and needs of
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
300 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
Table 1. Definitions of key terms and constructs
Construct Definition
Drawings A line drawing or photograph representing an event, object, person, or placeScribbles A wavy, circular, or continuous line that may, or may not, show directionalityLetter-like forms One or more forms representing or resembling printed or cursive alphabetic
lettersWords A group of letters written in a sequence set off by spaces; includes
intelligible words not separated by spaces and adjacent to random lettersor other intelligible words
Partially formedsentence
At least two words in proximity that appear to be related grammatically asparts of a sentence
Complete sentence A set of words organized grammatically with a subject and a verb(punctuation not required)
Organized Discourse that conveys temporal, causal, categorical, or other logicalrelationships that are consistent with the author’s apparent purpose inconveying information, narrating a story, making a persuasive argument,or some other emergent discourse form
Coherence A central main theme or topic maintained across multiple sentencesCohesion Intra- and intersentence language connections made by using cohesive
devices (e.g., pronoun or synonym replacement, logical connectors,conclusions that refer to prior content); one test of cohesion is thatsentences cannot be reordered without changing meaning
Note. From “Cohesion in English,” by M. Halliday and R. Hasan, 1976, London: Longman; Newkirk (1987); “Theachievement and antecedents of labeling,” by A. Ninio and J. Bruner, 1978, Journal of Child Language, 5(1), pp. 1-15.Copyright 2010 by Janet Sturm; and “Forms of Writing and Re-rereading From Writing: A Preliminary Report (TechnicalReport No. 20),” by E. Sulzby, J. Barnhart, and J. Hieshima, 1989, Berkeley, CA: National Center for the Study of Writingand Literacy. Retrieved November 15, 2010, from http://www.nwp.org/cs/public/print/resource/606From “Outcome Measures for Beginning Writers With Disabilities,” by J. M. Sturm, N. W. Nelson, M. Staskowski, and K.Cali, 2010, November, Philadelphia, PA: Miniseminar presented at the American Speech-Language-Hearing Convention;used with permission of the author.
students with severe speech impairments andphysical disabilities who use augmentativeand alternative communication (AAC) tech-niques and supports and who may be de-scribed as having complex communicationneeds (CCN). Slow writing rates (i.e., around1.5 words per minute) are reported for in-dividuals using AAC, which has a significantimpact on writing fluency and makes compos-ing extremely difficult (Koke & Neilson, 1987;Smith, Thurston, Light, Parnes, & O’Keefe,1989). Many students with CCN also demon-strate delays in phonology, spelling, vocabu-lary, syntax, and discourse knowledge that im-pact their writing development (Berninger &Gans, 1986; Harris, 1982; Nelson, 1992; Sturm& Clendon, 2004; Sturm, Erickson, & Yoder,
2003; Udwin & Yule, 1990; Van Balkom &Welle Donker-Gimbrere, 1996; Vandervelden& Siegel, 1999).
Less research is available on the writ-ing abilities and challenges of students withintellectual developmental disabilities (IDD)and social–communicative disabilities, suchas autism spectrum disorders (ASD). Disabil-ities such as IDD and ASD, individually or incombination, present risks to written commu-nication development on multiple levels (cog-nitive, communicative, and linguistic). Lin-guistically, students with IDD and ASD canpresent with a wide range of abilities (Prelock,2006). Some students with IDD and ASD areunable to produce spoken or written wordsat all, whereas others demonstrate relative
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 301
strengths in these areas. Students with IDDare at risk for demonstrating difficulties acrossfeatures of writing (e.g., spelling, vocabu-lary, syntax) (Sturm, Knack, & Hall, 2011).Many students with ASD present with finemotor limitations that impact text production(Broun, 2009), which can contribute to theproduction of texts that lack fluency and in-telligibility. Impairments in social interactionin students with IDD and ASD might also in-terfere with these students’ understandingsand production of communicative aspects ofwritten discourse, although limited researchis available in this area.
Students with severe and multiple disabili-ties risk remaining beginning writers for life.This risk has been compounded historicallyby a lack of serious effort and expectation onthe part of educators to teach such children towrite (Koppenhaver & Yoder, 1993). That pic-ture has been changing in recent years (e.g.,Bedrosian, Lasker, Speidel, & Politsch, 2003;Koppenhaver & Erickson, 2003), spurred on,in part, by policy-driven expectations for spe-cial educators to target achievement of ac-tual academic standards with their studentswith severe and complex disabilities. Thisheightens the need for an assessment tool thatcould quantify the indicators of small units ofprogress that characterize the early steps inlearning to communicate through writing.
CURRENT ASSESSMENT PRACTICESAND LIMITATIONS
As an initial step in the development ofthe DWS, existing writing measures werereviewed to identify the writing constructstargeted, the developmental range of themeasure, and the limitations. Espin, Weis-senburger, and Benson (2004) classified class-room writing assessments as holistic, primarytrait, and analytic scoring. They contrastedsuch “typical” assessments with curriculum-based measurement, which they indicated asbeing “developed with special education inmind” (p. 56).
Holistic scoring criteria are commonly usedto score samples produced for end-of-grade
writing assessments. Such assessments maybe found on Web sites describing statewideassessments (e.g., Massachusetts Departmentof Education, 2012; Tennessee Departmentof Education, 2012). They tend to focus onwhether or not students have acquired spe-cific writing traits during the course of aschool year, using separate rubrics for eachgrade level. Writers may be scored for levelsof proficiency on scales of 1–6, with scoresof 1, 2, or 3 denoting lack of proficiency andscores of 4, 5, and 6 denoting proficiency forthat grade level.
Primary trait and analytic scoring may alsobe used in end-of-grade state-level assess-ments. Especially at the lowest levels, meth-ods of this type may describe traits that areabsent, rather than present. An example is thedescription for kindergarten, Level 1 that in-dicates, “Writing/drawing shows little or nodevelopment of the topic” (Michigan Depart-ment of Education, 2000). Some rubrics forbeginning writers also seem to equate verydifferent constructs, which may not be in-terchangeable, such as writing and drawingwithin the early levels of the Michigan Liter-acy Progress Profile (Michigan Department ofEducation, 2000).
Primary trait and analytic scoring may alsobe used more appropriately for rating samplesproduced by children functioning beyond theearly writing stages. We were seeking appro-priate informal assessments of students’ orig-inal writing samples at the earliest develop-mental levels. As possible candidates, we iden-tified 2 types of measurement tools that mightbe suitable—developmental writing continuaand curriculum-based measurements.
Developmental writing continua
Developmental writing continua tend to fo-cus on students’ acquisition of positive traitsin their writing across the primary grades(e.g., Beginning Writer’s Continuum; North-west Regional Educational Laboratory, 2010;North Carolina K-2 Writing Continuum, NorthCarolina Department of Public Instruction,2009). Developmental rubrics within suchcontinua tend to follow Sulzby et al’s. (1989)
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
302 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
initial stages of writing development, fromearly emergent to independent/conventionalwriting and spelling. They describe howyoung students move from drawing to let-ter formation to words and sentences. Severaldevelopmental writing measures (e.g., the Be-ginning Writer’s Continuum) also measure thedevelopment of writing traits (e.g., organiza-tion, word choice, and conventions).
Some developmental continua includeevidence of student behaviors, such as,“pretends to read own writing,” and writingprocesses, such as, “reads own writing withfluency” (North Carolina Department of Pub-lic Instruction, 2009). This is helpful for somepurposes, but such measures cannot be usedfor scoring written products by themselves.
Developmental continua may also lacksufficient refinement to document minimalchanges in student writers. For example, pro-gressive levels of the North Carolina K-2 Writ-ing Continuum include descriptors that arenearly identical at the Emergent level, “writes1 or 2 sentences focused on a topic,” theEarly Developing level, “writes a few short,patterned, repetitive sentences focused on atopic,” and the Developing level, “writes sev-eral sentences about a topic.” Other prob-lems occur when descriptors are vague, suchas, “settles for the word or phrase that willdo” and “sections of writing have rhythm andflow” (Northwest Regional Educational Lab-oratory, 2010), making it difficult for teach-ers to score reliably. Developmental continuamay also be difficult to interpret because theyassign single scores to levels on the basis ofmultiple constructs across a range of poten-tially unrelated skills. One example is, “usesperiods correctly” along with “establishes a re-lationship between drawing and print” (NorthCarolina Department of Public Instruction,2009). In these instances, two writing sam-ples scored at the same level actually may havevery different attributes.
Curriculum-based measures of writtenexpression
Curriculum-based measures of written ex-pression have advantages for older students
with disabilities (Espin et al., 2004) and maybe useful for beginning writers as well, butthey also have limitations. Such measures aredesigned to monitor students’ progress fre-quently over time by administering timed,on-demand prompts of 3–5 min durationon a weekly or biweekly basis and usingcount data to document changes. Formatsfor assessing the writing of beginning writ-ers in kindergarten and first grade include (a)sentence copying, (b) picture-word prompt,(c) story prompt, and (d) sentence writ-ing (Coker & Ritchie, 2010; McMaster etal., 2011). Production-dependent quantitative(count) measures of writing fluency, suchas total words written (TWW), and correct-ness, such as words spelled correctly (WSC)and correct writing sequences (CWS), may bemore reliable measures of beginning writingsamples than production-independent mea-sures of accuracy (Jewell & Malecki, 2005);however, these quantitative measures oftenlack face validity with teachers (Gansle, Noell,VanDerHeyden, Naquin, & Slider, 2002) andjust adding more of something has limitedvalue for guiding instructional choices.
Curriculum-based measures of written ex-pression measures also are problematic forbeginning writers who cannot yet producemore than three intelligible words, as is com-mon at least through the middle of kinder-garten. Thus, count measures of TWW andwritten word accuracy may not be appropri-ate measures for kindergarten students (Coker& Ritchie, 2010) and other beginning writers.Similarly, timed, on-demand writing promptsmay not be appropriate for beginning writ-ers who are still mastering transcription andidea generation skills (Bereiter, 1980) and forstudents whose disabilities make it difficult toperform motor acts quickly.
In summary, the review of existing optionsmade it clear that a new tool, such as theDWS, was needed to achieve multiple goalsfor the target population. To be effective withchildren with more severe and complex dis-abilities, most of whom make slower gains inwriting over time, the new tool needed to ac-curately represent the fine-grained differences
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 303
in early development, spanning from emer-gent to early conventional writing. It alsoneeded to focus on abilities that are acquiredacross graduated levels of early writing devel-opment and be designed to capture the differ-ences in written products produced by chil-dren functioning at different levels within theearliest stages of beginning writing.
PURPOSES FOR THE DWS
To fill the unmet need, we conceptualizedthe DWS as a new tool that would be valid formeasuring the written language of studentsfunctioning at the earliest levels of writingdevelopment. Because validity must be mea-sured relative to the question, “Valid for whatpurpose?” we articulated five purposes for theDWS. It should (a) identify small differencesin beginning writing skills, (b) offer instruc-tionally relevant information about what totarget next, (c) serve as a functional outcomemeasure for periodic assessment probes andclassroom-produced writing artifacts, (d) beeasy for educators to learn and use reliably,and (e) quantify evidence of small but signifi-cant changes so that educators can celebrategrowth with students and their parents. Meth-ods of traditional test theory were used tobuild in construct and content validity dur-ing development and to evaluate the DWSwith regard to its reliability and validity for itsfive purposes. Research questions were posedabout (a) how to represent the constructs ofearly written communication development,(b) whether the new scale would compare fa-vorably with existing measures in terms of itsability to capture evidence of small advancesin early writing, (c) whether it would be pos-sible to use reliably, and (d) whether it wouldmeet teachers’ needs for a tool they could useto measure progress and guide instruction.
METHODS
Procedures for developingand evaluating the scale
Development of the DWS was conductedin four recursive steps aimed at meeting stan-
dards for educational and psychological test-ing established by the joint committee of theAmerican Educational Research Association(AERA), American Psychological Association(APA), and National Council on Measurementin Education (AERA/APA/NCME, 1999). Theywere to (a) clarify theoretical model of con-structs to be measured and the purposes ofscale, (b) generate items or scoring criteriaconsistent with the model and representativeof its content, (c) conduct recursive tryoutsand modifications until the measure could bescored reliably, and (d) evaluate whether themeasure could fulfill its stated purposes.
Step 1: Clarify theoretical constructsand purpose
A developmental progression provided thetheoretical framework for the DWS, usingconstructs described previously in the back-ground section of this article. Key constructsto be represented in the scale includeddrawing and scribbling, production of print,demonstration of alphabetic and syllabic prin-ciples, concepts of words with spaces, formu-lation of sentences, and production of cohe-sive and coherent texts. Definitions of thesekey constructs are provided in Table 1. Thisstep also involved generating the five pur-poses for the DWS.
Purpose 1
The first purpose was to distinguish varia-tions in beginning writing skills. As reviewedpreviously, a danger is that writing measuresat the emergent level may capture only whata child cannot do. If assessment tools as-sign a low score based on a student’s limitedskills (e.g., “insufficient to score” or “unde-veloped”), they may fail to capture emergingindicators of the child’s growing awarenessthat writing allows one to share and commu-nicate ideas through text (e.g., Scott, 2012).Under deficit-oriented systems, students withdisabilities who produce writing at beginninglevels of development may be viewed as “non-writers.” It is critical that educators and spe-cial service providers be able to measure andshow what a student can do with text.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
304 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
Traditional writing assessments also maynot have enough sensitivity to detect re-fined changes as a student moves from sin-gle letter writing to more conventional skilldevelopment. Thus, finer-grained descriptorswere added to existing developmental con-tinua consistent with preliminary samples ofwriting gathered from the target population.Samples were also drawn from typically de-veloping students. Every child is a beginningwriter in the early grades. The DWS is de-signed to be used with a beginning writerof any age, but it is grounded in the writingdevelopment of typically developing childrenbetween the ages of 3 and 7 years.
Purpose 2
The second purpose was to serve as a for-mative assessment measure that could sup-port teachers in identifying instructional goalsthat would help individual students move tothe next level in development. Traditionalmeasures might provide a quantitative mea-sure (e.g., TWW) for a particular product;but without indicators of what should comenext, teachers might lack clarity about whatshould be targeted next. The DWS is basedon quantitative additions to the written prod-uct as well as qualitative ones, such as move-ment from three unrelated words to threerelated words. An educator using this sys-tem then could choose instructional goalsdesigned to improve students’ writing qual-ity and progress to the next level, such as“The student will connect two to three wordsto convey sentence-like meanings while writ-ing.” Instruction to support achievement ofthat step could include having the child tell(orally or gesturally) something about a cho-sen writing topic first. Then, a shared pen-cil approach or keyboarding with scaffoldingcould be used to help the child to representthe connected ideas with intelligible words inwriting.
Purpose 3
The third purpose was to provide a meansof measuring either periodic probes of stu-dents’ independent writing abilities or natu-
rally occurring writing artifacts that studentscompose within their classroom writing ac-tivities. As a functional outcome measure, theDWS was intended to serve also as a forma-tive assessment tool to assign scores to writ-ing samples produced across time. This wouldsupport educators in profiling growth in thework of a student writer. The goal was to de-sign the DWS so that a student participating ina classroom writing program might progressacross its levels, such as moving in small stepsfrom single word writing to writing using mul-tiple sentences, over a time frame in which tra-ditional measures would reflect no growth atall. The measure was also designed to be usedfrequently, on a weekly, monthly, or quarterlybasis to monitor progress on existing writtensamples so that teachers would not have totake time out for “testing.”
Purpose 4
The fourth purpose was to provide educa-tors with a measure that is time efficient andeasy to use reliably. By designing the DWS tobe applicable to existing artifacts with a scor-ing system that educators could apply quickly,we hoped that time savings would make itpossible for teachers to examine many sam-ples for multiple students over time. We rea-soned that the instructional relevance of thetool would be enhanced if it were intuitive foreducators to learn and to apply quickly to dis-tinguish differences between writing samples.
Purpose 5
The final purpose of the DWS was to makeit possible to celebrate students’ positivechange as writers with them and their parents.This goal stemmed from observations that,far too often, beginning writers who strugglethroughout their school-age years with basicwriting skills may not be aware of the posi-tive gains they actually are making in learningto communicate through writing. The DWSwas developed as a simple ordinal scale sothat it could measure writing advances ashigher numbers. This would make it possibleto show students, through the use of graphsor tables, what they have accomplished with
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 305
writing. By plotting data across time, a sim-ple numerical scale could support students insetting personal goals and talking with oth-ers (e.g., educators or family members) abouttheir writing strengths and gains they havemade.
Step 2: Generate criteria for scoringcontent
For a norm-referenced test, items must begenerated that provide a representative sam-ple of content for each construct being as-sessed. For a criterion-referenced scale, suchas the DWS, the content is generated by stu-dents, who produce original writing samples,and the scoring criteria must be generated tocapture the features of the content. In ourwork, this step involved a combination of lit-erature review and consideration of existingempirical data in the form of students’ writtenlanguage samples. An extensive review of theliterature was conducted on the developmentof typically developing writers and a list ofinitial constructs (e.g., scribbling or randomletter patterns) produced by children duringtext generation was created. The list was usedto create a preliminary hierarchy of levels ofwriting quality. Because a core goal was to cre-ate a scale that ranged from emergent to con-ventional writing skill development, a bank of460 naturally occurring writing samples pro-duced by typically developing kindergartenand first grade students was used to validatethe accuracy of the tool levels and to anchoreach level on the DWS. In addition, samples ofstudents with LD between fourth and eighthgrades were reviewed to verify the levels onthe scale.
Step 3: Conduct recursive tryouts andmodify criteria as needed
The initial scoring criteria were revisedmultiple times after reviewing the scoring ofwriting samples that had been produced bystudents with DD. During this process, the re-search team revised descriptions of linguisticfeatures and skills linked to student writingsamples. This initial scale was then used bythe authors to code, as a group and then in-
dependently, a wide range of beginning writ-ing samples. The first author also assembleda cadre of undergraduate and graduate stu-dents to collect and code samples to allowcomparison of scores for samples producedsequentially.
In the process of development, samplescoded with evolving versions of the tool in-cluded more than 1500 samples produced bystudents with DD (ages 5–25 years) and morethan 200 samples of typically developing stu-dents. During this coding process, outlier sam-ples were identified, reviewed by the authorsas a group, and additional coding rules werecreated. If a particular sample was not cur-rently represented on the scale, a new levelwas created. If a sample could not be reliablycoded, we revised the descriptors for a givenlevel. The process of creating the DWS, thus,was recursive in nature, and refinements weremade to resolve difficulties in coming to initialagreement.
A substudy was also conducted on inter-rater reliability in which the DWS was used toscore 285 samples produced by 11 studentswith disabilities. All samples were scored ini-tially by one graduate student, a research as-sistant who was trained and experienced inthe use of the scale. Then, 20% of the sam-ples were rescored by a second graduate stu-dent, who was trained to use the scale aspart of a graduate course. Percent agreementand Cohen’s κ were calculated to determinethe level of agreement between scorers be-yond chance agreement (Cohen, 1960; Hayes& Hatch, 1999).
Step 4: Evaluate the tool’s validity for itsprimary purposes
Construct and content validity were par-tially ensured by the steps used to generatethe components of the tool. Evaluation of thetool’s validity for achieving its five purposesis ongoing. This includes a large-scale valida-tion study (Cali, manuscript in preparation),in which writing samples of typically develop-ing kindergarten and first-grade students arebeing used to test the sensitivity of the DWSto advances in children’s writing (Purposes
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
306 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
1–3). The first author is also using the scalewith children with various disabilities in anevaluation of an Enriched Writers’ Workshopapproach to understand better how it worksfor guiding instruction and documentingprogress (Purposes 2, 3, and 5) (see Sturm,2012).
Preliminary evaluation of the validity of theDWS for meeting its purposes was also con-ducted by comparing the DWS with existingdevelopmental scales from Clay (2006) andSulzby et al. (1989). A goal of DWS develop-ment was to build on existing tools and also toimprove on them by capturing developmentallinguistic changes in writing quality; thus, theconstructs measured should be related, butnot exactly the same. Unlike traditional eval-uation of concurrent validity, in which onelooks for commonalities between the newtool and existing tools, we also were look-ing for distinctions. To test further for distinc-tions, existing writing samples were codedusing the DWS and the two additional scales(i.e., Clay, 2006; Sulzby et al., 1989), and com-parisons were made between results.
Finally, a small pilot survey study was con-ducted to investigate the validity of the DWSfor its purpose of being easy for educatorsto learn and useful for them to apply. Us-ing a protocol approved by a Human Sub-jects Institutional Review Board, profession-als who had been helping to pilot test thescale were invited to participate in the survey.The 8-item examiner-created Likert-style ques-tionnaire used the choices, very much agree,agree, undecided, disagree, and very muchdisagree. Open spaces were provided follow-ing each item to allow for written comments.Item 8 on the questionnaire used a categori-cal response format (daily, weekly, biweekly,monthly, and quarterly) to obtain the esti-mated frequency of use of the DWS when ex-amining student writers.
RESULTS
The developmental writing scale
The process of recursive tryouts and mod-ifications resulted in the current version of
the DWS, with 14 writing levels and scor-ing criteria as outlined in Table 2. This tablepresents the ordinal 14-point developmentalscale, standardized scoring criteria, examples,and accommodations related to each level. Us-ing this scale, examiners assign level “scores”to samples of original written products pro-duced by beginning writers with or withoutdisabilities.
An important feature of the DWS is thatit is not genre specific. Thus, it can be usedto score and measure any genre (e.g., nar-rative or expository) composed by a begin-ning writer. These should be samples of orig-inal text production, not immediately influ-enced by teacher or clinician scaffolding. Dur-ing their production, students should be al-lowed access to any accommodations (e.g.,bins of pictures that might stimulate topic se-lection and/or access to a keyboard to com-pose text) while producing the samples tobe scored. Examples of how accommodationsmight be accounted for in scoring appear inTable 2.
Comparisons of the DWS with othertools
Several comparisons were made to illumi-nate similarities and differences with existingtools. Table 3 shows how scoring levels onthe DWS correspond to comparable languagelevels for writing by Clay (2006) and formsof writing by Sulzby et al. (1989), illustrat-ing gaps in the other measures that the DWScould fill. Copying was the only construct onone of the other measures not represented inthe DWS scale, and copying is not a constructthat is appropriate when the goal is to en-courage students to produce original writingsamples. This table also demonstrates that theDWS, which targets qualitative growth in lin-guistic development, aligns more closely withClay’s (2006) language levels for writing thanwith the early forms of writing described bySulzby et al., which emphasized spelling de-velopment.
Table 4 shows how the results would dif-fer if the three different systems were ap-plied to the same samples. It illustrates how
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 307
Tab
le2
.D
evel
op
men
talw
riti
ng
scal
efo
rb
egin
nin
gw
rite
rsa
Leve
lsSc
ori
ng
Cri
teri
aD
escr
ipti
on
Acc
om
mo
dat
ion
s
1D
raw
ing
Lin
esan
dcu
rves
that
app
ear
tore
pre
sen
to
bje
cts
Sele
ctio
no
fa
pic
ture
by
ach
ildw
ho
can
no
th
old
atr
adit
ion
alp
enci
lor
mar
ker
2Sc
rib
blin
gC
on
tin
uo
us
vert
ical
,cir
cula
r,o
rw
avy
lines
arra
nge
dlin
earl
yac
ross
the
pag
e,w
hic
hm
ayin
clu
de
lett
er-li
kefo
rms,
bu
tw
ith
the
maj
ori
tyo
fsh
apes
no
tre
cogn
izab
leas
lett
ers.
Ifa
child
use
sa
keyb
oar
d,t
his
leve
lwo
uld
no
tb
eu
sed
3Le
tter
stri
ngs
(no
gro
up
s)H
and
wri
tten
or
typ
edst
rin
gso
fle
tter
sb
ut
no
tgr
ou
ped
into
wo
rds.
Exam
ple
s:tt
tttt
ksh
pppn
s
Alp
hab
etd
isp
lay
(e.g
.,p
aper
cop
y)an
dst
and
ard
or
elec
tro
nic
keyb
oar
dac
cess
(e.g
.,o
nsc
reen
keyb
oar
do
rA
AC
syst
em)
4Le
tter
stri
ngs
gro
up
edin
wo
rds
Stri
ngs
of
lett
ers
gro
up
edin
to“w
ord
s”(i
.e.,
wit
hsp
aces
bet
wee
nat
leas
ttw
ogr
ou
ps
of
lett
ers)
bu
tw
ith
no
inte
lligi
ble
wo
rds.
Exam
ple
:iL
CR
6a
iLK
VK
CC
PR
SBW
RK
eB
RK
e
Alp
hab
etd
isp
lay
(e.g
.,p
aper
cop
y)an
dst
and
ard
or
elec
tro
nic
keyb
oar
dac
cess
(e.g
.,o
nsc
reen
keyb
oar
do
rA
AC
syst
em)
5O
ne
inte
lligi
ble
wo
rdSt
rin
gso
fle
tter
sgr
ou
ped
into
“wo
rds,
”w
ith
on
lyo
ne
po
ssib
lere
alw
ord
(i.e
.,tw
oo
rm
ore
lett
ers
inle
ngt
h)
set
apar
t,w
ritt
enre
pea
ted
ly(e
.g.,
do
g,d
og,
do
g),o
rem
bed
ded
ina
stri
ng
of
lett
ers.
Exam
ple
:IM
PlC
OTh
eC(I
am
pla
yin
gou
tsid
eon
the
swin
g.)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
6T
wo
toth
ree
inte
lligi
ble
wo
rds
Tw
oo
rth
ree
dif
fere
nt
inte
lligi
ble
wo
rds
emb
edd
edin
stri
ngs
,sep
arat
edb
ysp
aces
,or
ina
list
form
at.S
ingl
ele
tter
wo
rds
such
as“I
”an
d“a
”m
ust
be
sep
arat
edb
ysp
aces
toco
un
tas
anin
telli
gib
lew
ord
.Ex
amp
le:
IYTK
TO
SMN
TH
ETR
(Ili
ke
tosw
imu
nder
the
wa
ter.
)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
7T
hre
eo
rm
ore
dif
fere
nt
inte
lligi
ble
wo
rds
ina
list
Th
ree
or
mo
rere
late
dw
ord
s.Ex
amp
le:
Lion
sD
etro
itfo
otb
all
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e (con
tin
ues
)
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
308 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
Tab
le2
.D
evel
op
men
talw
riti
ng
scal
efo
rb
egin
nin
gw
rite
rsa
(Con
tin
ued
)
Leve
lsSc
ori
ng
Cri
teri
aD
escr
ipti
on
Acc
om
mo
dat
ion
s
8P
arti
alse
nte
nce
of
mo
reth
anth
ree
wo
rds
Mo
reth
anth
ree
dif
fere
nt
inte
lligi
ble
wo
rds,
wit
hat
leas
ttw
oo
fth
emin
ap
arti
ally
form
edse
nte
nce
(i.e
.,gr
amm
atic
ally
rela
ted
par
tso
fa
ph
rase
,cl
ause
,or
sen
ten
ce).
Exam
ple
:M
YD
AD
DY
WA
SLIE
GA
GA
RIL
A(M
yda
ddy
wa
sli
ke
ago
rilla
.)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
9O
ne
totw
oco
mp
lete
sen
ten
ces
Sen
ten
ces
hav
ea
sub
ject
ph
rase
and
ave
rbp
hra
se.E
nd
pu
nct
uat
ion
isn
ot
nec
essa
ry.E
xam
ple
:I
am
hpe
Ea
ster
ish
ere.
Icw
the
Ea
ster
bn
ny.
(Ia
mh
appy
Ea
ster
ish
ere.
Isa
wth
eE
ast
erbu
nn
y.)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
10T
hre
eo
rm
ore
un
rela
ted
sen
ten
ces
(nei
ther
coh
eren
tn
or
coh
esiv
e)
Sen
ten
ces
hav
en
oco
her
ent
top
ic(i
.e.,
sen
ten
ces
are
no
tre
late
d)
Ipla
ya
gam
e.I
wen
tto
my
fnid
hou
se.I
wen
tto
get
aeg
gto
eat.
Iw
ent
toch
an
shon
Sun
da
y.I
kis
sm
ym
om
ersu
nda
y.I
can
wa
lkm
ydog.
Isa
tin
my
hou
se.I
wen
tto
the
sain
gin
rin
g.
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
11T
hre
eo
rm
ore
rela
ted
sen
ten
ces
(co
her
ent
bu
tlim
ited
coh
esiv
e)
Org
aniz
edw
riti
ng
wit
hth
ree
or
mo
rese
nte
nce
so
na
coh
eren
tto
pic
bu
tw
ith
limit
edco
hes
ion
bet
wee
nse
nte
nce
s(i
.e.,
sen
ten
ces
can
be
reo
rder
edw
ith
ou
tch
angi
ng
mea
nin
g).E
xam
ple
:Fr
ogs
are
eggs
.Fro
ga
reco
ol.
In
oh
ow
afr
og
grow
seg
gth
engr
ow
mory
.Fro
gea
tlo
tof
thin
gsth
at
we
don
’tea
tli
ke
bu
gs.I
wa
nt
afr
og
topla
yw
ith
.Ith
ak
frogs
are
mu
mlo
ssbec
au
seth
ae
swim
.
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
12T
hre
eo
rm
ore
rela
ted
sen
ten
ces
that
can
no
tb
ere
ord
ered
(co
her
ent
and
coh
esiv
e)
Org
aniz
edw
riti
ng
wit
ha
coh
eren
tto
pic
(i.e
.,o
na
con
sist
ent
them
e)an
du
seo
fco
hes
ive
dev
ices
(e.g
.,p
ron
ou
no
rsy
no
nym
rep
lace
men
t,lo
gica
lco
nn
ecto
rs,s
ub
ord
inat
ing
con
jun
ctio
ns,
con
clu
sio
ns
that
refe
rto
pri
or
con
ten
t)ac
ross
thre
eo
rm
ore
sen
ten
ces
soth
atse
nte
nce
sca
nn
ot
be
reo
rder
edw
ith
ou
tch
angi
ng
mea
nin
g(s
eeSu
pp
lem
enta
lDig
ital
Co
nte
nt
[ava
ilab
leat
htt
p:/
/lin
ks.lw
w.c
om
/TLD
/A10
]A
pp
end
ixA
[ava
ilab
leat
htt
p:/
/lin
ks.lw
w.c
om
/TLD
/A10
]fo
rex
amp
les)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
13T
wo
coh
eren
tp
arag
rap
hs
of
atle
ast
thre
eco
hes
ive
sen
ten
ces
each
Org
aniz
edw
riti
ng
wit
ha
coh
eren
tm
ain
top
ican
dtw
oco
hes
ive
sub
sect
ion
s(s
ub
top
ics
or
sto
ryp
arts
),w
ith
atle
ast
two
sen
ten
ces
elab
ora
tin
gth
em
ean
ing
of
each
(see
Sup
ple
men
talD
igit
alC
on
ten
tA
pp
end
ixA
[ava
ilab
leat
htt
p:/
/lin
ks.lw
w.c
om
/TLD
/A10
]fo
rex
amp
les)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
14T
hre
eo
rm
ore
coh
eren
tp
arag
rap
hs
of
atle
ast
thre
eco
hes
ive
sen
ten
ces
each
Org
aniz
edw
riti
ng
wit
ha
coh
eren
tm
ain
top
ican
dat
leas
tth
ree
coh
esiv
esu
bse
ctio
ns
(su
bto
pic
so
rst
ory
par
ts),
wit
hat
leas
ttw
ose
nte
nce
sel
abo
rati
ng
the
mea
nin
go
fea
ch(s
eeSu
pp
lem
enta
lDig
ital
Co
nte
nt
Ap
pen
dix
A[a
vaila
ble
ath
ttp
://l
inks
.lww
.co
m/T
LD/A
10]
for
exam
ple
s)
Wo
rdb
ank
or
wo
rdp
red
icti
on
soft
war
e
Note
.AA
C=
augm
enta
tive
and
alte
rnat
ive
com
mu
nic
atio
n.
a Defi
nit
ion
so
fke
yte
rms
are
pro
vid
edin
Tab
le1.
Ifd
ebat
ing
bet
wee
ntw
ole
vels
,ass
ign
the
low
erle
vel.
Fro
m“O
utc
om
eM
easu
res
for
Beg
inn
ing
Wri
ters
Wit
hD
isab
iliti
es,”
by
J.M
.Stu
rm,N
.W.N
elso
n,M
.Sta
sko
wsk
i,an
dK
.Cal
i,20
10,N
ove
mb
er,P
hila
del
ph
ia,P
A:M
inis
emin
arp
rese
nte
dat
the
Am
eric
anSp
eech
-Lan
guag
e-H
eari
ng
Co
nve
nti
on
;R
evis
ion
sco
pyr
igh
t20
12b
yJ.
Stu
rm,
K.
Cal
i,N
.N
elso
n,
&M
.St
asko
wsk
iu
sed
wit
hp
erm
issi
on
of
the
auth
ors
.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 309
Table 3. Scoring correspondence for the developmental writing scale, with language levels forwriting by Clay (2006), and forms of writing by Sulzby et al. (1989)
DWS Level Clay’s Language Level Sulzby’s Forms of Writing
1Drawing
– 1Drawing
2Scribbling
– 2Scribble—wavy3Scribble—letter-like
3Letter strings (no groups)
1Alphabetical (letters only)
4Letter-like Units5Letters—random
4Letters grouped in “words”
with spaces
– 6Letters—patterns
7Letters—name-like elements
– – 8Copying
5One intelligible word
2Word (any recognizable word)
9Invented spelling—syllabic
6Two to three intelligible words
– 10Invented
spelling—intermediate7More than three intelligible
words in a list
– 11Invented spelling—full
12a
Conventional spelling8Partial sentence
3Word group (any two-word phrase)
–
9One to two sentences
4Sentence (any simple sentence)
–
10Three or more sentences (not
coherent)
5Punctuated story (of two or more
sentences)
–
11Three or more sentences
(coherent but limitedcohesive)
– –
12Three or more sentences in
one paragraph (coherent +cohesive)
– –
13Two paragraphs (coherent +
cohesive)
6Paragraphed story (two themes)
–
14Three paragraphs (coherent +
cohesive)
– –
Note. DWS = Developmental Writing Scale.aSulzby et al. (1989) also included a Level 13, described as “other,” which is not represented in this table.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
310 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
Tab
le4
.Sa
mp
les
pro
du
ced
by
beg
inn
ing
wri
ters
,co
ded
for
wri
tin
gd
evel
op
men
tb
yfo
ur
met
ho
ds
En
d-o
f-G
rad
eLa
ngu
age
Leve
lsF
orm
so
fW
riti
ng
Wri
tin
gSa
mp
lea
Tes
tSc
ore
(Cla
y,2
00
6)
(Su
lzb
yet
al.,
19
89
)D
WS
AT
CA
TD
OG
–Le
vel2
(rec
ogn
izab
lew
ord
)Le
vel1
2(c
on
ven
tio
nal
spel
ling)
Leve
l6(t
wo
toth
ree
inte
lligi
ble
wo
rds)
FoR
stIl
lTaL
eY
ou
weN
ItA
PP
AN
DSh
eW
asA
LIIT
Lb
Ab
YSo
ShE
ASC
TH
AR
Mo
thR
–Le
vel5
(tw
oo
rm
ore
sen
ten
cest
ory
)Le
vel1
1(i
nve
nte
dsp
ellin
g)Le
vel1
2(m
ore
than
thre
ese
nte
nce
sin
ap
arag
rap
h;
coh
eren
t+
coh
esiv
e)
BU
tHeR
MO
tHR
Sad
She
was
To
LIIT
LN
OW
IllT
eLY
ou
wo
tM
AD
EH
OR
wan
tto
DO
ItB
eeco
sesh
eH
AD
AB
IGB
RO
tHR
AN
DH
eC
OD
RID
EH
ISA
ND
that
’sA
LL(c
on
tin
ues
)
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 311
Tab
le4
.Sa
mp
les
pro
du
ced
by
beg
inn
ing
wri
ters
,co
ded
for
wri
tin
gd
evel
op
men
tb
yfo
ur
met
ho
ds
(Con
tin
ued
)
En
d-o
f-G
rad
eLa
ngu
age
Leve
lsF
orm
so
fW
riti
ng
Wri
tin
gSa
mp
lea
Tes
tSc
ore
(Cla
y,2
00
6)
(Su
lzb
yet
al.,
19
89
)D
WS
SRO
DU
FD
EFP
OT
1.H
irI
wh
sse
ntn
din
rin
d.S
RO
DU
FD
EFP
OT
1.Sc
ore
1Le
vel1
(let
ters
on
ly)
Leve
l5(r
and
om
lett
ers)
Leve
l4(l
ette
rsgr
ou
ped
inw
ord
s)M
yro
lem
od
elis
my
dad
bec
ause
he
wal
edo
nto
ols
,m
osh
ens,
and
mat
elb
aero
ls.I
thin
kit
’sve
rey
intu
ora
stin
gto
do
bec
ause
Ilik
eto
fix
thin
g’s,
and
it’s
fun
tod
ow
aled
ing.
Sco
re1
Leve
l5(t
wo
or
mo
rese
nte
nce
sto
ry)
Leve
l11
or
12(i
nve
nte
do
rco
nve
nti
on
alsp
ellin
g)
Leve
l11
(3+
sen
ten
ces—
coh
eren
tb
ut
limit
edco
hes
ion
)P
ow
Po
wP
ow
,Ith
ink
itw
ou
ldb
ea
goo
did
eafo
rte
ach
ers
toh
ave
agu
np
erm
it.I
tw
ou
ldre
du
cevi
ole
nce
insc
ho
ols
and
ou
tsid
eo
fsc
ho
ols
.It
will
also
pro
tect
them
selv
esas
wel
las
the
stu
den
ts.I
thin
kit
wo
uld
be
agr
eat
idea
for
it.
Sco
re1
Leve
l5(t
wo
or
mo
rese
nte
nce
sto
ry)
Leve
l12
(co
nve
nti
on
alsp
ellin
g)
Leve
l12
(mo
reth
anth
ree
sen
ten
ces
ina
par
agra
ph
;co
her
ent+
coh
esiv
e)
DW
S=
Dev
elo
pm
enta
lWri
tin
gSc
ale.
a Th
efi
rst
two
sam
ple
sin
this
tab
lew
ere
use
db
ySu
lzb
yet
al.(
1989
);th
efi
nal
thre
ear
efr
om
the
Ten
nes
see
Dep
artm
ent
of
Edu
cati
on
(201
0)an
dar
eu
sed
,wit
hp
erm
issi
on
,to
intr
od
uce
this
arti
cle.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
312 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
the linguistic and communicative constructsthat underpin the DWS categories differ fromthe spelling accuracy of the forms of writingdescribed by Sulzby et al. (1989) as movingfrom drawing and scribbling to conventionalspelling. The DWS, in contrast, is meaning-based, focusing on linguistic development atthe letter, word, sentence, and paragraphlevel. This difference in focus is revealed inscoring of the two writing samples drawnfrom Sulzby et al. that are shown in this table.
Table 5 shows how the results would differfor three of these samples using the analyti-cal scoring method of TWW compared withusing the 14 levels of the DWS. Using totalwords, Sample A, which has the least num-ber of words (24), would appear to be theleast sophisticated of the three writing sam-ples. However, using the DWS, Sample A isassessed as the most sophisticated becauseit demonstrates organized writing on a topicthat is both coherent and cohesive. In con-trast, Sample B, with 34 words, is evaluatedas the least sophisticated sample within thisset because the sentences are not related, giv-ing it a rating of Level 10. Sample C, on theother hand, is more cohesive than Sample Bbecause each sentence is about the sametopic, earning a Level 11 score. Sample A israted as being more coherent than Sample Cbecause it has a clear beginning, middle, andend, and its sentences cannot be rearranged
without changing the meaning of the passage;therefore, it is rated as a Level 12.
The comparisons in Table 5 also show thatDWS ratings could provide specific informa-tion on a student’s current developmentallevel and direction for what instruction shouldtarget to help the student move to the nextlevel. This supports the validity of the toolfor meeting Purposes 2, which is to providea formative assessment measure that is in-structionally relevant, and 3, which is toprovide a means of measuring either peri-odic probes of student’s independent writ-ing abilities or naturally occurring writingartifacts that students compose within theirclassroom writing activities. Related to Pur-pose 2, for example, the next goal for studentA would be to target writing an organized, co-herent topic containing two cohesive subsec-tions. Work with student B could target writ-ing three or more topically related sentencesthat are organized and coherent. Student Cwould need a goal aimed at producing orga-nized writing on a topic that is both coherentand cohesive.
Scoring reliability and specializedscoring rules
The results of the substudy of inter-raterreliability based on independent scoring bytwo trained graduate assistants showed a per-centage agreement of 91%. The correlation
Table 5. Writing samples comparing total words to the DWS level
Writing Total Words DWSSample Text Written Level
A On monday my frid came over my house. We played and we had fun.She lath. She what houm I clin up my mast.
24 12
B Happy Birthday Matthew. I like chocolate please Mom. I have a newschool. Am 14. A new pet is a puppies and a dog and a cat and ashirt and a new baby.
34 10
C I love to watch the garbageman to pick up our trash can to. I don’twatch the garman out to my window to. I love to watch therecycling person to get my recycling from my house to
38 11
Note. DWS = Developmental Writing Scale.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 313
between scores using Cohen’s κ (Cohen,1960; Hayes & Hatch, 1999) also was strong,with κ = .898 (p < .001).
The procedures for developing the DWSalso revealed some areas in which challengingsituations called for special scoring rules. Ingeneral, the score assigned to the sample isthe one that best fits the description at aparticular level. We found that scoring agree-ment could be improved if a scorer debat-ing between two levels assigned the lowerlevel being considered. Another rule estab-lished through this process was that the scorershould focus on the nature of the student’swriting (or prewriting) and not the spatialplacement of text on a page (e.g., paragraphspacing, indentation, or margins). In addition,we found it helpful to remind ourselves thatthe concepts of word, sentence, and para-graph represented in this scale are meant tobe primarily linguistic in nature. One shouldlook beyond technical accuracy when assign-ing scores. For example, if a student producesone large paragraph, examination may revealthat three cohesive and coherent subsectionsare present and a Level 14 is the best score.Another student might have a true word (e.g.,the) embedded within random letters. Thisstudent would be assigned a Level 5. If thesame word is repeated in a list format (e.g.,dog, dog, dog) the student also would be as-signed a Level 5. Student names at the topof the page (denoting who wrote it) are not
counted; however, student names in the bodyof the text are scored on the scale.
Another challenging scoring element re-lates to judgments of word intelligibility. DWSscoring allows examiners to use graphic con-tent, such as hand-drawn pictures or picturesselected from a picture bank, to support “read-ing” of the student’s text. Figure 1 providesan example in which the picture in Sample1 makes it possible to detect the words, “to”and “field trip,” and the picture in Samples 2makes it evident that the student was writing“I’m playing basketball.” A caution is that ex-aminers should use graphic content only (i.e.,context embedded in the work to communi-cate with an absent audience) to aid in inter-preting children’s text. To the extent possible,they should avoid being influenced by addi-tional context provided orally by the studentfrom the author’s chair or in face-to-face com-munication about the work because such con-text would not be available to an absent audi-ence. The rationale is that the scoring shouldbe based on the messages that can be gleanedby a remote audience assessing the written ar-tifacts the student has produced only, and notoral or gestural communication.
Evidence from the pilot survey for DWSease, efficiency, and utility
The brief pilot survey on the ease and util-ity of the DWS was completed by two teach-ers and two speech–language pathologists
Figure 1. Examples of students’ writing in which graphic illustrations contribute to the intelligibility ofstudents’ written content.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
314 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
(SLP). All four participants had been using theDWS to examine writing samples producedby students with DD and CCN across a 1-yearperiod. Results of this small pilot study aresummarized in Table 6, which also providesexcerpts from qualitative comments writtenin the open-ended sections of the survey.Overall, the teachers’ and speech–languagepathologists’ reported perceptions of theDWS indicated that they found it easy, effi-cient, and useful when used to examine thewriting outcomes of students with DD.
DISCUSSION
The DWS described in this article was de-signed as a comprehensive measure of qualita-tive change in beginning writers that can cap-ture refined changes in growth over time. Theconstruct and content validity of the tool aresupported by the recursive development pro-cess with foundations in existing literature onearly writing development (e.g., Sulzby et al.,1989), modified on the basis of empiricalevidence drawn both from young typically de-veloping students and students with severeand complex disabilities of a wide range ofages. These results provide preliminary evi-dence for the summative and formative usesof this assessment tool for quantifying devel-opmental advances in the beginning writingof students with and without disabilities.
Educational implications for beginningwriters
Currently, measures for monitoring theprogress of typically developing beginningwriters tend to be either literacy profiles thatmeasure multiple constructs of both read-ing and writing or curriculum-based measuresthat focus on quantitative measures of writingfluency (e.g., TWW). One advantage of theDWS is that it provides teachers with specificinformation about students’ conceptual un-derstandings of written language that cannotbe determined by measures of TWW or otherquantitative measures of writing progress.This is consistent with Purposes 1 (to distin-guish variations in beginning writing skills)
and 5 (to make it possible to celebrate stu-dents’ positive change as writers) summa-rized previously.
In summary, as the previously mentionedexamples illustrate, the DWS can be used todistinguish variations in the writing quality ofbeginning writers that are instructionally rel-evant. Results of pilot research on the ease,efficiency, and utility of the DWS, based onthe small survey, are also promising, suggest-ing that teachers may find it relatively easy touse the DWS for periodic probes of studentprogress. Displaying DWS outcomes in tablesfor students, educational teams, and familiesprovides a way to celebrate student writers.An anecdotal example of this naturally oc-curred when a student excitedly shared hisoutcome book with his parents as part of a“Meet the Author” event in an Enriched Writ-ers’ Workshop taught by the first author (seeSturm, 2012). The student’s outcome bookcontained tables of his writing progress andhis writing for the year. The student lookedat each table and showed them to his parents.During this sharing moment clinicians wereable to use the DWS to talk easily with parentsabout the changes their child had made andto show how the DWS levels were reflectedin the writing samples.
Limitations and future directions
A limitation of the research reported in thisarticle was sample size, particularly for theinitial survey of only four participants. Pre-liminary evidence regarding teacher and SLPperceptions about ease, efficiency, and utilityof the DWS was reported for only four partic-ipants. These participants work closely withthe first author; therefore, an additional limita-tion is response bias based on social desirabil-ity. Because the participant responses couldnot be kept completely anonymous, their re-sponses may have been influenced by whatthey thought the researchers wanted to hear.Future research could examine perceptionsof the DWS with a larger group of general ed-ucation teachers, special education teachers,SLPs, and university students.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 315
Table 6. Results of pilot survey on the ease, efficiency, and utility of the DWS
ParticipantSurvey Responses ParticipantItem (N = 4) Comments
1. The DWS iseasy to use
Very muchagree = 3
Clear instructions and very detailed, helpful examples wereprovided.
Agree = 1 The description of each benchmark (level) is clear.The DWS is easy and straightforward. Best of all, it allows me
to plot my students no matter what tools and supports mystudents use to write.
2. The DWS isfast.
Very muchagree = 4
Rating the samples goes fast. So far my samples are Levels 1–5.Most writing samples take less than 2 min to assign a writing
level.3. The DWS is
useful.Very much
agree = 4The DWS is useful in many ways—it helps to see, “at a glance,”
the skills of a student writer and to consider the next levelthat the student is working toward.
The DWS is very helpful when working with writing acrossthe curriculum—all instruction is centered on developingwriting with the scale benchmarks (levels) as goals. TheDWS is really useful as a point of reference when teamteaching or when working with educators teaching othersubjects.
It is extremely useful to have a scale that reflects writingdevelopment across learners.
4. The DWS iseasy to learn.
Very muchagree = 4
The examples are so helpful when learning the tool!Learning the DWS required a bit of training. The examples are
really helpful.Very easy and straightforward.
5. The DWSwill help mewith mywritinginstruction.
Very muchagree = 4
I’ve written IEP goals with the DWS as the measurement tool(progress indicator).
It has helped center instruction for our educational team. Veryclear for all members of the team and I am hoping it will beeasily transferred to new teams as students transition to newclassrooms.
It helps me to know where my students are in their writingabilities and where I should help them go.
6. The DWSwill make myjob easier.
Very muchagree = 3
The DWS is helpful in explaining writing development toparents and IEP teams
Agree = 1 It is very useful in addressing student writing goals.7. I will use the
DWS again.Very much
agree = 4I will continue to use the DWS to inform student written
language assessment.8. How often
would youuse the DWSto measureyourstudents’writing?
Quarterly = 1Monthly = 1Biweekly and
monthly = 1Monthly and
quarterly = 1
I use it quarterly at trimester, progress reporting times. I couldincrease the frequency.
I’d use it at least monthly, perhaps also during team meetings,or parent conferences during the year.
Biweekly and monthly—it depends, based on the level ofwriter. I use it more often as a student’s writing progresses.
I work with preschool-age children, so, depending on thechild, I may use it monthly. I would also use it quarterly, so Ican look at several samples at one time.
Note. DWS = Developmental Writing Scale.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
316 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
Although evidence of reliability usingCohen’s κ was reported for analysis of 285samples, future research should examine scor-ing reliability using a greater number of writ-ing samples from both developmentally dis-abled and typically developing students. Itshould also examine scoring reliability forteachers who have received minimal training.Such research should also examine the lengthof training time and type of training neededto use the DWS with greater reliability. In ad-dition, reliability studies should address reli-ability issues related to scoring words withquestionable intelligibility. For example, thesample “IKTO the BC” has been scored as a6 and a 3 by two different individuals (theauthors would score it as a 6).
Currently, research is being conducted tovalidate the DWS with typically develop-ing kindergarten and first-grade writers (Cali,manuscript in preparation). Future researchshould be conducted to validate the levels ofthe DWS for beginning writers with disabil-ities as they change longitudinally. Natural-istic samples, from both groups of students,could be examined to further validate eachlevel of the scale and to understand the rangeof writing levels of students in beginning gen-eral education classrooms. To further validatethe DWS, teachers and SLP could be askedto rank order sets of samples to confirm theoverall developmental sequence of the tool.
CONCLUSIONS
This article began with three writing sam-ples that all scored a “1” on a state assess-ment, highlighting the multiple limitations of
the scoring system. In contrast, the DWS,as a formative and summative assessment,provides refined developmental levels thatallow teachers to measure small amounts ofprogress across grade levels with disabled andnondisabled writers. This was illustrated inTable 4, in which the same samples were as-signed DWS levels of 4, 11, and 12. As a for-mative assessment, the DWS provides teach-ers with information about student’s writingability within a natural context. Teachers canuse this information to plan instruction thatsupports the student in moving to the nextlevel. The DWS is grounded in research onthe development of beginning writers and isfocused on growth in linguistic quality. There-fore, the face validity for this measure is en-hanced by making it intuitive and easy for edu-cators to use and implement. As the CommonCore State Standards (National Governors As-sociation Center for Best Practices, Council ofChief State School Officers, 2010) are imple-mented, there is an expectation that writinginstruction will be differentiated for learnerswhile also expecting high levels of teacheraccountability for student progress. As a sum-mative measure, the sensitivity of the DWSmeasure will support educators in achievingthat goal. The DWS is not only for the edu-cator; it also allows students to see positivegrowth in their own writing and can be usedto celebrate their writing gains and foster in-trinsic motivation to write. The preliminarydata presented here suggest that the DWSmay be a powerful comprehensive measureof qualitative change in beginning writers andwill be useful to enhance both assessment andinstruction.
REFERENCES
American Educational Research Association, AmericanPsychological Association, & National Council on Mea-surement in Education (AERA/APA/NCME). (1999).Standards for educational and psychological test-ing. Washington, DC: American Educational ResearchAssociation.
Bedrosian, J., Lasker, J., Speidel, K., & Politsch, A. (2003).Enhancing literacy in individuals with autism and
severe communication impairments. Topics in Lan-guage Disorders, 23(4), 305–324.
Bereiter, C. (1980). Development in writing. InW. Gregg,& E. R. Steinberg (Eds.), Cognitive processes in writ-ing (pp. 73–93). Hillsdale, NJ: Lawrence Erlbaum As-sociates.
Berninger, V., & Gans, B. (1986). Language profiles innonspeaking individuals of normal intelligence with
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
The Developmental Writing Scale 317
severe cerebral palsy. Augmentative and AlternativeCommunication, 2, 45–50.
Broun, L. (2009). Taking the pencil out of the process.Teaching Exceptional Children, 42, 14–21.
Cali, K. S. (manuscript in preparation). The validity andreliability of the Developmental Writing Scale forbeginning writers.
Clay, M. (1975). What did I write? Exeter, NH: Heine-mann.
Clay, M. (2006). An observation survey of early literacyachievement: Revised second edition. Portsmouth,NH: Heinemann.
Coker, D. L., & Ritchey, K. D. (2010). Curriculum-basedmeasurement of writing in kindergarten and firstgrade: An investigation of production and qualitativescores. Exceptional Children, 76, 175–193.
Cohen, J. (1960). A coefficient of agreement for nomi-nal scales. Educational and Psychological Measure-ment, 20, 37–46.
Dyson, A. H. (1985). Individual differences in emerg-ing writing. InM. Farr (Ed.), Advances in writing re-search: Children’s early writing development (pp.59–126). Norwood, NJ: Ablex.
Dyson, A. H. (1986). Transitions and tensions: Interrela-tionships between the drawing, talking, and dictatingof young children. Research in the Teaching of En-glish, 20, 379–409.
Ehren, B. J. (1994). New directions for meeting the needsof adolescents with language learning disabilities. InG.P. Wallach, & K. G. Butler (Eds.), Language learningdisabilities in school age children and adolescents(pp. 393–417). New York: Merrill.
Espin, C. A., Weissenburger, J. W., & Benson, B. J. (2004).Assessing the writing performance of students in spe-cial education. Exceptionality, 12(1), 55–67.
Ferreiro, E., & Teberosky, A. (1982). Literacy beforeschooling. Portsmouth, NH: Heinemann.
Fitzgerald, J., & Spiegel, D. L. (1986). Textual cohesionand coherence in children’s writing. Research in theTeaching of English, 20, 263–280.
Fitzgerald, J., & Spiegel, D. L. (1990). Textual cohe-sion and coherence in children’s writing. Revis-ited. Research in the Teaching of English, 24,48–66.
Gansle, K. A., Noell, G. H., VanDerHeyden, A. M., Naquin,G. A., & Slider, N. J. (2002). Moving beyond totalwords written: The reliability, criterion validity, andtime cost of alternative measures for curriculum-basedmeasurement in writing. School Psychology Review,31(4), 477–498.
Graham, S., & Harris, K. (2005). Writing better: Effectivestrategies for teaching students with learning diffi-culties. Baltimore, MD: Brookes Publishing Co., Inc.
Graham, S., MacArthur, C., Schwartz, S., & Page-Voth, V.(1992). Improving the compositions of students withlearning disabilities using a strategy involving productand process goal setting. Exceptional Children, 58,322–334.
Halliday, M., & Hasan, R. (1976). Cohesion in English.London: Longman.
Harris, D. (1982). Communicative interaction processesinvolving nonvocal physically handicapped children.Topics in Language Disorders, 22, 21–37.
Hayes, J. R., & Hatch, J. (1999). Issues in measuring re-liability: Correlation versus percentage of agreement.Written Communication, 16, 354–367.
Heritage, M. (2010). Formative assessment and next-generation assessment systems: Are we losing an op-portunity? Washington, DC: Chief Council of StateSchool Officers.
Jewell, J., & Malecki, C. K. (2005). The utility of CBM writ-ten language indices: An investigation of production-dependent, production-independent, and accurate-production scores. School Psychology Review, 34(1),27–44.
Joseph, L. M., & Konrad, M. (2009). Teaching stu-dents with intellectual or developmental disabilitiesto write: a review of the literature. Research in Devel-opmental Disabilities, 30(1), 1–19.
Kaderavek, J., & Rabidoux, P. (2004). Interactive to inde-pendent literacy: A model for designing literacy goalsfor children with atypical communication. Readingand Writing Quarterly, 20(3), 237–260.
Koke, S., & Neilson, J. (1987). The effect of auditoryfeedback on the spelling of nonspeaking physicallydisabled individuals. Unpublished master’s thesis,University of Toronto, Toronto, Ontario, Canada.
Koppenhaver, D. A., & Erickson, K. A. (2003). Natu-ral emergent literacy supports for preschoolers withautism and severe communication impairments. Top-ics in Language Disorders, 23(4), 283–292.
Koppenhaver, D. A., & Yoder, D. E. (1993). Classroomliteracy instruction for children with severe speechand physical impairments (SSPI): What is and whatmight be. Topics in Language Disorders, 13(2), 1–15.
Kress, G. (1982/1994). Learning to write. London: Rout-ledge.
Langer, J. A. (1986). Children reading and writing: Struc-tures and strategies. Norwood, NJ: Ablex.
MacArthur, C., Schwartz, S. S., & Graham, S. (1991). Amodel for writing instruction into a process approachto writing. Learning Disabilities Practice, 6, 230–236.
Massachusetts Department of Education (2012). Mas-sachusetts comprehensive assessment system: Scor-ing guides for MCAS English Language Arts Compo-sition. Retrieved August 28, 2012, from http://www.doe.mass.edu/mcas/student/elacomp_scoreguide.html
McCabe, A., & Bliss, L. S. (2003). Patterns of narra-tive discourse: A multicultural, life span approach.Boston, MA: Allyn & Bacon.
McMaster, K., Du, X., Yeo, S., Deno, S.L., Parker, D., &Ellis, T. (2011). Curriculum-based measures of begin-ning writing: Technical features of the slope. Excep-tional Children, 77(2), 185–206.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.
318 TOPICS IN LANGUAGE DISORDERS/OCTOBER–DECEMBER 2012
Michigan Department of Education. (2000). Michiganliteracy progress profile. Retrieved June 2, 2012, fromhttp://www.misd.net/MLPP/assessments/writing/Writing-A.pdf
National Governors Association Center for Best Prac-tices, Council of Chief State School Officers.(2010). Common Core State Standards. Washing-ton, DC: Author. Retrieved September 1, 2012, from,http://www.corestandards.org/the-standards.
Nelson, N.W. (1992). Performance is the prize: Languagecompetence and performance among AAC users. Aug-mentative and Alternative Communication, 8, 3–18.
Newcomer, P. L., & Barenbaum, E. M. (1991). Thewritten composing ability of children with learningdisabilities: A review of the literature from 1980–1990.Journal of Learning Disabilities, 24, 578–593.
Newkirk, T. (1987). The non-narrative writing of youngchildren. Research in the Teaching of English, 21(2),121–144.
Newkirk, T. (1989). More than stories: The range of chil-dren’s writing. Portsmouth, NH: Heinemann.
Ninio, A., & Bruner, J. (1978). The achievement andantecedents of labeling. Journal of Child Language,5(1), 1–15.
North Carolina Department of Public Instruction. (2009).North Carolina K-2 literacy assessment. English lan-guage arts resources. Retrieved June 2, 2012, fromhttp://www.ncpublicschools.org/docs/curriculum/languagearts/elementary/k2literacy/2009k2-literacy.pdf
Northwest Regional Educational Laboratory. (2010). Be-ginning Writer’s Continuum. 6+1 Trait R© Rubrics(aka Scoring Guides). Retrieved June 2, 2012, fromhttp://educationnorthwest.org/webfm_send/772
Peterson, C., & McCabe, A. (1983). Developmental psy-cholinguistics. New York: Plenum.
Prelock, P. A. (2006). Autism spectrum disorders: Issuesin assessment and intervention. Austin, TX: ProEd.
Salahu-Din, D., Persky, H., & Miller, J. (2008). The Na-tion’s Report Card: Writing 2007 (NCES 2008–468).Washington, DC: National Center for Education Statis-tics, U.S. Department of Education. Retrieved June2, 2012, from http://nces.ed.gov/nationsreportcard/pdf/main2007/2008468.pdf
Scott, C. M. (1989). Problem writers: Nature, assess-ment, and intervention. InA. G. Kamhi, & H. W. Catts(Eds.), Reading disabilities: A developmental lan-guage perspective (pp. 303–344). Boston, MA: Allyn &Bacon.
Scott, C. M. (2012). Learning to write. InA. G. Kamhi, &H. W. Catts (Eds.), Language and reading disabilities(3rd ed., pp. 248–268). Boston, MA: Pearson.
Smith, A., Thurston, S., Light, J., Parnes, P., & O’Keefe,B. (1989). The form and use of written communica-tion produced by physically disabled individuals us-ing microcomputers. Augmentative and AlternativeCommunication, 5, 115–124.
Sturm, J. M. (2012). An enriched writers’ workshopfor beginning writers with developmental disabilities.Topics in Language Disorders, 32(4), 1–35.
Sturm, J. M., & Clendon, S. A. (2004). AAC, language,and literacy: Fostering the relationship. Topics in Lan-guage Disorders, 24(1), 76–91.
Sturm, J. M., Erickson, K. A., & Yoder, D. E. (2003).State of the science: Enhancing literacy participationthrough AAC technologies. Journal of Assistive Tech-nology, 14, 45–54.
Sturm, J. M., Knack, L., & Hall, J. (2011, November).Writing instruction in primary classrooms: Impli-cations for students with disabilities. Poster sessionpresented at the American Speech-Language-HearingConvention, San Diego, CA.
Sturm, J. M., Nelson, N. W., Staskowski, M., & Cali, K.(2010, November). Outcome measures for beginningwriters with disabilities. Miniseminar presented atthe American Speech-Language-Hearing Convention,Philadelphia, PA.
Sulzby, E., Barnhart, J., & Hieshima, J. (1989). Formsof writing and re-rereading from writing: A pre-liminary report (Technical Report No. 20). Berke-ley, CA: National Center for the Study of Writingand Literacy. Retrieved November 15, 2010, fromhttp://www.nwp.org/cs/public/print/resource/606
Sulzby, E., & Teale, W. (1991). Emergent literacy. InR.Barr, M. Kamil, P. Mosenthal, & P. D. Pearson (Eds.),Handbook of reading research (Vol. II, pp. 727–757).New York: Longman.
Tennessee Department of Education. (2010). Tennesseecomprehensive assessment program writing assess-ments. Retrieved January 2, 2011, from http://www.tn.gov/education/assessment/writing.shtml
Tennessee Department of Education. (2012). Writ-ing assessment scoring information. RetrievedAugust 28, 2012, from http://www.tn.gov/education/assessment/writing_scoring.shtml
Tolchinsky, L. (2006). The emergence of writing. InS. Gra-ham, C. MacArthur, & J. Fitzgerald (Eds.), Handbookof writing research (pp. 83–95). New York: GuilfordPress.
Udwin, O., & Yule, W. (1990). Augmentative communi-cation systems taught to cerebral-palsied children—A longitudinal study. 1. The acquisition of signs andsymbols, and syntactic aspects of their use over time.British Journal of Disorders of Communication, 25,295–309.
van Balkom, H., & Welle Donker-Grimbrere, M. (1996).A psycholinguistic approach to graphic language use.InS. von Tetzchner (Ed.), Augmentative and Alter-native Communication: European Perspectives (pp.153–170). London: Whurr.
Vandervelden, M., & Siegel, L. (1999). Phonological pro-cessing and literacy in AAC users and children withmotor speech impairments. Augmentative and Alter-native Communication, 15, 191–211.
Copyright © 2012 Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.