Teaching Statistics in Language Testing Courses

This article was downloaded by: [Universite De Paris 1]On: 20 August 2013, At: 00:08Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Language Assessment QuarterlyPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/hlaq20

Teaching Statistics in Language TestingCoursesJames Dean Brown aa University of Hawai‘i at Mānoa , Honolulu , Hawai‘i , USA

To cite this article: James Dean Brown (2013) Teaching Statistics in Language Testing Courses,Language Assessment Quarterly, 10:3, 351-369, DOI: 10.1080/15434303.2013.769554

To link to this article: http://dx.doi.org/10.1080/15434303.2013.769554

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoever orhowsoever caused arising directly or indirectly in connection with, in relation to or arisingout of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/loi/hlaq20

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/15434303.2013.769554

http://dx.doi.org/10.1080/15434303.2013.769554

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Language Assessment Quarterly, 10: 351–369, 2013Copyright © Taylor & Francis Group, LLCISSN: 1543-4303 print / 1543-4311 onlineDOI: 10.1080/15434303.2013.769554

COMMENTARY

Teaching Statistics in Language Testing Courses

James Dean BrownUniversity of Hawai‘i at Manoa, Honolulu, Hawai‘i, USA

The purpose of this article is to examine the literature on teaching statistics for useful ideas thatteachers of language testing courses can draw on and incorporate into their teaching toolkits as theysee fit. To those ends, the article addresses eight questions: What is known generally about teachingstatistics? Why are students so anxious about statistics? How can teachers help students overcomestatistics anxiety? What teaching approaches that have been tried and worked? What classroom toolshave proven effective? What topics should we cover? What advice does the literature offer? And,where can teachers find more information? This overview of research and reflections on these topicsshould prove useful to anyone facing the prospect of teaching testing statistics to language teachers.

INTRODUCTION

This article does not attempt to deal with the topics of why statistics should be taught in languagetesting courses or what makes a good teacher of statistics in a language testing course. Thoseare topics for entirely different publications. This article is aimed at instructors who have alreadydecided to include statistics in their language testing course, who might want to improve theirteaching of statistics or incorporate more ideas into their statistics teaching toolkits. In effect,I am advocating reflective teaching (Farrell, 2008; Richards & Lockhart, 1994; Wallace, 1991)and taking the view that, with some reflection, any teacher can be provided additional new insightsand strategies to try out that might help in becoming a better teacher.

To investigate the available strategies, I have examined the literature in search of answers tocertain questions directly related to the job of effectively teaching testing statistics to languageteachers:

1. What is generally known about teaching statistics?2. Why are students so anxious about statistics?

Correspondence should be sent to James Dean Brown, University of Hawai‘i at Manoa, Second Language Studies,1890 East-West Road, Honolulu, HI 96744. E-mail: [email protected]

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

352 BROWN

3. How can teachers help students overcome statistics anxiety?4. What teaching approaches have been tried and worked?5. What classroom tools have proven effective?6. What statistics topics should we cover?7. What advice does the literature offer?8. Where can I find more information?

For each of these questions, I start by reviewing the literature on teaching statistics generallyand then, where appropriate, I zero in on the more limited literature on teaching language testingstatistics. I end each section by reflecting on what all this means for those of us who have chosento teach statistics to language teachers.

QUESTIONS WORTH ADDRESSING

What Is Generally Known About Teaching Statistics?

Concern about teaching statistics is nothing new. Back in the 1940s, Hotelling (1940) was con-cerned about who should teach statistics and what kind of person that should be. He was alsoconcerned about what the professional functions of a statistics specialist ought to be. He felt thatthose functions were teaching, doing research, and, of interest, advising colleagues about statisti-cal methods. Since then, the responsibilities of statistics specialists have expanded along with thecontent of the field of statistics.

Becker (1996) provided a decent overview of the topic, addressing the general literature onteaching statistics. She found that the print literature was “largely anecdotal and comprises mainlyrecommendations for instruction based on the experiences and intuitions of individual instruc-tors” (p. 71). She also noted that the existing literature covered a wide range of topics includingempirical literature on materials and strategies for teaching statistics and using computers for thatpurpose, as well as nonempirical literature that describes particular statistics courses and lessonsthat provide useful examples. She also discusses the growing availability of online resourcesfor teachers of statistics including data sets, teaching resources, discussion groups, and an elec-tronic journal. Given that this article was published in the mid-1990s, it is now years out of date.Nonetheless, it is a good place to start because it surveys many of the key issues that teachers ofstatistics confront and provides many references on these issues.

The literature generally indicates that the teaching of statistics to the uninitiated can be prob-lematic wherever it is done regardless of what sort of nonspecialist audience is involved (seecitations in the next section). The literature also indicates a wide range of possible topics thatshould be considered in dealing with the issue of teaching statistics. Language testers, then, donot need to feel as though we are dealing with a special problem when we are trying to teach statis-tics to people in our language testing courses who may have become language teachers (ratherthan mathematicians, engineers, etc.) for a reason. In addition to learning that this problem is notunique to our language testing courses, we should perhaps draw on the general literature to helpus zero directly in on what we need to know in order to effectively teach statistics in our languagetesting courses.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

TEACHING STATISTICS IN LANGUAGE TESTING COURSES 353

Why Are Students So Anxious About Statistics?

Statistics anxiety is a surprisingly big subfield of study within the larger literature on statisticseducation. In brief, statistics anxiety is a complex of behaviors, including uneasiness, trepidation,nervousness, and even debilitating fear, that may occur in some students when they are confrontedwith studying or using statistics. In considerably more detail, Zeidner (1991) defined statisticsanxiety as follows:

Statistics anxiety may be construed as a particular form of performance anxiety characterizedby extensive worry, intrusive thoughts, mental disorganization, tension, and physiological arousal.Statistics anxiety arises in people when exposed to statistics content, problems, instructional situa-tions, or evaluative contexts, and is commonly claimed to debilitate performance in a wide variety ofacademic situations by interfering with the manipulation of statistics data and solution of statisticsproblems. (p. 319)

Onwuegbuzie and Wilson (2003) provided a comprehensive and fairly recent overview of theliterature on statistics anxiety, which I do not attempt to duplicate here. Nonetheless, statisticsanxiety appears to be a fairly important issue, given that “between two-thirds and four-fifths ofgraduate students appear to experience uncomfortable levels of statistics anxiety” (Onwuegbuzie& Wilson, 2003, p. 195).

Is it possible to assess students’ levels of statistics anxiety? Yes. For example, the StatisticalAnxiety Rating Scale, developed by Cruise and Wilkins (as cited in Onwuegbuzie, 2004), is fairlywidely used in research related to this topic. The Statistical Anxiety Rating Scale has 51 five-pointLikert items with subscales on the following: the worth of statistics, interpretation anxiety, testand class anxiety, computational self-concept, fear of asking for help, and fear of the statisticsinstructor. Anyone interested in following up on this issue of measuring statistical anxiety mightalso be interested in the Statistics Attitude Survey (Roberts & Bilderback, 1980; Roberts & Saxe,1982), Attitude Toward Statistics (Wise, 1985), Survey of Attitudes Toward Statistics (Schau,Stevens, Dauphinee, & Vecchio, 1995; applied in Tempelaar & Nijhuis, 2007), and StatisticalAnxiety Scale (Chiesi, Primi, & Carmona, 2011; Pretorius & Norman, 1992). (On a tangentialissue, Gal and Garfield, 1997, provided an online collection of 18 articles all directly related toassessment in statistical courses.)

The issue of what causes statistics anxiety has been widely studied. For example, Birenbaumand Eylath (1994) found no relationships in a sample of 151 female students in education betweenstatistics anxiety and the following variables: acquaintance with the subject, willingness to fur-ther study statistics, grades in the statistics course, and inductive reasoning ability. Blalock (1987)argued that it is necessary to deal with math anxiety and resistance as well as overreliance onmemorization before we can teach statistics. However, according to Onwuegbuzie and Wilson(2003, pp. 196–197), considerable research indicates an inverse relationship between mathemat-ics anxiety and statistics anxiety. Mathematics anxiety is nonetheless predictive of, though distinctfrom, statistics anxiety (Onwuegbuzie, DaRos, & Ryan, 1997). Onwuegbuzie (1997) examinedresearch proposal writing anxiety and found that it is far from simple, showing that it includesat least four components: library anxiety, statistics anxiety, composition anxiety, and researchprocess anxiety. Onwuegbuzie and Daley (1999) studied statistics anxiety and perfectionism andreported that participants who have high degrees of socially prescribed perfectionism tend to have

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

354 BROWN

higher degrees of certain types of statistics anxiety. Onwuegbuzie (2000) investigated the rela-tionships between six dimensions of statistics anxiety and seven dimensions of self-perception.Onwuegbuzie (2004) examined the relationship between academic procrastination and six aspectsof statistics anxiety.

Clearly then, many variables have been shown to be related to statistics anxiety. However,most of these articles have addressed one, two, or three variables at a time as they relate tostatistics anxiety. Onwuegbuzie (2003) was a bit more ambitious: He used structural equationmodeling to successfully model the relationships among a large number of variables (i.e., statis-tics anxiety, research anxiety, computation self-concept, study habits, course load, number ofcollege-level statistics courses, number of college-level research courses, and expectation ofstatistics achievement) and statistics achievement.

Teachers of language testing courses can learn from this literature that statistics anxiety isa widespread phenomenon that is not isolated in the language teaching population and that theissues involved are complex and many. That is why I turn next to practical ways that we can helpstudents overcome this anxiety.

How Can Teachers Help Students Overcome Statistics Anxiety?

The technical literature addressed in the previous section has gotten a good start on trying tofigure out what causes statistical anxiety, but in actuality, those studies have only revealed thatthere are many variables related to statistical anxiety (and to each other) to varying degrees, andof course, relationship is not causality. What is the language testing statistics teacher to do?

A number of scholarly articles have considered variables that may prove more amenable toteacher manipulation. For example, Schacht (1990) examined 12 statistics textbooks written forsocial or behavioral sciences audiences in terms of their potential effects on statistics anxiety anddecided to stop using a textbook until he finds one that meets his students’ needs. Suanpang,Petocz, and Kalceff (2004) found that teaching statistics online resulted in significantly betterattitudes toward statistics than traditional methods. Schacht and Stewart (1990) provided evidencethat humor, especially in the form of cartoons, lowered both mathematics anxiety and statisticsanxiety. Royse and Rompf (1992) argued for bringing games and puzzles into the teaching ofstatistics. In short, the literature offers a number of strategies that language testing course teachersmight want to consider to help reduce statistics anxiety in their students.

So far the discussions in this and the previous section have covered variables that affect statis-tics students’ anxiety. One article offers insights of interest to those who may themselves haveexperienced statistics anxiety and yet are now teaching statistics in a language testing course.Forte (1995) described his own transformation “from a math-phobic and anti-quantitative doc-toral student to a moderately successful applied statistics instructor.” Perhaps using some ofthe suggestions in the previous paragraph will not only help language teachers overcome theirstudents’ statistics anxiety but also help the instructor to enjoy the process a bit more.

What Teaching Approaches Have Been Tried and Worked?

A large proportion of the literature on teaching statistics has been devoted to what I think of as the-approach-I-use-and-how-I-know-it-works articles. Even though they are not empirical studies,they do describe the experiences of others who have struggled with the issues involved and may

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


therefore serve as a rich source of ideas for strategies that language testing course teachers mightwant to try.

Approaches focused on the students’ worlds. The need-to-know approach (Fischer,1996) focuses on what students need to be able to do with statistics, given that most of themare not academic researchers:

We have found that by formulating a course which is designed to be taught from the student’s need-to-know perspective, which is lab-based, and which provides students with repeated opportunities todemonstrate mastery of their skills, we produce students who can perform at a level of professionalsuccess far above that of other undergraduates. (p. 229)

The reasoning-from-data approach (Ridgeway, Nicholson, & McCusker, 2007) argues that cit-izens need to understand statistical reasoning and thus must be taught at least the following:experience with multivariate problems (including at least nonlinear relationships, lags in timebetween cause and effect, and varying effect sizes), examination of causal links, linking data anal-ysis to the contexts in which the data were collected, and speculation about alternative coursesof action to those already suggested (p. 46). The real data approach (Singer & Willett, 1990,pp. 224–225) involves using real-world data rather than artificial data sets for seven reasons:authenticity, background information, interest and relevance, substantive learning, availability ofmultiple analyses, the importance of raw data, and case identifiers. Schacht and Stewart (1992)went even further advocating that statistics teachers use the students themselves as the data orhave them create the data followed by problem solving. The teaching experiments and studentfeedback approach (Barone & Lo Franco, 2010) “primarily addressed to statistics teachers, allowspractical aspects to be organized and decisions to be made based on data that has [sic] been col-lected from students and scientifically analyzed” (p. 1). In his linking statistics to the real worldapproach, Yilmaz (1996) argued that real-world use of statistics requires three competencies:“ability to link statistics and real-world situations, knowledge of basic statistical concepts, andability to synthesize the components of a statistical study and to communicate the results in aclear manner” (see section 2.5).

Approaches focused on how to organize teaching. The lab approach (Nolan & Speed,1999) involves a good deal of hands-on experience with statistics; each lab session is divided intofive parts: “an introduction, data description, background material, investigations, and theory”(p. 371). In the small-group cooperative learning approach (Garfield, 1993), a number differenttypes of small-group activities are used as follows:

• Having groups individually solve a problem and then compare their solutions (e.g.,homework problems or problems from the textbook requiring particular skills).

• Having groups discuss a concept or procedure, or compare different concepts or procedures(e.g., discuss the steps involved in testing a hypothesis, or compare the advantages anddisadvantages of using the mean, median, and mode to summarize a data set).

• Giving groups a data set to analyze and to discuss, followed by a written report of what theyhave learned about the data (e.g., data sets from the Quantitative Literacy materials or datagenerated in class such as estimating the distribution of the number of raisins in a smallbox).

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

356 BROWN

• Having each group collaborate on a large project involving collecting, analyzing, and inter-preting data. Groups may meet in and/or outside of class to work on these projects and maypresent the results in a written report and/or an oral in-class presentation.

• Using groups as a way to learn new material. The jigsaw method can be used, where studentsare assigned temporarily to new groups, and each new group learns something new, such asa different type of plot. Then, students return to their original groups and teach each otherthe material they just learned.

• Having groups compare their ideas about chance phenomena, and then generate or simulatedata to test their beliefs (e.g., distributions of heads and tails when coins are tossed, or thebest strategy to choose in the Monte Hall problem). (section 7).

Garfield also provides three example group activities in appendix. Forte (1995) described a group-learning approach, which includes “computer usage, real-world applications, humor, languagepractice, and group-learning principles.”

Simulation approaches are described in Hodgson and Burke (2000) in terms of their promisesand pitfalls, but the authors come down on the side of using simulations to teach statistics. Mills(2003) discussed a related computer simulation approach, assuring readers that it provides stu-dents practice in constructing their own knowledge and ideas about statistical concepts fromworking with computer simulations and assimilating all new information into their previousknowledge base (p. 58).

Approaches focused on how students learn. The structured approach (Bessant, 1992)is designed to overcome common instructional barriers (p. 149) by following five general stages:warm-up sessions, organizational models, application exercises, pattern recognition, and mean-ing component (pp. 145–148). The discovery approach (Dillbeck, 2009) involves a series ofproblems for groups in class in addition to problem sets that students should work out indi-vidually. In addition to classroom activities, “which spontaneously brought the material closerto the students, the statistical content was also related conceptually to the students’ own expe-rience of gaining knowledge” (p. 19). The problem-solving approach (Bland, 2004) involvesintegrating research methods and statistics into the more general problem-solving approachused in medical education in Australia. The exploratory data analysis approach describedin McBride (1996) is designed to help students develop their critical thinking skills. Theexploratory data analysis approach makes few assumptions about the nature of data, evenavoiding the assumption that data are normally distributed, and instead using graphic presen-tations (e.g., histograms, stem and leaf displays, etc.) together with summary statistics likethe mean, median, standard deviation, and so on, to examine the degree to which the data aresymmetrical/normal (p. 518). The mapping approach (Schau & Mattern, 1997), as the namesuggests, uses connected understanding in the form of mapping techniques like concept mapsand graphic organizers for the planning of instruction, as a tool for learning, and for assessing theinstruction (p. 171).

Approaches focused on how we can make learning stick. The long-term learningapproach promotes learning of statistical concepts that will endure because of techniques thatmake it memorable (Sowey, 1995) by linking structure (including coherence and perspective) andworthwhileness (including intellectual excitement, knowledge of both the strengths and weak-nesses, and practical usefulness). The demonstrations are made memorable or striking by making

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


all aspects of the lesson (a) easily grasped because it is clear and self-contained; (b) immedi-ately informative; (c) provocative of curiosity and reflection; and (d) presented in a manner thatenhances (a), (b), and (c) (Sowey, 2001).

Language testers may wish to consider any and all of these approaches as sources of ideasfor what their students need, how the course should be organized, how their students learn, andhow the learning can be made to stick with the students after they have finished the course. Andnaturally, any approach that seems particularly appealing can be further explored by reading thereferences cited here.

What Classroom Tools Have Proven Effective?

Focusing a bit more narrowly, a number of articles discuss particular tools that can be used inteaching statistics. These fall into three categories: exploiting technology to teach statistics, usinghumor, and applying effective questioning techniques.

Exploiting technology to teach statistics. Although we tend to think of computers whenwe think of technology, there are of course many options that could result in multimediastatistics classrooms. As Velleman and Moore (1996) pointed out, “Multimedia comprises a vari-ety of communication channels. Each has distinctive strengths and weaknesses. A successfulmultimedia system must balance these channels, using each for what it does best, but lettingno one channel dominate” (p. 219). The authors go on to discuss each of the following chan-nels: video, animation, narration, sound, statistics software, randomness and simulation, datageneration, and exercises (pp. 219–222).

With regard specifically to computers, Suanpang et al. (2004) conducted a study that demon-strated that teaching statistics online resulted in significantly better attitudes toward statistics thantraditional methods. So teachers might want to consider some of these computer options as wellas multimedia use in general. More specifically, Varnhagen, Drake, and Finley (2009) used anonline questionnaire to evaluate students’ perceptions of the effectiveness of using the Internetin the course revealing that the students found the Internet laboratory’s communication compo-nents more useful than the information components, found few learning obstacles, and rated thesystem positively overall (p. 275). From the perspective of using a course home page to supportthe teaching of statistics (as opposed to serving as the primary delivery system), Leon and Parr(2000) suggested several dozen ways that a home page can be used to support classroom activi-ties, including soliciting feedback; facilitating student–professor interaction; keeping permanentrecords; posting handouts, old exams, or data files; providing a class calendar, homework log,online mailing lists, blog forum, and links to online resources; giving feedback on assignments;and sharing student assignments with the whole class (pp. 45–46). The authors discuss these ideasin terms of what was successful and what was not. For many more ideas on applying technology,see Garfield and Burrill (1996), which is an edited collection of articles on the roles of technologyin teaching and learning statistics.

Using humor to teach statistics. “Q: How many statisticians does it take to screw in a lightbulb? A: One plus or minus three” (Friedman, Friedman, & Amoo, 2002, section 4.4). If you findthat joke funny, you may be on the way to understanding why humor might be a good tool to usein livening and lightening up your statistics class.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

358 BROWN

Friedman et al. (2002, abstract) provided an excellent overview of the topic of using humor toteach statistics in support of their claim that “humor strengthens the relationship between studentand teacher, reduces stress, makes a course more interesting, and, if relevant to the subject, mayeven enhance recall of the material.” Schacht and Stewart (1990) provided evidence that usinghumor in the form of cartoons can lower mathematics anxiety and statistics anxiety (also seeSchacht, 1991). Lomax and Moosavi (2002) also provided an overview of the research on usinghumor in the classroom and then supplied ample examples of humor used in their statistics classesfor 14 major topics and discussed the strategies they used in planning for humor.

Berk (1996) discussed ratings on the Humor Effectiveness Evaluation assigned by 316 univer-sity students for 10 strategies for using humor in statistics classes:

(a) humorous material on syllabi; (b) descriptors, cautions, and warnings on the covers of hand-outs; (c) opening jokes; (d) skits/dramatizations; (e) spontaneous humor; (f) humorous questions; (g)humorous examples; (h) humorous problem sets; (i) Jeopardy!TM -type reviews for exams; and (j)humorous material on exams. (p. 71)

He provided many useful examples and concluded the following:

1. Students view humor as an effective teaching tool to facilitate their learning.2. A wide range of low-risk humor techniques can be very effective in reducing anxiety and

improving learning and performance.3. Strategies for using humor must be planned well and executed systematically to achieve

specific outcomes.4. Both content-specific and generic humorous material tailored to the characteristics of each

class can be effective in appropriate applications.5. Humor tends to be more effective when two or more of the senses, especially visual and

aural (written and oral), are involved rather than just one sense.6. Offensive humor should never be used in the classroom.7. The 10 strategies for using humor in this study are adaptable and can be generalized to

any discipline and course content. (pp. 87–88)

In using humor in language testing classes, it is worth remembering that our students are oftennon-native speakers of English. As a result, using humor that involves wordplay, puns, or nativeculture references will probably go over the heads of many of them. I find that all of my languagetesting students eventually catch on to verbal slips like repeatedly saying sadisitcs instead ofstatistics, or spontaneous physical humor like repeating something that happened accidentally,for example, if I drop my marker, I may drop it again, and fumble it, and drop it a third time. Thismay seem silly, but it does get a laugh and does not depend on language at all.

Applying effective questioning techniques. Few teachers spend much time thinking aboutwhat question-and-answer techniques might be the most effective for classroom discussions.However, we can learn from researchers who have put a fair amount of time and thought intowhat works and what does not. As Brualdi (1998) put it,

As one may deduce, questioning is one of the most popular modes of teaching. For thousands of years,teachers have known that it is possible to transfer factual knowledge and conceptual understandingthrough the process of asking questions. Unfortunately, although the act of asking questions has the

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


potential to greatly facilitate the learning process, it also has the capacity to turn a child off to learningif done incorrectly.

Wilen and Clegg (as cited in Brualdi, 1998) suggested that teachers employ the following researchsupported practices to foster higher student achievement:

• Phrase questions clearly.• Ask questions of primarily an academic nature.• Allow 3 to 5 s of wait time after asking a question before requesting a student’s response,

particularly when high-cognitive level questions are asked.• Encourage students to respond in some way to each question asked.• Balance responses from volunteering and nonvolunteering students.• Elicit a high percentage of correct responses from students and assist with incorrect

responses.• Probe students’ responses to have them clarify ideas, support a point of view, or extend their

thinking.• Acknowledge correct responses from students and use praise specifically and discrimi-

nately.

Wilen (2001) furthered the debate by discussing nine myths about teacher use of questions inclassrooms, explaining each and then discussing it in terms of a principle that should be applied aswell as an application. For example, Myth 3 is “There are no bad questions.” Part of the principlethat should be applied is, “Good questions are clear, purposeful, sequential, responsive, adapt-able, and thought-provoking, and they help achieve the objectives for which they are intended.Inappropriate questions are vague, run-on, intimidating, nonsequential, and lacking direction andintended exclusively for recall of factual information” (p. 28). Thus by exploding the myths,Wilen imparted information that many teachers might find useful about their in-class questioningtechniques.

Teachers of language testing courses should perhaps seriously consider exploiting technologyto teach statistics, using humor, and applying effective questioning techniques to reduce statisticsanxiety and help their students learn. The articles discussed in this section certainly argue for thatposition but also provide many ideas for how all of that might be done in language testing classes.

What Statistics Topics Should We Cover?

Becker and Greene (2001) advocated teaching certain key concepts in a statistics class includingat least “probability, sampling distribution of an estimator, hypotheses testing, regression to themean, the least squares estimator and alternatives to the least squares estimator” (p. 173). Thatclearly does not describe the set of statistics that language teachers or language testers need toknow. So it seems we cannot rely on the teaching statistics literature to inform our choice(s) oftopics to teach in language testing courses.

Perhaps what we are shooting for is something like statistical literacy, which might include theknowledge and skills necessary to understand and use the statistical language and tools requiredfor full adult numeracy (Garfield & Ben-Zvi, 2007; Rumsey, 2002) including the statistical rea-soning necessary to make sense of statistical information (Garfield, 2002), and even the statisticalthinking that professional statisticians use including how and why we use particular statistics and

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

360 BROWN

research designs, as well as what the assumptions and limitations are of statistics and inferences,and so on (Chance, 2002; Pfannkuch & Wild, 2004; Wild & Pfannkuch, 1999).

More likely our goal is assessment literacy, which can be defined very generally to meanthe knowledge that the general educated public needs to understand the uses and abuses of test-ing and assessment instruments and practices within their society. That should probably be onegoal in teaching statistics in a language testing class. However, educators may need an addi-tional dose of more comprehensive assessment literacy. Popham (2006) asserted, “What mostof today’s educators know about education assessment would fit comfortably inside a kinder-gartner’s half-filled milk carton” (p. 84). In another article, Popham (2009) pointed out that“the sort of assessment literacy that is typically recommended refers to a teacher’s familiar-ity with those measurement basics related directly to what goes on in classrooms” (p. 4). R. J.Stiggins (1991) argued for a fuller definition of assessment literacy for educators that would needto include knowing what high-quality assessment is, knowing the importance matching assess-ment methods to well-defined achievement targets, recognizing the importance of fully samplingperformances, understanding superfluous factors that can obscure assessment results, and recog-nizing when results are presented in a useful and understandable format (p. 535). Popham (2009,pp. 8–10) provided an even more complete list of the knowledge that assessment literate educatorsshould have:

1. The fundamental function of educational assessment, namely, the collection of evidencefrom which inferences can be made about students’ skills, knowledge, and affect.

2. Reliability of educational assessments, especially the three forms in which consistencyevidence is reported for groups of test takers (stability, alternate-form, and internalconsistency) and how to gauge consistency of assessment for individual test takers.

3. The prominent role three types of validity evidence should play in the building ofarguments to support the accuracy of test-based interpretations about students, namely,content-related, criterion-related, and construct-related evidence.

4. How to identify and eliminate assessment bias that offends or unfairly penalizes test takersbecause of personal characteristics such as race, gender, or socioeconomic status.

5. Construction and improvement of selected-response and constructed-response test items.6. Scoring of students’ responses to constructed-response tests items, especially the distinc-

tive contribution made by well-formed rubrics.7. Development and scoring of performance assessments, portfolio assessments, exhibitions,

peer assessments, and self-assessments.8. Designing and implementing formative assessment procedures consonant with both

research evidence and experience-based insights regarding such procedures’ likely suc-cess.

9. How to collect and interpret evidence of students’ attitudes, interests, and values.10. Interpreting students’ performances on large-scale, standardized achievement and apti-

tude assessments.11. Assessing English Language Learners and students with disabilities.12. How to appropriately (and not inappropriately) prepare students for high-stakes tests.13. How to determine the appropriateness of an accountability test for use in evaluating the

quality of instruction.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


Naturally, Popham describes each of these 13 in considerably more detail, but this list aloneshould be food for thought for anyone teaching a language testing course (for more on generalassessment literacy, see DeLuca & Klinger, 2010).

In an overview article on assessment literacy in language testing, Taylor (2009, p. 24) stated,“The term assessment literacy has been coined by several writers in recent years to describe whatthe constituency of language teachers and instructors needs to know about assessment matters(Inbar-Lourie, 2008; Malone, 2008; G. Stiggins, 1997; Stoynoff & Chapelle, 2005).” Amongother things, Taylor briefly covered the components of assessment literacy in language testing(especially on pp. 26–27). She pointed to Davies (2008), who described the current state ofteaching language testing as trending from skills and knowledge to include principles. As Davieshimself put it,

Skills provide the training in necessary and appropriate methodology, including item writing, statis-tics, test analysis and increasingly software programmes for test delivery, analysis and reportage.Knowledge offers relevant background in measurement and language description, as well as in con-text setting, and may involve an examination of different models of language learning, of languageteaching and of language testing such as communicative language testing, performance testing andnowadays, socio-cultural theory. Principles concern the proper use of language tests, their fairnessand impact, including questions of ethics and professionalism, thus a consideration of the growingprofessionalism of language testing, of the responsibilities of language testers and of the impact oftheir work on a range of stakeholders and of the ethical choices they must make. (p. 328)

Taylor (2009) also cited Bailey and Brown (1996) and their later replication of their own studyin Brown and Bailey (2008). Note that these two studies offered an extensive list of possiblelanguage testing topics and asked teachers of such courses to rate the amount of time they spendon each topic on 6-point Likert items (from none to extensive). Notably, the 1995 questionnairehad 76 topics, whereas the 2008 study had 96 topics for a gain of 20 topics in 13 years. Theexpanded list of topics included in the 2008 questionnaire (pp. 375–381) is shown in Table 1.Notice that those topics most likely to be statistical in nature are in italics and that there are 55 ofthem out of 96 topics, which is about 57%.

Taylor (2009) went on to make the argument (based primarily on Inbar-Lourie, 2008, andMcNamara & Roever, 2006) that

this current trend in thinking seems to be that training for assessment literacy entails an appropriatebalance of technical know-how, practical skills, theoretical knowledge, and understanding of princi-ples, but all firmly contextualized within a sound understanding of the role and function of assessmentwithin education and society. (p. 27)

Inbar-Lourie (2008) in particular is cited by Taylor (2009, p. 27) as including

initiatives that seek to reconceptualize assessment paradigms and the competencies needed by thoseinvolved, including, among others, work on assessment for learning (Assessment Reform Group,2002), classroom-based assessment (Rea-Dickins, 2008), alternative assessment (Fox, 2008), anddynamic assessment (Lantolf & Poehner, 2008).

Fulcher (2012) pulled many of these topics together in his analysis of the assessment trainingneeds of language teachers.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

TAB

LE1

Topi

csC

over

edin

the

Bro

wn

&B

aile

y(2

008)

Que

stio

nnai

re

Glo

bal

Item

Ana

lysi

sTe

stC

onsi

sten

cyVa

lidi

ty

item

wri

ting

item

wri

ting

sour

ces

ofte

stin

ger

ror

cont

entv

alid

ityte

stad

min

istr

atio

nite

mw

ritin

gfo

rdi

ffer

ents

kills

relia

bilit

yan

dte

stle

ngth

item

-obj

ectiv

eco

ngru

ence

test

scor

ing

item

cont

enta

naly

sis

relia

bilit

yan

dra

nges

ofta

lent

item

-spe

cific

atio

nco

ngru

ence

test

scor

ein

terp

reta

tion

item

qual

ityan

alys

iste

st-r

etes

trel

iabi

lity

cons

truc

tval

idity

test

revi

sion

item

faci

lity

(a.k

.a.i

tem

diffi

culty

orea

sine

ss)

para

llelf

orm

sre

liabi

lity

crite

rion

-rel

ated

valid

ityte

stcr

itiqu

ing

item

disc

rim

inat

ion

(tra

ditio

nal)

inte

rnal

cons

iste

ncy

relia

bilit

yco

rrel

atio

nala

naly

sis

test

taki

ngbi

seri

alco

rrel

atio

nst

anda

rder

ror

ofm

easu

rem

ent

sim

ple

regr

essi

onan

alys

iste

stan

alys

isite

mag

reem

ent

Split

-hal

fad

just

edm

etho

dst

anda

rder

ror

ofes

timat

eG

ener

al:

Item

beta

[sic

]b-

inde

xSp

earm

an-B

row

nPr

ophe

cyfo

rmul

apr

inci

ple

com

pone

nts

anal

ysis

norm

-ref

eren

ced

test

ing

dist

ract

oref

ficie

ncy

anal

ysis

Kud

er-R

icha

rdso

n20

(K-R

20)

rota

ted

fact

oran

alys

iscr

iteri

on-r

efer

ence

dte

stin

gon

e-pa

ram

eter

IRT

Kud

er-R

icha

rdso

n21

(K-R

21)

confi

rmat

ory

fact

oran

alys

isac

hiev

emen

ttes

ting

two-

para

met

erIR

TC

ronb

ach

alph

am

ulti-

trai

t/m

ulti-

mea

sure

aptit

ude

test

ing

thre

e-pa

ram

eter

IRT

Rul

onco

nseq

uent

ialv

alid

itydi

agno

stic

test

ing

Ras

chan

alys

isFl

anag

anva

lues

impl

icat

ions

inva

lidity

plac

emen

ttes

ting

Des

crip

tive

Stat

istic

s:G

uttm

anm

ultip

lere

gres

sion

profi

cien

cyte

stin

gm

ean

inte

rrat

erst

ruct

ural

equa

tion

mod

elin

gm

easu

ring

attit

udes

mod

ein

trar

ater

anal

ysis

ofva

rian

ce(A

NO

VA

)pr

oced

ures

mea

suri

ngth

edi

ffer

ents

kills

med

ian

phic

oeffi

cien

tm

any

face

ted

Ras

ch(F

AC

ET

S)an

alys

isas

sess

men

tatd

iffe

rent

leve

lsm

idpo

int

phi(

lam

bda)

valid

ityas

aun

itary

conc

ept

criti

quin

gpu

blis

hed

test

sra

nge

agre

emen

tcoe

ffici

ent

alte

rnat

ive

asse

ssm

entp

roce

dure

sst

anda

rdde

viat

ion

kapp

aW

ashb

ack

vari

ance

gene

raliz

abili

tyco

effic

ient

test

bias

stan

ines

gene

raliz

abili

tyth

eory

test

ing

inre

latio

nshi

pto

curr

icul

umz

scor

esst

anda

rds

(cut

-poi

nt)

setti

ngT

scor

escr

itica

lapp

roac

hes

tola

ngua

gete

stin

gC

olle

geE

ntra

nce

Exa

min

atio

nB

oard

(CE

EB

)sc

ores

lang

uage

prog

ram

eval

uatio

nC

ompu

ter-

base

dT

OE

FLsc

ores

clas

sroo

mte

stin

gpr

actic

esIn

tern

et-b

ased

TO

EFL

scor

es

362

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


As time passes, increasing numbers of topics will continue to surface that could or shouldbe covered in language testing courses. Some topics that already come immediately to mindinclude teaching about intelligence testing, a topic that all educated citizens should probablyunderstand (see, e.g., Cody & Prieto, 2000) or ethics in testing (see, e.g., Rupert, Kozlowski,Hoffman, Daniels, & Piette, 1999), both of which should concern us all. Other potential topicsare differential assessment (e.g., classroom language assessment designed to identify and accom-modate individual differences in learning styles) or continuous assessment, wherein assessmentis designed to provide constant, even daily, feedback to students in the language learning process.I am sure that any language testing course teacher can add other topics to the list. The point is thatclass time is limited and so, as always, it will be up to individual teachers to select topics that willbe the most suitable for the students they are teaching. The purpose of this section was to supplyideas of topics that other language testers apparently include or would like to include in their lan-guage testing courses. Clearly, there are many topics that language testing course teachers mightwant to consider adding to the list shown in Table 1, and importantly, such topics should probablybe selected and organized with the goal of helping language teachers attain assessment literacy“that integrates knowledge, skills, and principles in a procedural text that attempts to balancewhat will be required for both classroom and normative assessment” (Fulcher, 2012, p. 126).

What Advice Does the Literature Offer?

Based on her experience, Potter (1995) advised the following: enlist students as an audience (i.e.,talk to them about the teaching/learning process, about how statistics relate to their real worldexperiences, etc.); engage the students in the process (e.g., through problem solving, small-groupwork, and an activity she labels call and response); and urge the students to become practitionerswho answer real questions, solve real problems, and read and interpret statistical research criti-cally (pp. 259–262). Naturally, she expands on each of those points, but that is the outline of whatshe suggests.

Based on her experience teaching in Sydney, Simpson (1995) listed some principles that sheargued should guide the teaching of statistics:

. . . statistics should be taught as non-mathematically as possible, only introducing formulae whenabsolutely necessary and explaining their components; plenty of practical applications should begiven; there should be ample opportunity for practice to gain hands-on experience using both cal-culators and computers (preferably with MINITAB); and tutorials should be streamed according toperceived mathematical ability, with remedial mathematics teaching available to those who need it.(p. 199)

Garfield (1995) proposed 10 principles for learning statistics. These principles were laterregrouped by Garfield and Ben-Zvi (2007, pp. 387–389) into the following eight principles(which serve as headings in their article):

1. Students learn by constructing knowledge.2. Students learn by active involvement in learning activities.3. Students learn to do well only what they practice doing.4. It is easy to underestimate the difficulty students have in understanding basic concepts of

probability and statistics.5. It is easy to overestimate how well students understand basic concepts.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

364 BROWN

6. Learning is enhanced by having students become aware of and confront their errors in reasoning.7. Technological tools should be used to help students visualize and explore data, not just to follow

algorithms to pre-determined ends.8. Students learn better if they receive consistent and helpful feedback on their performance.

What makes this list particularly interesting is that it is backed up point-by-point in the article byempirical research.

McBride (1996) suggested a number of ways to help students learn statistics by showingthem how to develop their study habits including class attendance and punctuality. McBride alsosuggested the following to students:

1. Maintain a positive attitude toward your work. People who excel at solving problems, such asthose that we will be working on this semester, believe that reasoning problems can be solvedthrough careful persistent analysis.

2. Read slowly and carefully. We will be reading materials that use a vocabulary that will be unfa-miliar to most of you. There is evidence that reading out loud can be of considerable assistance tocomprehension. Write down complex pieces of information in some kind of organized manner tohelp keep track of it.

3. Be concerned with accuracy. Make sure you understand the basic facts in a problem fully andcompletely.

4. Break problems down into manageable parts.5. Avoid guessing. Don’t jump to conclusions. Make sure you check any intuitive leaps. (p. 518)

Bradstreet (1996) argued that it is important to consider teaching statistical reasoning (includ-ing thinking and concepts) and statistical methods (including computation), but also to considerthat “workshop-based courses effectively provide situated learning and an intimate teaching envi-ronment. Use real (or realistic) data, graphics, and teach exploratory data analysis before classicalmethods. Evaluate the students’ levels of statistical anxiety prior to and during the course” (p. 69).More specifically, he supplied the following general lists of dos and don’ts (pp. 75–76):

Do . . .

• Know your target audience.• Keep it simple.• Provide intimate teaching.• Encourage interactive learning.• Learn the clients’ subject matter and local terminology before designing or teaching a course.• Provide numerous relevant examples.• Show energy and enthusiasm in the classroom.• Insert humor into the lectures, either your own or borrow some [see, e.g., Gonick & Smith, 1993].• Adjust course length and timing to accommodate the clients’ organizational constraints.• Minimize note taking by providing effective and outs and course notebooks.• Get feedback from the students.• Be accessible before, during, and after class.• Provide a physically pleasant learning environment.• Obtain support of top management.

Do Not . . .

• Talk down to students.• Be dogmatic.• Be too mysterious by giving cookbook lectures.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


Perhaps readers would benefit from making a list of their own dos and don’ts based on their ownobservations (or the suggestions cited in this section) that best fit their own personality, students,and teaching situations. For me such a list might look something like the following:

Don’t . . .

• Administer a pretest on testing statistics unless you are trying to get your (languageteaching) students to drop your course.

• Talk down to the students.• Go too fast through the material.• Leave anybody behind.• Teach like the worst statistics teachers you ever had.

Do . . .

• Empathize with the students (i.e., remember how frightening it was to be a mathophobelearning baby statistics).

• Remember that statistics, like skydiving and running into a fire, are not natural activities formany humans.

• Check constantly that students are understanding.• Recall your best statistics teacher (Richard J. Shavelson in my case) and use what he or she

did in combination with your own personality and strengths to become the best statisticsteacher you can.

• Consistently use one set of symbols and stick with them (Why not APA: M, S, S2, r, rxy,rxx’, etc.?).

• Consistently use one set of formulas and stick with them (Why not use conceptual formulasrather than short-cut formulas?).

• Consider having students learn and use computer software tools like Microsoft Excel (afterBrown, 2005, or Carr, 2011) or SPSS (after Bachman, 2004).

Where Can I Find More Information?

A number of websites are worth exploring for more information including some devoted to teach-ing statistics, others covering assessment in general, or even some about language testing. Onevery useful website, which is related to teaching statistics is chock full of resources for statisticsteachers. It is found at http://www.statsci.org/teaching.html. This site leads to a large number ofother websites in categories like the following: directories of resources, online tutorials or text-books, interactive demonstrations, video series/CDs, historical, journals, and general resources.Three journals that cover teaching statistics are also worth considering: Practical AssessmentResearch and Evaluation is a free online journal available at http://pareonline.net/; the Journalof Statistics Education is also free and online at http://www.amstat.org/publications/jse/; andTeaching Statistics is available at http://www.rsscse.org.uk/ts/. This last journal is by subscriptiononly, but a free Best of Teaching Statistics Articles compilation is available at that website, andyou can buy several other compilations. In addition, a statistics glossary is available at http://www.stats.gla.ac.uk/steps/glossary/index.html; a HyperStat Online Statistics Textbook can befound at http://davidmlane.com/hyperstat/index.html; a pdf file by Saporta that includes manyuseful URLs for teaching statistics with the Internet is downloadable from http://citeseerx.ist.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

366 BROWN

psu.edu/viewdoc/download?doi=10.1.1.66.4576&rep=rep1&type=pdf; and a similar webpageis available from Robin Lock at http://it.stlawu.edu/~rlock/tise98/onepage.html.

In the general testing area, useful websites include Assessment and Evaluation on theInternet ERIC/AE Digest at http://www.ericdigests.org/1996-1/evaluation.htm; Assessmentand Evaluation Resources on the NET at http://www.natd.org/assessmentandevalresources.htm; Assessment and Evaluation: Resources on the Internet at http://www.natd.org/assessmentandevalresources.htm; and Education Standards and Testing at http://dir.yahoo.com/Education/Standards_and_Testing/.

A number of websites specialize in language testing issues, but the single best source ofinformation is the Language Testing Resources Website, maintained by Glenn Fulcher at http://languagetesting.info/. Fulcher’s web page includes videos, features, articles, links, podcasts,scenarios, statistics, and a bookstore along with information about language testing organiza-tions, journals, conferences, employment opportunities, and much more. This is really a one stopwebsite for language testing specialists.

Another clearing house for language testing is the JALT Testing and Evaluation SIGHomepage: Links page at http://jalt.org/test/links.htm. A modest overview of “resources avail-able in language testing” is available at http://jalt.org/test/bro_23.htm. Indeed, you will find manyuseful language testing articles at the website of The Shiken (a newsletter of the Testing andEvaluation N-SIG of the JALT organization) available at http://jalt.org/test/pub.htm.

CONCLUSION

This article has addressed eight questions that every teacher of language testing courses hasprobably wondered about. In the process, I have explored what is known generally about teach-ing statistics, why students are so anxious about statistics, how teachers can best help studentsovercome statistics anxiety, the variety of teaching approaches that have been tried and worked,classroom tools that have proven effective, the various statistics topics people consider coveringin their language testing courses, advice that people have provided over the years about teachingstatistics, and where teachers can find more information online. This article does not solve all theproblems that teachers of language testing courses are likely to encounter in teaching statistics.As I stated at the beginning of the article, however, my purpose was to provide ideas that teach-ers of such courses might want to think about and may decide to incorporate into their teachingtoolkits.

Other people have walked these paths before. Some have succeeded, some have failed. Thisarticle offers an overview of their research and reflections. If a new generation of language testingcourse teachers can benefit from this overview, that is all the better for language testing as a whole.

REFERENCES

Assessment Reform Group. (2002). Assessment for learning: 10 principles. Retrieved from http://assessmentreformgroup.files.wordpress.com/2012/01/10principles_english.pdf

Bachman, L. F. (2004). Statistical analyses for language assessment. Cambridge, UK: Cambridge University.Bailey, K. M., & Brown, J. D. (1996). Language testing courses: What are they? In A. Cumming & R. Berwick (Eds.),

Validation in language testing (pp. 236–256). Clevedon, England: Multilingual Matters.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


Barone, S., & Lo Franco, E. (2010). TESF methodology for statistics education improvement. Journal of StatisticsEducation, 18(3), 1–25. Retrieved from http://www.amstat.org/publications/jse/v18n3/barone.pdf

Becker, B. J. (1996). A look at the literature (and other resources on teaching statistics). Journal of Educational andBehavioral Statistics, 21, 71–90.

Becker, B. J., & Greene, W. H. (2001). Teaching statistics and econometrics to undergraduates. The Journal of EconomicPerspectives, 15, 169–182.

Berk, R. A. (1996). Student ratings of 10 strategies for using humor in college teaching. Journal on Excellence in CollegeTeaching, 7, 71–92.

Bessant, K. C. (1992). Instructional design and the development of statistical literacy. Teaching Sociology, 20, 143–149.Birenbaum, M., & Eylath, S. (1994). Who is afraid of statistics? Correlates of statistics anxiety among students of

educational sciences. Educational Research, 36, 93–98.Blalock, H. M., Jr. (1987). Some general goals in teaching statistics. Teaching Sociology, 15, 164–172.Bland, J. M. (2004). Teaching statistics to medical students using problem-based learning: the Australian experience.

BMC Medical Education, 4(31), 1–5.Bradstreet, T. E. (1996). Teaching introductory statistics courses so that nonstatisticians experience statistical reasoning.

The American Statistician, 50, 69–78.Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment (New ed.).

New York, NY: McGraw-Hill.Brown, J. D., & Bailey, K. M. (2008). Language testing courses: What are they in 2007? Language Testing, 25, 349–383.Brualdi, A. C. (1998). Classroom questions. Practical Assessment, Research & Evaluation, 6(6). Retrieved from http://

PAREonline.net/getvn.asp?v=6&n=6Carr, N. (2011). Designing and analyzing language tests. Oxford, UK: Oxford University.Chance, B. L. (2002). Components of statistical thinking and implications for instruction and assessment. Journal of

Statistics Education, 10(3). Retrieved from http://www.amstat.org/publications/jse/v10n3/chance.htmlChiesi, F., Primi, C., & Carmona, J. (2011). Measuring statistics anxiety: Cross-country validity of the Statistical Anxiety

Scale (SAS). Journal of Psychoeducational Assessment, 29, 559–569.Cody, M. S., & Prieto, L. R. (2000). Teaching intelligence testing in APA-accredited programs: A national survey.

Teaching of Psychology, 27, 190–194.Davies, A. (2008). Textbook trends in teaching language testing. Language Testing, 25, 327–347.DeLuca, C., & Klinger, D. A. (2010). Assessment literacy development: Identifying gaps in teacher candidates’ learning.

Assessment in Education: Principles, Policy, & Practice, 17, 419–438.Dillbeck, M. C. (2009). Teaching statistics in terms of the knower. Teaching of Psychology, 10, 18–20.Farrell, T. S. C. (2008). Reflective language teaching: From research to practice. New York, NY: Continuum International.Fischer, H. W., III. (1996). Teaching statistics from the user’s perspective. Teaching Sociology, 24, 225–230.Forte, J. (1995). Teaching statistics without sadistics. Journal of Social Work Education, 31(2).Fox, J. (2008). Alternative assessment. In E. Shohamy & N. Hornberger (Eds.), Encyclopedia of language and education:

Vol. 7. Language testing and assessment (2nd ed., pp. 97–109). New York, NY: Springer.Friedman, H. H., Friedman, L. W., & Amoo, T. (2002). Using humor in the introductory statistics course. Journal of

Statistics in Education, 10(3).Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9, 113–132.Gal, I., & Garfield, J. B. (Eds.). (1997). The assessment challenge in statistics education. Amsterdam, the Netherlands:

IOS Press. Retrieved from http://www.stat.auckland.ac.nz/~iase/publications/assessbkrefGarfield, J. (1993). Teaching statistics using small-group cooperative learning. Journal of Statistics Education, 1(1).

Retrieved from http://www.amstat.org/publications/jse/v1n1/garfield.htmlGarfield, J. (1995). How students learn statistics. International Statistical Review, 63, 25–34.Garfield, J. (2002). The challenge of developing statistical reasoning. Journal of Statistics Education, 10(3). Retrieved

from http://www.amstat.org/publications/jse/v10n3/garfield.htmlGarfield, J., & Ben-Zvi, D. (2007). How students learn statistics revisited: A current review of research on teaching and

learning statistics. International Statistical Review, 75, 372–396.Garfield, J. B., & Burrill, G. (Eds.). (1996). Research on the role of technology in teaching and learning statis-

tics: Proceedings of the 1996 IASE Round Table Conference—University of Granada, Spain. Voorburg, theNetherlands: International Statistical Institute. Retrieved from http://www.dartmouth.edu/~chance/teaching_aids/IASE/IASE.book.pdf

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

368 BROWN

Gonick, L., & Smith, W. (1993). The cartoon guide to statistics. New York, NY: Harper Collins.Hodgson, T., & Burke, M. (2000). On simulation and the teaching of statistics. Teaching Statistics, 22, 91–96.Hotelling, H. (1940). The teaching of statistics. Annals of Mathematical Statistics, 11, 457–470.Inbar-Lourie, O. (2008). Constructing a language assessment knowledge base: A focus on language assessment courses.

Language Testing, 25, 385–402.Lantolf, J., & Poehner, M. (2008). Dynamic assessment. In E. Shohamy & N. Hornberger (Eds.), Encyclopedia of

language and education: Vol. 7. Language testing and assessment (2nd ed., pp. 273–284). New York, NY: Springer.Leon, R. V., & Parr, W. C. (2000). Use of course home pages in teaching statistics. American Statistician, 54, 44–48.Lomax, R. G., & Moosavi, S. A. (2002). Using humor to teach statistics: Must they be orthogonal? Understanding

Statistics, 1, 113–130.Malone, M. (2008). Training in language assessment. In E. Shohamy & N. Hornberger (Eds.), Encyclopedia of language

and education: Vol. 7. Language testing and assessment (2nd ed., pp. 225–239). New York, NY: Springer.McBride, A. B. (1996). Creating a critical thinking learning environment: Teaching statistics to social science

undergraduates. Political Science and Politics, 29, 517–521.McNamara, T., & Roever, C. (2006). Language testing: The social dimension. Oxford, UK: Blackwell.Mills, J. D. (2003). A theoretical framework for teaching statistics. Teaching Statistics, 25, 56–58.Nolan, D., & Speed, T. P. (1999). Teaching statistics theory through applications. American Statistician, 53, 370–375.Onwuegbuzie, A. J. (1997). Writing a research proposal: The role of library anxiety, statistics anxiety, and composition

anxiety. Library & Information Science Research, 19, 5–33.Onwuegbuzie, A. J. (2000). Statistics anxiety and the role of self-perceptions. Journal of Educational Research, 93,

323–330.Onwuegbuzie, A. J. (2003). Modeling statistics anxiety achievement among graduate students. Educational and

Psychological Measurement, 63, 1020–1038.Onwuegbuzie, A. J. (2004). Academic procrastination and statistics anxiety. Assessment & Evaluation in Higher

Education, 29, 3–19.Onwuegbuzie, A. J., & Daley, C. E. (1999). Perfectionism and statistics anxiety. Personality and Individual Differences,

26, 1089–1102.Onwuegbuzie, A. J., DaRos, D. A., & Ryan, J. (1997). The components of statistics anxiety: A phenomenological study.

Focus on Learning Problems in Mathematics, 19, 11–35.Onwuegbuzie, A. J., & Wilson, V. A. (2003). Statistics anxiety: Nature, etiology, antecedents, effects, and treatments—A

comprehensive review of the literature. Teaching in Higher Education, 8, 195–209.Pfannkuch, M., & Wild, C. (2004). Towards an understanding of statistical thinking. In D. Ben-Zvi & J. Garfield

(Eds.), The challenge of developing statistical literacy, reasoning and thinking (pp. 17–46). New York, NY: KluwerAcademic.

Popham, W. J. (2006, March). Needed: A Dose of assessment literacy. Educational Leadership, pp. 84–85.Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory into Practice, 48, 4–11.Potter, A. M. (1995). Statistics for sociologists: Teaching techniques that work. Teaching Sociology, 23, 259–263.Pretorius, T. B., & Norman, A. M. (1992). Psychometric data on the statistics anxiety scale for a sample of South African

students. Educational and Psychological Measurement, 52, 933–937.Rea-Dickins, P. (2008). Classroom-based language assessment. In E. Shohamy & N. Hornberger (Eds.), Encyclopedia of

language and education: Vol. 7. Language testing and assessment (2nd ed., pp. 257–271). New York, NY: Springer.Richards, J. C., & Lockhart, C. (1994). Reflective teaching in second language classrooms. Cambridge, MA: Cambridge

University.Ridgway, J., Nicholson, J., & McCusker, S. (2007). Teaching statistics—despite its applications. Teaching Statistics,

29(2), 44–48.Roberts, D. M., & Bilderback, E. W. (1980). Reliability and validity of a statistics attitude survey. Educational and

Psychological Measurement, 40, 235–238Roberts, D. M., & Saxe, J. E. (1982). Validity of a statistics attitude survey: A follow-up study. Educational and

Psychological Measurement, 47, 907–912.Royse, D., & Rompf, E. L. (1992). Math anxiety: A comparison of social work and non-social work students. Journal of

Social Work Education, 28, 270–277.Rumsey, D. J. (2002). Statistical literacy as a goal for introductory statistics courses. Journal of Statistics Education,

10(3). Retrieved from http://www.amstat.org/publications/jse/v10n3/rumsey2.html

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3


Rupert, P. A., Kozlowski, N. F., Hoffman, L. A., Daniels, D. D., & Piette, J. M. (1999). Practical and ethical issues inteaching psychological testing. Professional Psychology: Research and Practice, 30, 209–214.

Schacht, S. P. (1990). Statistics textbooks: Pedagogical tools or impediments to learning? Teaching Sociology, 18,390–396.

Schacht, S. (1991). What does Opus know about statistics? The cartoon technique for reducing student anxiety. InL. R. Gaydosh (Ed.), Syllabi and instructional material for social statistics (pp. 55–57). Washington, DC: AmericanSociological Association Teaching Resources Center.

Schacht, S. P., & Stewart, B. J. (1990). What’s funny about statistics: A technique for reducing anxiety. TeachingSociology, 18(1), 52–56.

Schacht, S. P., & Stewart, B. J. (1992). Interactive/user-friendly gimmicks for teaching statistics. Teaching Sociology, 20,329–332.

Schau, C., & Mattern, N. (1997). Use of map techniques in teaching applied statistics courses. American Statistician, 51,171–175.

Schau, C., Stevens, J., Dauphinee, T. L., & Vecchio, A. D. (1995). The development and validation of the Survey ofAttitudes Toward Statistics. Educational and Psychological Measurement, 55, 868–875.

Simpson, J. M. (1995). Teaching statistics to non-specialists. Statistics in Medicine, 14, 199–208.Singer, J. D., & Willett, J. B. (1990). Improving the teaching of applied statistics: Putting the data back into data analysis.

American Statistician, 44, 223–230.Sowey, E. R. (1995). Teaching statistics: Making it memorable. Journal of Statistics Education, 3(2). Retrieved from

http://www.amstat.org/publications/jse/v3n2/sowey.htmlSowey, E. R. (2001). Striking demonstrations in teaching statistics. Journal of Statistics Education, 9(1). Retrieved from

http://www.amstat.org/publications/jse/v9n1/sowey.htmlStiggins, G. (1997). Student centered classroom assessment. Upper Saddle River, NJ: Prentice Hall.Stiggins, R. J. (1991). Assessment literacy. The Phi Delta Kappan, 72, 534–539.Stoynoff, S., & Chapelle, C. (2005). ESOL tests and testing: A resource for teachers and program administrators.

Alexandria, VA: TESOL.Suanpang, P., Petocz, P., & Kalceff, W. (2004). Student attitudes to learning business statistics: Comparison of online and

traditional methods. Educational Technology & Society, 7(3), 9–20.Taylor, L. (2009). Developing assessment literacy. Annual Review of Applied Linguistics, 29, 21–36.Tempelaar, D. T., & Nijhuis, J. F. H. (2007). Commonalities in attitudes and beliefs toward different academic subjects. In

M. K. McCuddy, H. van den Bosch, J. W. B. Martz, A. V. Matveev, & K. O. Morse (Eds.), Educational innovation ineconomics and business X: The challenges of educating people to lead in a challenging world (pp. 225–250). Berlin,Germany: Springer.

Varnhagen, C. K., Drake, S. M., & Finley, G. (2009). Teaching statistics with the Internet. Teaching of Psychology, 24,275–278.

Velleman, P. F., & Moore, D. S. (1996). Multimedia for teaching statistics: Promises and pitfalls. American Statistician,50, 217–225.

Wallace, M. J. (1991). Training foreign language teachers: A reflective approach. Cambridge, UK: Cambridge University.Wild, C. J., & Pfannkuch, M. (1999). Statistical thinking in empirical enquiry. International Statistical Review, 67,

223–265.Wilen, W. W. (2001). Exploring myths about teacher questioning in the social studies classroom. Social Studies, 92,

26–32.Wise, S. L. (1985). The development and validation of a scale measuring attitudes towards statistics. Educational and

Psychological Measurement, 45, 401–405.Yilmaz, M. R. (1996). The challenge of teaching statistics to non-specialists Journal of Statistics Education, 4(1).

Retrieved from http://www.amstat.org/publications/jse/v4n1/yilmaz.htmlZeidner, M. (1991) Statistics and mathematics anxiety in social science students—Some interesting parallels, British

Journal of Educational Psychology, 61, 319–328.

Dow

nloa

ded

by [

Uni

vers

ite D

e Pa

ris

1] a

t 00:

08 2

0 A

ugus

t 201

3

Documents

Teaching Statistics in Language Testing Courses