Response to Cizek

Historical High-Stakes Policies Relating to Unintended Consequences of HighHStakes Testing William Rich, California State Universitg, Chico

Current high-stakes, standardized testing policy is discussed through historical analogy with Chairman Mao‘s famine in China and the Maginot Line in France. Both of these national, high-stakes policies resulted in catastrophic failure. I f the accountability movement‘s goals are to improve our ability to compete economically with other nations, we may also be heading for failure. Assessment should be aligned with the skills and knowledge crucial to our success in the future such as collaboration, experimentation, and comfort with ambiguity# Funds currently allocated to standardized testing should be reallocated to the development of measures for such skills and knowledge.

Keywords: testing

accountability, assessment, high-stakes policy, standardized

rofessor Cizek (2001) has brought P some balance to the emotionally charged discussion surrounding high- stakes, standardized testing in his ar- ticle, “More Unintended Consequences of High-Stakes Testing.” However, the article misses the mark over two large issues surrounding this debate. First, high-stakes policy results in unforgiv- ing forces through which the system is reshaped to meet only the high-stakes goals. Non-high-stakes activities may disappear. Second, if the high-stakes goals are wrong, then the entire sys- tem may face disaster.

While Cizek clarifies his argument by relating the old testament account of the Gilead Guards (Cizek, 2001) to the poor

test performers of today, other more re- cent historical events also speak volumes regarding the consequences of high- stakes testing. Chairman Mao’s high- stakes Great Leap Forward policies and the decision to build the Maginot Line in post World War I France are cases in point.

Death to the Birds Chairman Mao’s leadership is illustra- tive of a high-stakes approach with crop management in China, not only for the failure to provide food for the Chinese, but also for the destruction of birds. Under Mao, communist offi- cials attempted to copy Soviet agricul-

tural methods, which from an ideolog- ical perspective were guaranteed to boost crop production. Within this ill- fated plan was the recognition that birds, especially sparrows, were respon- sible for destroying much of the crop before it could be harvested. Peasants were sent to the fields to kill sparrows so that more grain could be harvested and production would rise. Pots and pans were beaten by villages of peas- ants in the fields until the birds fell from exhaustion (Becker, 1996). Sadly, the unwise communists destroyed not just the birds but the ecological bal- ance that, among other things, kept the insects in check. With the birds gone, the insects virtually destroyed the crops, causing widespread famine and leaving the people in even worse condition than before the bird killing policywas enacted (Becker, 1996).

Furthermore, anyone who questioned the effects of Mao’s policies was immedi- ately purged. This practice severely dis-

This commentary is a response to the article “More Unintended Consequences of High-Stakes Testing” by Gregory J. Ckek, which was published in the Winter 2001 issue of EM:IP ( p p 19-27).

William Rich is Assistant Professor, Department of Education, Modoc Hall 212, Calgomia State University, Chico, CA 95929- 0222; e-mail: wrich@csuchico.edu. His areas ojspecialixation are educational leadership and administration.

Spring 2003 33

couraged anyone (scientist, town mayor, or peasant) from speaking up with a point of view that would criticize the Chairman’s policies. The communists believed such reactionaries were bene- fiting somehow from the sta.tus quo and, therefore, could not be trusted to carry the torch of the revolution (Becker, 1996).

It is not agreat stretch to relate Mao’s disastrous and faulty centralized con- trol of crop management to California’s recent disastrous National Assessment of Educational Progress (NAEP) scores in science for fourth and eighth graders for the year 2000. Ranking only above Mississippi, Louisiana, and Hawaii, Cal- ifornia’s scores reflected its high-stakes testing policy. Teachers are teaching the tested areas, not science, because it is not on the high-stakes test. Not sur- prisingly, in schools all over California teachers are doing exactly what the lit- erature says they do in such situations. Increased scores may be the result of better test-taking skills, not more howl- edge (Hambleton, 1997). Additionally, teachers do not focus on learning but on whatever it takes to avoid low scores (Smith, Edelsky, Draper, Rottenberg, & Cherland, 1990). In the name of ini- proved education, out come the pots and pans. The high-stakes standardized achievement tests, made into public ranking instruments for humiliation or rewards, provide the beating pots and pans that keep teachers away from flights of imaginative and construc- tivist curriculum, and focused on nar- row, test preparation curriculum for their students.

Hold that Line The accountability movement is based on a business model and ideology which asserts that our national economic com- petitiveness and our students’ abilities to compete for jobs is based on the per- formance of our schools as measured by standardized achievement tests (Odden, 1995). Framed in this manner, account- ability and high-stakes standardized achievement testing is really a matter of national defense. The Japanese, Germans, and other competitors who supposedly do better on such tests give us cause for great worry (Bracey, 2001). It is true that our national wealth stems in large part from the values, beliefs, skills, and knowledge of our people. But is putting all our economic defense eggs

into the basket of high-stakes standard- ized testing the best way to approach this problem? Again historical analogy can help frame our understanding,

Having been ravaged by the Germans in World War I, the French decided they would protect their territory and never again allow the hated “Hun” to cross the border into France. Accordingly, the highest policymakers prepared a defense structure that like our current tests was “. . . . surely the most meticulously developed, carefully constructed. . , .” (Cizek, p. 25) in the history of France. This miracle of modern technology, named the Maginot Line after the French Minister of War, Andre Maginot, is leg- endary for its failure. Great concrete tunnels were built to house troops safely underground and guns were pointed east to stop any attack. Unfortunately, the French should have been paying at- tention to other ways to measure and de- velop their defenses. The one-way strat- egy proved obsolete. At the beginning of the next war, the Germans simply went around and over the line with tanks, in- fantry, and dive bombers. The stationary French could not respond to a fast, mo- bile force. The French could not even turn the guns around to shoot at the enemy when it passed the line because such an occurrence was simply unimag- inable. Blind spots in the defense caused its downfall (Kaufmann & Kaufmann, 1997).

Democratic school communities ad- dress their blind spots through the in- volvement of all stakeholders to set local priorities based on the needs they see for the development of their chil- dren. Teachers and parents do not all believe that high scores on standard- ized tests alone means they are giv- ing their children an excellent educa- tion (Popham, 2001). While it is true, as Cizek (2001) reports, that the pub- lic gives approval to standardized test scores (Phelps, 1998), one could suspect that it is not the test they like, but the simplicity of the information (Eisner, 1994). It may be that they are reallygiv- ing approval to an easy way to under- stand the work of schools: give a test and rank schools from 1 to 10. If it is true that the public likes an easy method of un- derstanding how well schools are doing, then it is up to us in the educational community to develop a kind of index of educational productivity that includes both achievement tests and other kinds of measures.

Suggestions for School-Level Assessment The factors measured in an account- ability index must provide more than objective scientific information. The in- formation can serve to guide and build the coalition of communityforces, dem- ocratic ideals, and best instructional practices that will achieve the dream of eliminating the achievement gap, sig- nificantly raising overall achievement and preparing our youth to work in the economy of the present and future. On the school level, creative and dedicated educators have joined with business and community members to create schools that look like work of the future.

The recently created New Technology High School in Napa, California, exem- plifies thisvision. The assessment of stu- dents represents a departure from the test-driven, high-stakes curriculum crit- icized above. Continual evaluation is conducted by teachers and student members of collaborative groups fo- cused on eight learning outcomes: tech- nology literacy, citizenship and ethics, critical thinking, career preparation, collaboration, written communication, oral communication, and curricular lit- eracy. The curriculum is project based and the school experience for students simulates going to work in the real world. The teachers and students seem to be doing much of the same kinds of work: thinking, analyzing, cooperating, and producing (Borja, 2002).

The New Tech High School assess- ment is similar to the report card in Thornburg’s (2002) New School. The report card contains a list including ab- straction, systems thinking, collabora- tion, experimentation, comfort with am- biguity, and improvisation, to name a few. Like Thornburg, Spady (2001) also decries using standardized high-stakes tests and recommends assessing for two central roles within a life performance wheel. They are listener and communi- cator as well as searcher and learner (Spady, 2001). McCain and Jukes (2001) also recommend retargeting the educa- tional system away from content-driven learning and toward process orientation. Constructs such as measurement of problem solving, critical thinking, com- munication skills, new personal skills within changing cultures, and, most im- portant, the skill of learning will be the center of future assessment in successful schools (McCain % Jukes, 2001).

34 Educational Measurement: Issues and Practice

On a systems level, it is vital to assess the learning progress of the students. It also seems reasonable to give a report card in the areas of community health, family factors, and school/teacher ca- pacity, all of which contribute to or cre- ate barriers to the success of students and schools. Such work would be among the most productive undertaken by both researchers and school professionals (Rich, 2000).

Conclusion The lesson from Mao teaches that high- stakes policies will both skew outcomes and present high risk. If the policies are wrong, they can cause great damage such as famines and purges. In the context of education, those actually teaching may decide not to teach certain important subjects. The lesson of the Maginot Line goes to the heart of the accountability movement in which high-stakes testing has become an integral part. How can we be sure we are really measuring and as- sessing what is important for economic improvement?

Leading the charge for the account- ability movement would be a difficult role for any group, curriculum, or assess- ment system. Standardized tests were available, cheap by federal standards, and easily simplified for the public to understand when the political move- ment reached the stage to take action. Cizek (2001) is correct in stating that it is not clear that future accountability systems will contain standardized tests. The tests are most likely not up to the

task of judging the educational quality and performance of schools and teach- ers. If we know this now, wouldn’t it be wise to reduce the widespread and re- dundant use of standardized achieve- ment tests? Precious resources should be allocated to the development of mea- sures such as those recommended by Spady (200l), McCain and Jukes (2001), and Thornberg (2002). Real-world labo- ratories for new assessment exist today in schools such as the New Technology High School (Borja, 2002). Investing so deeply in ways to increase student scores on standardized tests reflects an over- confidence in this part economic, part defense strategy reminiscent of Mao and Maginot, yet leaves us equally ulnerable.

References Becker, J. (1996). Hungryghosts:Mao’sse-

cret famine. New York: Henry Holt & Co. Borja, R. R. (2002, May 29) High tech

haven. Education Week, 21 (38), 27-31. Bracey, G. W. (2001). The war against

America’spublic schools. Boston: Allyn & Bacon.

Cizek, G. J. (2001). More unintended conse- quences of high-stakes testing. Educa- tional Measurement: Issues and Practice, 20(4), 19-27.

Eisner, E. W. (1994). Theeducational imag- ination: On the design and evaluation of school programs, 3rd ed. Upper Saddle River, NJ: Prentice-Hall.

Hambleton, R. K. (1997). Measurement quality of the Kentucky Instructional Results Information System, 1991-1994. In J. Millman (Ed.), Grading teachers,

grading schools, is student achievement a valid evaluation measure? Thousand Oaks, C A Corwin Press.

Kaufmann, J. E., & Kaufmann, H. W. (1997). The Maginot line: None shall pass. West- port, CT: Greenwood Publishing.

McCain, T., & Jukes, I . (2001). Windows on the future, education in the age of technology. Thousand Oaks, CA: Corwin Press.

Millman, J. (Ed.). (1997). Grading teachers, grading schools, is student achievement a valid evaluation measure? Thousand Oaks, CA Corwin Press.

Odden, A. R., (1995). Educational leadership for America’s schools. New Yorli: McGraw- Hill.

Phelps, R. P. (1998). The demand for stan- dardized student testing. Educational Measurement: Issues and Practice, 17( 3) ~

Popham, W. J. (2001) The truth a6out test- ing, a n educator’s call to action. Alexan- dria, VA: Association for Supervision and Curriculum Development.

Rich, W. M. (2000). Accountability) a case study. Unpublished doctoral dissertation, University of Southern California, Rossier School of Education, Los Angeles.

Smith, M. L., Edelsky, C., Draper, K., Rottenberg, C., & Cherland, M. (1990). The role of testing in elementary .schools. Los Angeles: University of California, Los Angeles, Center for Research on Educa- tional Standards and Student Tests, Graduate School of Education.

Spady, W. (2001). Beyond counterfeit re- fomzs. Lanham, MD: Scarecrow Press, Inc.

Thornburg, D. (2002). The new basics, edu- cation and the future of work in the telematic age. Alexandria, V A Associa- tion for Supervision and Curriculum Development.

5-23.

Spring 2003 35