Large Scale Assessment in the 1970’s

Edward RoeberMichigan State University

Local Assessment in the 70’so Local districts administered NRTs at virtually

every elementary and junior high grade levelo Districts tested because ‘good districts should

test their students annually….’o Results used to look at trends in performanceo Results not used for instructional improvemento Districts often participated in “state testing

programs,” run by the state or a state university

State Assessment in the 70’so Some states began to assess students with

state-selected or state-developed assessment instruments

o Originally, individual student instructional improvement was the goal

o As scores got publicized, however, school (and then educator) accountability emerged as an assessment purpose

State Assessment in the 70’so During the 70’s, NAEP was new and there was

considerable curiosity about it and its scoreso Some states (CT, MN, ME, others) ran

“piggyback” assessments in which they administered some or all of the NAEP assessments to state samples of students for comparative purposes

o Some states embedded released NAEP items in their assessment programs

State Assessment in the 70’so During the 1970’s, less than one-half of the

states had a state assessment programo Two important trends occurred during this time:

• Statewide implementation of CRTs• High school graduation testing

o Debra P v. Turlington was the landmark court case/appeal that set the stage for competency testing for high school graduation in almost half of the states today

State Assessment in the 70’so The predominant form of assessment until the

70’s was norm-referenced testso In the early 70’s, CTB and Michigan began the

creation of an objective-referenced testo This ORT morphed into a CRT since results

from an ORT are inherently difficult to interpreto CRTs were condemned by those in the testing

industry as an unreliable, invalid, fad that would soon disappear

State Assessment in the 70’so Criterion-referenced testing spread - aided by

the ESEA reauthorizations such as IASA and NCLB

o States began to assess, typically, about 3-4 grades, with assessments in math, reading/language arts, and sometimes science and social studies

o Scores were reported in terms of percent-proficient, which added accountability pressures on schools

State Assessment in the 70’so Because of the pressures on states and the

new programs, the Association of State Assessment Programs, a virtual organization, was formed in 1977

o Co-chaired by Tom Fisher and Ed Roebero The group met twice yearly and served as an

induction for a number of state assessment staff and states until the late 1990’s

LSAC Conference History

o First Conference - 1971• Sponsored by ECS• Boulder, CO• Engineering College• Attended by about 50-75 persons• Focused on NAEP exclusively

• National NAEP assessments• Assessment methodologies• Piggy-back state assessments

Conference History - 1974

o Conference shifted to the Harvest House hotel itself

o 100 attendeeso Three-day conferenceo Still focused almost entirely on the NAEP

project, but there were a few state sessionso States began to complain about conference

emphasis on NAEP

Conference History - Late 1970’s

o By late 1970’s, assessment directors still were complaining that the conference focused too much on NAEP

o Past attendees became part of the conference planning

o Assessment directors became part of the conference session selection

Conference History

o The conference (number 0) that did not count• Ann Arbor, MI• 1970• Focused on NAEP exclusively• SEAs were the only attendees

Conference Sites

Numbers Years City

0 1970 Ann Arbor, MI

1-6 1971-1976 Boulder, CO

7 1977 Denver, CO

8-20 1978-1990 Boulder, CO

21 1991 Breckinridge, CO

22 1992 Boulder, CO

Conference Sites

Numbers Years City

23-24 1993-1994 Albuquerque, NM

25-26 1995-1996 Phoenix, AZ

27-28 1997-1998 Colorado Springs

29-30 1999-2000 Snowbird, UT

31 2001 Houston, TX

32 2002 Palm Desert, CA

33 2003 San Antonio, TX

Conference Sites

Numbers Years City

34 2004 Boston, MA

35 2005 San Antonio, TX

36 2006 San Francisco, CA

37 2007 Nashville, TN

38 2008 Orlando, FL

39 2009 Los Angeles, CA

40 2010 Detroit, MI

Conference Site Criteria (Roeber)

o Ideally, site should be located in the Mountain time zone; some sites in the Pacific time zone may also work.

o Can participants leave the site at noon on the final day, get to the airport and return to the east coast (e.g., Washington, DC) the same day?

o Site should be located near or in mountainous areas. o Big cities are a last choice.

o Daytime temperature should not normally exceed 90 degrees.

Conference Site Criteria (Roeber)

o Site should be near a major airport, served with jet service non-stop from the major seven airline company’s main hubs

• (American: Chicago/Dallas; Continental: Cleveland/Newark/ Houston; Delta: Cincinnati/Dallas/Salt Lake City; Northwest: Detroit/Memphis/Minneapolis; TWA: St. Louis; United: Chicago/Denver; U.S. Air: Pittsburgh/Charlotte/Indianapolis). Service by a discount airline such as Southwest, Kiwi, Western Pacific, and so forth is an added plus.

o Six non-stop connections from two or more airline companies (not their commuter affiliates or subsidiaries) is a minimum.

Conference Site Criteria (Roeber)

o The site should be no more than a one-hour drive from the airport, with frequent shuttle service (twice an hour minimum) at reasonable cost ($25 per person round trip).

o Return service, when many participants may be leaving at the same time, is also important.

Hotel(s)o Ideally, all sleeping rooms should be located in one

hotel or hotels no more than 2 blocks away from each other.

Conference Site Criteria (Roeber)

o The hotel should have space for an opening session (700 persons classroom or luncheon), four-five large-group sessions (200 each classroom), and eight-ten small group sessions (100 persons each classroom)

o The vast majority (90%) of rooms should not exceed the Federal rate of $120/night; the rooms should be good quality and quiet.

o Catering costs should be reasonable: under $10 (++) for buffet breakfast, under $15 (++) for catered lunch, and under $25 (++) for catered dinner.

Conference Site Criteria (Roeber)

o There should be at least two restaurants on-site, and they should offer reasonably priced lunches and dinners; “reasonable” is defined as the Federal travel rate for the city.

o Fast service for lunch is a plus.o The hotel should provide good inside and outside fitness

opportunities.o Conference registration area should be near the sessions,

adjacent to the office. o On-site audio-visual service is a plus.

o There should be plenty of space near the opening and large-group sessions for the group to gather and meet informally.

Conference Site Criteria (Roeber)

Communityo There should be restaurants not connected with the hotel(s) within

walking distance of the hotel. o Ideally, some of these should be high quality ones.

o There should be one or more interesting places to hold off-site functions to be sponsored by vendors.

o There should be high quality/unique places for vendors to take their clients.

o There should be interesting sightseeing opportunities for adults and for families in the area of the conference.

Conference Social Activities

o In the early years• No conference-wide, company-sponsored social

activities• Opening night BBQ on a dude ranch• Good restaurants in Boulder used for vendor dinners

o In early 1980’s• First conference-wide social activity was a few

bottles of wine and a few bags of chips time after the sessions were over each day (MRC was the leader)

Conference Social Activities

o By late 1980’s• Vendors were competing with one another for elegant events• Had to set up a “pecking order” for sponsors• A day was added to the conference so that more events could

be held• On some evenings, there were two or more events• Sample events - elegant events, dances, musical groups• No need to buy breakfast, lunch or dinner for entire week

o In 1990’s• Companies cut back - response to perceived anti-NRT tenor of

the conference• Larger vendor dinners or events for clients only

Innovations Presented as LSAC

o The conference has been a place where new ideas, controversial issues, and state innovations have been presented and discussed. For example, in the 1970s:

• John Cannell• Rasch• State NAEP• FERPA• Truth in Testing

Topics Covered in the 1970’so NAEP in generalo NAEP exercise developmento NAEP secondary researcho NAEP operationso Use of NAEP resultso NAEP piggyback assessmentso Defining mastery levels

Topics Covered in the 1970’so Evaluating an assessment programo Using and reporting state assessment

resultso FERPAo Searching for Truth in Truth in Testingo Computer softwareo Measuring change

Topics Covered in the 1970’so Teacher competency testingo Student competency testingo Assessment of bilingualso Assessment and accreditationo Indices of educational effectivnesso Affective measuremento Psychomotor assessment

Conference Speakers in 1975o Alan Morgan (NM)o Jack Schmidt (NAEP and conf. organizer)o John Adams (MN)o Dave Bayless (RTI)o Dick Hulsart (NAEP)o Ina Mullis (NAEP)o Lorrie Shepard (University of Colorado)

Conference Speakers in 1975o Wayne Martin (NAEP)o Carmen Finley (AIR)o Frank Womer (University of Michigan)o Ed Roeber (Michigan)o Evelyn Brzeszinski (NWREL)o Judy Shoemaker (USED)o Rich Hill (RMC)

Conference Speakers in 1975o Gerald Bracey (Virginia)o Walt Haney (Huron Institute)o Richard Stiggins (NWREL)o Roy Forbes (NAEP)o James Hertzog (Pennsylvania)o Ron Hambleton (Univ. of Massachusetts)o Tom Fisher (Florida)

Conference Speakers in 1975o Steve Koffler (New Jersey)o Stan Bernkopf (Georgia)o Ross Green (CTB)o Joe Ryan (University of South Carolina)o David Wright (NAEP)o Jim Impara (Oregon)o Brud Maxcy (Maine)

Reactions from Past Attendees

o I greatly valued the small group discussions that characterized the conferences of the early and middle 1970’s. There always was sufficient time for hallway conversations and personal contacts over a cup of coffee. By contrast, recent conferences have been so crowded that it is impossible to participate in all of the relevant sessions, and it is quite likely that one cannot squeeze into the rooms where the most important topics are being discussed. The old intimacy and in-depth discussions have been replaced, and I miss them.

• Tom Fisher, 1972

Reactions from Past Attendeeso I attended my first conference in 1974 when the “large” in Large

Scale only referred to the scope of the testing programs, not the size of the conference. It seems to me that there were only 100 participants back then, and everyone knew one another. Back in the “old days” the conference was always held in Boulder, Colorado. One of the receptions was usually outdoors, and it always seemed to rain just when the bar opened. For many years, CTB hosted the opening reception, although sometimes it was on the second night. These receptions were often theme events—typically related to the Wild West environment. Folks wore cowboy boots, cowboy hats, and bandanas, and the band was usually “country.”

• Michael Kean, 1974

Reactions from Past Attendeeso This conference has been the one constant activity in my

professional career. It has been an annual time to take stock professionally. I have benefited by keeping current on innovative assessment policies and practices, plus interacting with many of those active in the field. On a personal note, I have loved to watch the growth of the conference and how much it means to a number of other attendees. I have special memories of the year in Boulder that I tore cartilage in my knee and had to have surgery on it, as well as the year that it snowed the night before the conference began (while I was camping with family).

• Ed Roeber, 1974