Evaluation:Telling the Story of UIUC’s GK-12

Delwyn Harnisch, University of Nebraska, Lincoln

Chip Bruce, University of Illinois, U-C

Sharon Comstock, University of Illinois, U-C

Jim Buell, University of Illinois, U-C

Why evaluate?To document for the

National Science Foundation:

Collaboration between the academic and K-12 education communities

Enriched learning opportunities for K-12 students

Meaningful technology integration that has demonstrable impact

What is “evaluation?”*

Idea of “quality”Differing subjective views of quality not

invalidatingUnique meanings in unique settings;

“situated”*from What is Evaluation? by Chip Bruce, et al. Inquiry Unit:

http://www.inquiry.uiuc.edu/bin/update_unit.cgi?command=select&xmlfile=u10377.xml

In particular, see the uploaded video of Prof. Bob Stake, Director of The Center for Instructional Research and Curriculum Evaluation at UIUC, and a critical scholar in qualitative case and evaluation studies. See also, Stake, R. E. (1995). The Art of Case Study Research. Thousand Oaks: Sage Publications.

What is “evaluation?”

The term’s meaning to educators: “Evaluation” occurs constantly in the

classroom. There is an intuitive sense of what is “working” moment to moment

“Evaluation” as a term conjures formal assessment with defined criteria (grading, testing, job performance, etc.)

What is “evaluation?”

The Evaluation Team’s approach: Informal

Examines the individual and the complexity of a situation (situational, contextual)

FormalFormalized and invites comparison across

groups

Evaluation:Understanding Context

Observation

Unstructured interviews

Formal online surveys: http://gk12.ncsa.uiuc.edu/evaluation.html Intake (Fellows)Mid-year (Fellows)End-of-year (Fellows, teachers, mentors)

What is “evaluation?”

Evaluation as a new way of thinking about information technologies

As process, not product (balancing “result”)Less about the tools, and more about the

cultures in which these technologies are used

Site “stories” this yearCivil War comes alive for general historySupercomputing apps for general physics classesChemistry connections with real-world entomology“Animated” understanding: writing new curriculum for another departmentModeling college as reachable for studentsCritical thinking and advanced problem solving key

What is “evaluation?”

“Responsive Evaluation” of complexityProgressive focusing, emergent themesTriangulation for validationFormative—informing project

Emerging Pictures: 2001/02 - 2003/04

Fellow-teacher partnerships positively impact their respective learning ecologiesCollaboration inspires creativity = authentic learning experiencesVisualization assists students in grasping conceptsInquiry- and problem-based learning = genuine experience

What is “evaluation?”

Evaluation is a movement from

naïve, early perceptions toward

subtle, sophisticated understanding.

Selected resourcesBruce, B. C., & Rubin, A. (1993). Electronic Quills: A situated evaluation

of using computers for writing in classrooms. Hillsdale, NJ: Lawrence Erlbaum.

Bruce, B. C. (1999). Challenges for the evaluation of new information and communication technologies. JAAL, 42(6):450--455. (see online: http://www.readingonline.org/electronic/jaal/March_Column.html )

Educational Technology and Society-- special issue on Evaluation of Learning Technology.

Frechtling, Joy, & Sharp, Laure (1997, August). User-friendly handbook for mixed method evaluations. Washington, DC: National Science Foundation.

Quiñones, Sherri, & and Kirshstein, Rita (1998, December). An educator's guide to evaluating the use of technology in schools and classrooms. Washington, DC: U.S. Department of Education, Office of Educational Research and Improvement.

GK-12 Publications Comstock, S.L., Gabric, K., Harnisch, D., Bruce, B.C., and Buell, J. “Revitalizing the High School Curriculum:  Visualization/Modeling

Technologies and Inquiry,” The National Center for Information Technology in Education (NCITE) at the University of Nebraska Lincoln (UNL), June 16-18, 2004.

Gabric, K., Comstock, S.L., Moore, S., Harnisch, D., Shope, R. Bruce, B. C. & Buell, J. What Science and Technology Mean to the High School Learner: Impact of the NSF GK-12 Society for Information Technology & Teacher Education Intl. Conf. Annual (15th Annual Conf., March 1-6, Atlanta, GA), XX.

Harnisch, D. Comstock, S.L., Bruce, B.C., & Buell, J. Development of Professional Learning Communities: Factors within the NSF GK-12 Program Society for Information Technology & Teacher Education Intl. Conf. Annual (15th Annual Conf., March 1-6, Atlanta, GA), XX.

Buell, J Harnisch, D. Bruce, B.C Comstock, S.L., ., & Braatz, R. New Tools Supporting New Partnerships: Technology Development within the NSF GK-12. Society for Information Technology & Teacher Education Intl. Conf. Annual (15th Annual Conf., March 1-6, Atlanta, GA), XX.

Comstock, S.L. "Making the Case for Story: Learning Technologies in K-12 Environments," Midwest Qualitative Research Conference, University of St. Thomas (9th Annual Conf. June 18-20, 2003, Minneapolis, MN), 10.

Harnisch, Delwyn; Comstock, Sharon L.; Thakkar, Umesh; Moore, Steve; Bruce, Bertram C.; Jakobsson, Eric; Abbott, Gypsy; & Gabric, Kathy. "Scientists Becoming Teachers: Lessons Learned from Teacher Partnerships," Society for Information Technology & Teacher Education Intl. Conf. Annual (14th Annual Conf., March 24-29, Albuquerque, NM), 79.

Comstock, Sharon L., Bruce, Bertram, & Harnisch, Delwyn (2003). "Inquiry Page: Collaborative Technology into Practice," Society for Information Technology & Teacher Education Intl. Conf. Annual (14th Annual Conf., March 24-29, Albuquerque, NM), 43.

Comstock, S., Bruce, B.C., Harnisch, D., & Mehra, B., "Fostering Inquiry-based Learning in Technology-rich Learning Environments: The Inquiry Page in the GK-12 Fellows Program," Proceedings of the AACE World Conference on Educational Multimedia, Hypermedia & Telecommunications, Denver, CO, (June 24-28, 2002).

Comstock, S., Harnisch, D., Bruce, B.C., & Mehra, B., "Who's Teaching Whom? Scientist/Teacher Teams and Administrative Strategies," Proceedings of the AACE World Conference on Educational Multimedia, Hypermedia & Telecommunications, Denver, CO, (June 24-28, 2002).

Mehra, B., Bruce, B.C., Harnisch, D., and Comstock, S. "Complexities in Development of Educational Collaboratories as a Socio-Technical System: A Situated Study of the GK-12 EdGrid Graduate Teaching Fellowship Program." In: N. Callaos, M. Loutfi, and M. Justan (editors), Proceedings of The 6th World Multi-Conference on Systemics, Cybernetics and Informatics SCI 2002, Volume VIII Concepts and Applications of Systemics, Cybernetics and Informatics II, Orlando, Florida, (July 14-18, 2002), pp. 188-194.