Designing Valid Assessments of Student Learning

Rodolfo Rincones

CETaL Workshop # 1

Enhancing Student LearningUTEP Library - Blumberg Auditorium

September 3, 2009

Outline

Context of assessment in higher education

Purpose of assessment

Linking learning and assessment

Types of assessment

Activity

What we know ( 5 minutes)

What we want to know ( 5 minutes)

What we learned (at the end 5 minutes)

Context of Higher Education

Decreasing public funding sources

Tight budgets

Expanding pressures to do more with less

Escalating tuition

Public trust

Increase productivity

Accountability

Context of Higher Education

Factory-production model based on semester credit hours, certification, transfer, articulation, and “student success”

Limited records that reflect actual student learning outcomes

Paradigm shift from teaching to learningDonohue-Lynch, B. Assessment with 21st Century Tools

http://www.carnegiefoundation.org/perspectives/sub.asp?key=244&subkey=2140

Are we assessing student learning to appease a growing tide of legislators and government executives who increasingly question the full impact of colleges and universities on student learning?

or

Are we assessing student learning to improve our own educational practices, curriculum choices, and instruction?

McKitrick, Sean , Assistant Provost for Curriculum, Instruction, & Assessment at Binghamton University (SUNY)

Undergraduate education in research universities requires renewed emphasis on a point strongly made by John Dewey almost a century ago: Learning is based on discovery guided by mentoring rather than on the transmission of information.

Inherent in inquiry-based learning is an element of reciprocity: Faculty can learn from students as students are learning from faculty

The Boyer Commission on Educating Undergraduates in the Research University, 1998

Today’s Learning

Today Learning in the 21st Century

Curriculum

“Curriculum is the formal and informal content and process by which learners gain knowledge and understanding, develop skills, and alter attitudes, appreciations, and values under the auspices of that school.” (Keller, 2004)

Although shared knowledge is an important component of a university education, no simple formula of courses can serve all students in our time.

Integrated educational experience

Collaborative learning experiences

Promote analysis, synthesis, and evaluation

The Boyer Commission on Educating Undergraduates in the Research University, 1998

Creating the learning environment

Teaching

Involves developing the knowledge, skill, mind, character, or ability of others. It “means not only transmitting knowledge, but transforming and extending it as well.”

“Dynamic endeavor involving all the analogies, metaphors, and images that build bridges between the teacher’s understanding and the student’s learning. Pedagogical procedures must be carefully planned, continuously examined, and relate directly to the subject taught.” (Boyer 1990)

Learner-Centered Teaching

How to tie teaching, curriculum, and objectives of learning rather than to the content delivery alone. Focuses attention on what the student is learning, how the student is learning, the conditions under which the student is learning, whether the student is retaining and applying the learning, and how current learning positions the student for future learning. (Weimer, 2002)

Purposes of educational assessment

Assessment is a comparison of achieved results to intended goals (Ewell, 1985)

TraditionalDiagnose students’ strengths and weaknesses

Monitor students’ progress

Assign a grade

Determine program and institutional effectiveness

Purposes of educational assessment

Formative - to aid learning

Summative - for review, transfer and certification

Summative - for accountability to the public

Reasons for Assessing

Shupe, D. (2006). Toward a higher standard: the changing organizational context of accountability for educational results, ON THE HORIZON, Vol. 16, No. 2 2008, pp. 72-96.

Student Learning Outcome

Student learning outcomes should refer normally to competencies or attainment levels reached by students on completion of an academic program

Courses must define two specific sets of learning objectives

Traditional knowledge and skills associated with the course

Abilities addressed in the course. (Keller, 2004)

Abilities

Communication

Analysis

Problem Solving

Valuing in Decision-Making

Integrated Developmental Transferable

Social Interaction

Global Perspectives

Effective Citizenship

Aesthetic Responsiveness

Learning Assessment

What is meant by “assessment” often varies greatly—embracing everything from job placement, through student satisfaction, to self-reported gains in skill and knowledge on the part of students and former students. (Ewell, 2001)

Types of Measures

Direct Measures- provide clear and compelling evidence of what students are learning

Course-embedded assessments, including written work and presentations scored using a rubricScores on locally designed tests and competency exams accompanied by test “blueprints” describing what is being assessedScore gains between entry and exit on tests, competency exams and writing samplesRatings of student skills in the context of class activities, projects and discussionsPortfolios of student workScores on nationally- norm instruments (Ewell, 2001)

Types of Measures

Indirect Measures-Provide signs that students are probably learning, but it is less clear exactly what they are learning.

Student grades Student evaluations and ratings of the knowledge and skills they have gainedStudent or graduate satisfaction with their learning in general education competenciesResults of nationally-norm surveys (Ewell, 2001)

Assessment Evaluation Criteria

ReliabilityConsistency with which assessment produce measures whatever it is measuring

ValidityReflects the defensibility of score-based inferences made on the basis of an educational assessment procedure

Assessment Evaluation Criteria

Absence-of-BiasDegree to which assessments are free of elements that would offend or penalize examinees on the basis of examinees’ gender, ethnicity or other characteristics (Popham, 2002)

Learning Objectives

Define a course in terms of the outcomes the instructor expects students to achieve

ComponentsA description of what the student will be able to do (verb)

The conditions under which the student will perform the task

The criteria for evaluating student performance

Importance of Learning Objectives

Selection of content

Development of an instructional strategy

Development and selection of instructional materials

Construction of tests and other instruments for assessing and then evaluating student learning outcomes

Efficient use of instructional time

Ongoing improvement and student feedback

Example

Objective: Given a set of data the student will be able to compute the standard deviation.

Condition - Given a set of data

Behavior - the student will be able to compute the standard deviation.

Criterion - the number computed will be correct

Course Assessment andEnhancement Model

Combs, K., et al.(2008). Enhancing curriculum and delivery: linking assessment to learning objectives. Assessment & Evaluation in Higher Education, Vol. 33, No. 1, pp. 87–102.

Your turn….

Connecting curriculum elements

Thank you!!!