RESEARCH TO PRACTICE IN HIGHER EDUCATION: USE THE RESULTS OF RECENT RESEARCH TO INFORM HOW YOU DOCUMENT AND EVALUATE STUDENT LEARNING

Wendy Machalicek & Melinda Leko UW-Madison 2010 Teaching and Learning Symposium

MELINDA LEKO, Ph.D.Assistant ProfessorDepartment of RehabilitationPsychology and Special Educationleko@wisc.edu

WENDY MACHALICEK, Ph.D.Assistant ProfessorDepartment of RehabilitationPsychology and SpecialEducationmachalicek@wisc.edu

Agenda

① Background

② Case studies

③ Discussion

Teaching is scholarly work

• Acts of discovery, application, integration, and teaching are all "scholarships” Boyer (1990)

• Teaching and research share questioning, exploring, testing, and professing Bender & Gray (1999)

Benefits of approaching teaching in scholarly way

• Accountability• Documentation of our commitment to excellence

in teaching

• Collegiality• Brings the usually private, lonely work of

teaching into the public sphere

• Preparing students as researchers• Potential peer-reviewed publication

Best practices for assessing student learning

Special Education Teacher Education Literature

Attitudes, Dispositions, & Perceptions

Marable, Leavitt-Noble, & Grande (2010)

Evaluated how the use of nonfiction book talks affected students’ attitudes regarding disability

Qualitative thematic analyses of reflection assignment and open-ended survey

Carroll, Petroff, & Blumberg (2009)

Evaluated students’ perceptions of an undergraduate inclusive program

Qualitative modified analytic induction of post program interviews with 12 students

Best practices for assessing student learning, cont.

Special Education Teacher Education Literature

Knowledge

Miller et al. (2009) Used concept maps to assess growth in student knowledge

•Qualitative analyses on the depth and breadth of concepts included on concept maps•Quantitative analyses on the number of concepts and interrelationships

Dieker et al. (2009) Evaluated student learning of various content area instructional techniques

•Pre/post written descriptions •Pre/post questionnaires•Pre/post teacher made assessment

Best practices for assessing student learning, cont.

Applied Behavior Analysis Literature

Knowledge

Malanga & Sweeney(2008)

Compared effects of daily assessment & response cards on weekly quiz scores

Single-subject alternating treatments design

Saville et al. (2006) Compared interteaching and lecture on quiz performance

•Single-subject alternating treatments design•Pretest•Social validity questionnaire

Best practices for assessing student learning, cont.

Applied Behavior Analysis Education Literature

Knowledge

Neef, McCord, & Ferreri (2006)

Compared guided lecture notes to completed lecture notes on quiz performance

Single-subject alternating treatments design

Saville & Zinn (2009) Evaluated effects of “quality” points on exam scores

•Single-subject alternating treatments design

Purpose of today’s presentation

• Present strategies that we have used in our own classes to evaluate the effects of instructional activities on student learning/growth

• Provide opportunity for you to brainstorm potential strategies to evaluate student learning in your own courses

EVALUATING STUDENT LEARNING IN THE CONTEXT OF TEACHER PREPARATION

An Example: RPSE 479

Language & Reading Instruction for Students with Learning & Behavioral Disabilities

Desired Outcomes:

1. Knowledge of content knowledge and pedagogical content knowledge for teaching reading

2. Beliefs and Attitudes to Support

evidence-based reading practices

3. Motivation, Interest, & Engagement

Desired Outcome: Knowledge

1. Pre/post concept maps

2. Online quizzes

3. In class summative assessment

4. Culminating research paperHigher levels of Bloom’s

Lower levels of Bloom’s

Desired Outcome: Beliefs & Attitudes

Online journaling

-students respond to 6 prompts over the course of the semester

- prompts are designed to elicit students’ beliefs and attitudes about various instructional practices in reading

Desired Outcome: Beliefs & Attitudes

Online journaling

First Prompt:

Describe your philosophy for teaching reading to students with disabilities.

Last Prompt:

Revisit your original philosophy for teaching reading. What revisions would you make?

Desired Outcome: Motivation, Interest, & Engagement

Informal student evaluations

-Administered at least twice a semester

-”Describe 2 things that are going well in the course and 2 suggestions you have for improving the course.”

RPSE 330

Behavior analysis: Applications for persons w/disabilities

• In this course you will… – Become familiar with the basic principles of

human behavior as they apply to children with exceptional learning and behavior characteristics

– Apply these principles to develop, implement, and evaluate programs of behavior change in educational settings

Desired Outcomes

Knowledge, understand, apply, analyze, evaluate, and create

1. Weekly quizzes

2. Dyadic discussions

3. Video clip of key concept

4. Annotated bibliography of

intervention literature

5. In class application activities

Higher levels of Bloom’s

Lower levels of Bloom’s

Instructional design

• Adapted inter-teaching method– reciprocal peer tutoring– problem-based learning– cooperative learning– Instructor reinforcement of student

engagement

Boyce & Hineline, (2002); Saville et al., (2005)

Saville, Zinn, Neef, Van Norman, & Ferrari (2006)

Review and student assessment

• Each review session based on student feedback concerning areas of interest and/or concepts and skills that require clarification

• Following review session, students complete a short quiz of the previous week’s readings and activities

Dyadic discussion

• Students discuss assigned readings with one or more peers. Weekly discussion guides created by each student used to prompt and guide discussion.

• The professor and teaching assistant move around the room taking notes on the topic and quality of discussions, engage students in dialogue, and help students to clarify any questions, concerns, or areas of interest that arise.

• Dyads randomly called on to summarize

key points of readings

Discussion guide

• Describe the 3 main points of this week’s readings

• Describe your own experiences in regard to the topics covered

• Describe 3 ideas, concepts, or strategies you would like to discuss further

• Describe any concerns or difficulties you have with this week’s assigned readings

• Any questions?

Example from discussion guide• Own experience

“I was homeschooled by my mother up until the 5th grade and I noticed how similar her teaching style was to modeling….When my sister finally learned how to read, I was even more inclined to learn myself because I wanted to follow their behavior pattern. The same thing happening when my mom showed enthusiasm towards various tasks and projects—instead of being against her, my sister and I seemed to be enthusiastic and had a good time learning with her.”  

Brief lecture and application activity

• Brief lecture based on and expanding upon concepts covered in the readings

• One or more problem-based small group application activities

• One or more groups randomly called upon to present response to activity to entire class.

Student self-evaluation and feedback

• At end of each class, students asked to provide feedback on the quality of their discussions/completion of the application activities and to describe concepts or skills that require further clarification or practice

• Professor uses feedback guides to structure a review for the following week

 

Student self-evaluation

Instructions: Circle the one number (1-6) that best describes the quality of your discussion today. Provide a brief explanation for your rating.

“I rated the discussion I had with 2 of my peers a 5, because I felt I understood the concepts better when I got to hear the answers they came up with and compare them to what I got.”

 

Student feedback

Instructor GuideInstructions: List and/or describe any

concepts covered today that require further clarification.

“Could you go over positive and negative reinforcement and punishment more in detail?”

“Discrete versus chained target behaviors, ABC for everyday functional use (besides severe behavior)”  

 

Adapted inter-teaching

process

Review of previous week’s

material

Dyadic discussion

Brief lecture on current

week’s materials

Application activity

Student self-

evaluation & feedback

RESULTSDISCUSSION

Some key questions

• What student outcomes are of most interest to you?

• What strategies or research methodologies for evaluating student learning might you want to try?

• How will you document student outcomes?

MELINDA LEKO, Ph.D.Assistant ProfessorDepartment of RehabilitationPsychology and Special Educationleko@wisc.edu

WENDY MACHALICEK, Ph.D.Assistant ProfessorDepartment of RehabilitationPsychology and SpecialEducationmachalicek@wisc.edu