CIRTL Class Meeting 9: Teaching as research

The College Classroom – Spring 2015

Class Meeting 9: Teaching as Research

Dave Gross dgross@ biochem.umass.edu

Thursday, March 26, 2015

1:00-2:30p ET, 12:00-1:30p CT, 11:00a-12:30p MT, 10:00-11:30a PT

Peter Newbury

[email protected]

@polarisdotca

Vocabulary check: SoTL and TAR

Teaching as research - collegeclassroom.ucsd.edu 2

research done by you

in your class to inform how

you will teach in the future

carefully

designed experiment

to explore learning

paper

in journal

Scholarship of Teaching and

Learning (SoTL),

Education Research

teaching as

research

Teaching as Research (TAR)


At the core of improving teaching and learning is the need

to accurately determine what students have learned as a result of

teaching practices. This is a research problem, to which STEM

instructors can effectively apply their research skills and ways of

knowing. In so doing, STEM instructors themselves become the

agents for change in STEM teaching and learning.

Teaching-as-Research involves the deliberate, systematic,

and reflective use of research methods to develop and implement

teaching practices that advance the learning experiences and

outcomes of students and teachers.

CIRTL Network [1]

Some of your Education journals


Institution of Chemical Engineers Education for Chemical Engineers

American Chemical Society Journal of Chemical Education

Academy of Management Academy of Management Learning &

Education (AMLE) (Journal)

American Geophysical Union Journal of Geoscience Education (under

National Association of Geoscience Teachers)

INFORMS INFORMS Transactions on Education

IOP Science Physics Education

Certified Hospitality Educator International Journal of Hospitality

Management

American Society for Cell Biology Life Sciences Education

Society for Neuroscience Journal of Undergraduate Science Education

Categories of Educational Research[2]


Action or practitioner research

Experimental

'Cause and effect' research

Categories of Educational Research[2]


Theoretical research

Action or practitioner research

Evaluative Experimental

'Cause and effect' research

Case study

Systematic review

Exploratory

Comparative

Grounded theory

Ethnography

What is the value of course-specific learning goals?

Simon & Taylor (2009)


3 classes (A = computer literacy Fa07, B = computer

literacy Sp08, C = microbiology Sp08)

Last week of course (Wk 13): students asked to

complete up to five copies of, “For me, the use of

learning goals in this course is . . .”

Comments put into categories using content-analysis

based coding

A B C total

Comments 225 252 120 597

Students 59 76 51 186

(Evaluative)

What is the value of course-specific learning goals?

Simon & Taylor (2009)


Research informs the

instructors about

how students are

responding

what’s working (or not)

what to use again (or

not)

Helps instructors become

better educators.

Active learning increases student performance

in science, engineering and mathematics[4]


Meta-analysis of 225 research studies that explored the

impact of active learning:

Active learning engages students in the process of learning

through activities and/or discussions in class, as opposed to

passively listening to an expert. It emphasizes higher-order

thinking and often involves group work.

(Freeman et al., pp 8413-8414)

(Systematic review)


Active learning increases student performance

in science, engineering and mathematics[4]

established active learning

has positive impact on

learning

advances field of education

research: no need to

continue to replicate study

(in STEM)

What can you study?


Students’ attitudes

what are they initially? (e.g., what are students’ views on academic integrity?)

how did they change after you did X? (e.g., what do students think about learning goals?)

Students’ knowledge and skills

what are students able to do now that they couldn’t do

before taking the course?

are students thinking more like experts? (e.g. what questions do they ask in lab/discussion?)

Common education research tools

Quantitative Qualitative

Teaching as research - collegeclassroom.ucsd.edu

What is happening? Why is it happening?

Multiple choice tests (pre- and post-testing?)

Attitude surveys (Likert SD, D, N, A, SA scale)

mini-writing (muddiest point, think-pair-share, etc.)

document students’

answers on a worksheets responses to a question

opinions/views peer instruction votes questions

12

Can You Solve This?


As you watch this video,

Play along with the Derek’s

puzzle

Watch from an educator’s

point-of-view: when did

learning happen?

www.youtube.com/watch?v=vKA4w2O61Xo

See you back here in 5 minutes.

Can You Solve This?


Beware of confirmation bias: the

tendency to search for, interpret,

or recall information in a way that

confirms one's beliefs or

hypotheses.[8]

(Educator’s point-of-view: When did learning happen?)

How do you think the students felt?


Suppose you were a student in one of the active learning

classes included in the meta-analysis by Freeman et al.[4]

Do you think that research would have any impact on your

experiences and/or success in that class?

A) yes

B) no

How do you think the students felt?


Suppose you were a student in one of the Biology or

Computer Science classes asked to complete the sentence,

“For me, the use of learning goals in this course is . . .” for

the study by Simon and Taylor about the value of course-

specific learning goals.

Do you think that experiment would have any impact on

your experiences and/or success in that class?

A) yes

B) no

Imagine this…


Professor Smith teaches 2 sections of the same course.

Section 1 is an active classroom using peer instruction

(clickers). Section 2 is a traditional lecture. Both Sections

take the same final exam. Professor Smith analyzes the

students’ grades to investigate the impact of active learning.

Do you think this experiment had any impact on the success

and/or experiences of students in Section 1, Section 2?

A) yes, yes

B) yes, no

C) no, yes

D) no, no

If you suspect your research could impact students physical

well-being, privacy, or success, you should speak with your

IRB. The IRB may insist you apply for approval or

suggest you apply for an exemption from IRB review.

Humans are involved in TAR


Mission of the Institutional Review Board (IRB):

1. Protection of human subjects from physical harm.

2. Protection of students’ privacy and students’ success.

IRB at your institution?


Take 5 minutes to find

1. the name of the organization that handles human

subjects research approval (it’s probably not the “IRB”)

2. what research activities are exempt from IRB approval?

3. which exemption(s) do you think apply to teaching-

as-research? (you’re going to copy-and-paste that

exemption onto our whiteboard later so keep the

window/tab open)

See you back here in 5 minutes.

Check one:

“exceptions” “exemptions”


Use the textbox tool to copy-n-paste

an exemption applicable to TAR


UC San Diego IRB

grants exemptions to…


(1) Research conducted in established or commonly accepted educational

settings, involving normal educational practices, such as (i) research on

regular and special education instructional strategies, or (ii) research on

the effectiveness of or the comparison among instructional techniques,

curricula, or classroom management methods. [5]

Things a scholarly, reflective instructor

might and should do anyway.

Involve your Center for Teaching


Before you apply for an exemption, check with your local

Center for Teaching. (At UC San Diego, we have a blanket

exemption for projects studying active learning, as long as

the Center is involved.)

A full inquiry cycle of TAR


1. Learning foundational knowledge

2. Creating objectives for student learning

3. Developing an hypothesis for practices to achieve the learning objectives

4. Defining measures of success

5. Developing and implementing teaching practices within an experimental design

6. Collecting and analyzing data

7. Reflecting, evaluating, and iterating

www.cirtl.net/CoreIdeas/teaching_as_research

Teaching as Research case study


Watch for the steps in the inquiry cycle in Dave’s

Biochem/Chem 471 research project. (Perhaps keep the

CIRTL webpage open and visible?)

At the end, together we’ll try to identify the inquiry cycle

in Dave’s project.

The course

Teaching as Research - collegeclassroom.ucsd.edu 26

Biochem/Chem 471

Elementary Physical Chemistry

• Upperclass majors

• Several prerequisites

AY #

students

#

sections

# meetings/

week

# minutes/

week Active Exam score

2007-8 79 1 3 150 N 63.3 ± 15.8a

2008-9 88 1 3 150 N 70.5 ± 14.0a

2009-10 78 1 3 150 N 72.4 ± 15.1

2011-12 111 3 1 75 Y, clickers,

peer 77.0 ± 14.8b

2012-13 133 2 2 100 Y, TBL 75.8 ± 15.1b

Evolution of Chem/Biochem 471


Fall 04 Half exams in class (78%) & Final (22%)

Standard lecture format

Fall 06 Midterm and final (25% ea)

Online OWL quizzes (20%)

Online OWL homework (30%)


Fall 07 Two exams (40%) & Final (20%)

Online OWL quizzes (10%)

Online OWL homework (25%)

Online OWLBook examples (5%)


Results - assessments


0

4

8

12

16

1 2 3 4

Pe

rce

nt

ch

ang

e

GPA quartile

Exam 1

Exam 2

Exam 3

45

55

65

75

85

Exam 1 Exam 2 Exam 3

Ex

am a

ve

rag

e

Pre

Post

Results – use of online materials


1

10

100

1000

20 40 60 80 100

Nu

mb

er

of

vid

eo

ac

ce

sse

s

Exam average

1st quartile

2nd quartile

3rd quartile

4th quartile

0.0

0.1

0.2

0.3

0.4

0.5

-2 -1 0 1 2

Fra

cti

on

of

vid

eo

ac

ce

sse

s

Week from exam

Top quartile

2nd quartile

3rd quartile

4th quartile

Timeline


Fall 04

Fall 06

Standard class

Introduce online homework

and online quizzes

Fall 07 Introduce online interactive textbook

Spring 12 Multiple sections, active classroom,

collaborative learning

Spring 13 Team-based classroom, peer evaluations

Timeline


Fall 04

Fall 06

Standard class


and online quizzes


Spring 12 Flip class, active classroom,



Fall 01 TEACHnology Fellow

TBL Fellow

Blended Learning Fellow

National Academies Summer Institute Summer 10

Timeline

32

Fall 04

Fall 06

Standard class


and online quizzes


Spring 12 Flip class, active classroom,



Fall 01 TEACHnology Fellow

TBL Fellow

Blended Learning Fellow

National Academies Summer Institute Summer 10 Hypothesize improved outcomes

Summer 11

Accepted with revisions, CBE

Life Science Education

Summer 13

Summer 13

Spring 15

Timeline


Hypothesize improved outcomes with the flipped class

Plan with M. Graham (data needed, analysis)

Fall 10

Accepted with revisions, CBE Life Science Education

Summer 11

Spring 12

Spring 15

Collect flipped data, initial analysis

Presentations (SABER - feedback) Summer 12

Spring 13 Collect round 2 data, obtain IRB approval, recruit data miners (GPA, SAT, OWL)

Revisit plan with M. Graham Fall 12

Summer 13 Recruit statistician, do deep analysis

Summer 14 Manuscript submission (3 X)

1. foundational knowledge 2. objectives 3. hypothesis for practices 4. defining success

5. designing and teaching 6. collect & analyze data 7. reflect/evaluate/reiterate/(share)

Next class meeting (our last one!)

Thursday, April 10

Watch the blog and email for information about

Growth and Fixed Mind

planning the first day of class

microteaching project and presentations

Before next week, all class meeting resources will be available on the

blog and you’ll have access to session recordings if you need them.


References


1. Center for the Integration of Research, Teaching and Learning (2010) Teaching as Research.

www.cirtl.net/CoreIdeas/teaching_as_research

2. Lambert, M. (2012). A Beginner's Guide to Doing Your Education Research Project. Thousand Oaks, CA: SAGE

Publications Inc. via Tomorrow’s Professor cgi.stanford.edu/~dept-ctl/tomprof/posting.php?ID=1233

3. Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning Goals? J. College Science Teaching, 39, 2,

52-57.

4. Freeman, S., Eddy, S.L. McDonough, M., Smith, M., Okoroafor, N., Jordt,. H. & Wenderoth, M.P. (2014) Active

learning increases student performance in science, engineering, and mathematics. PNAS 111, 23, 8410–8415.

5. Exemption from IRB Review, UCSD Human Research Protections Program, retrieved from

irb.ucsd.edu/Exempt_forms.shtml 24/2/ 2014.

6. Adams, W.K., Perkins, K.K., Podolefsky, N.S., Dubson, M., Finkelstein, N.D., & Wieman, C.E. (2006) A new

instrument for measuring student beliefs about physics and learning physics: The Colorado Learning Attitudes about Science

Survey (CLASS). Phys. Rev. ST Phys. Educ. Res. 2, 010101.

7. Confirmation bias (n.d.). In Wikipedia. Retrieved February 16, 2015, from

http://en.wikipedia.org/wiki/Confirmation_bias