Click to edit Master title style Content, Assessment …personal.cege.umn.edu/~smith/docs/Session 3...

Click to edit Master title style

2009 Workshop forThe Committee for the Formation

of Engineers Puebla-Tlaxcala

Content, Assessment and Pedagogy(CAP): An Integrated Design Approach

Session 3 – July 2, 2009

Instructional Team:Ruth Streveler, Karl Smith & Rocío Chavela

CAP - 2

Welcome & Overview

Reflections and Session 3Participant “Think-Pair-Share” – Highlights, Insights, and Questions from Session 2

PedagogyPlanning Active and Cooperative Learning

Aligning Content, Assessment, and Pedagogy

Assignments & Next Steps

Session 3 Overview

CAP - 3

Resource: Learning that Lasts

CAP - 4

What classroom assessment entails and how it works.

How to plan, implement, and analyze assessment projects.

Twelve case studies that detail the real-life classroom experiences of teachers carrying out successful classroom assessment projects.

Fifty classroom assessment techniques

Step-by-step procedures for administering the techniques

Practical advice on how to analyze your data

Resource: Classroom Assessment Techniques

http://honolulu.hawaii.edu/intranet/committees/FacDevCom/guidebk/teachtip/assess-2.htmhttp://www.celt.iastate.edu/teaching/cat.html

CAP - 5

Exercise 1

Three volunteers that would like to share one objective and get feedback from the instructional team

CAP - 6

Exercise 2

Find a new partner (someone that is not so familiar with your course).Share one learning objective and its assessment method(s) Provide feedback to your partner

CAP - 7

Think-Pair-Share about Session 2

1. Reflect on session 2. Briefly describe your major learning, insights, and questions.

2. Explain what you think is meant by:a. Pedagogies of Engagement – Active and

Cooperative Learningb. Alignment of Content, Assessment and Pedagogy

3. Talk with other members of your group.

CAP - 8CAP - 8

Lila M. Smith

CAP - 9CAP - 9

Amnesia

Fantasia

Inertia

Lee Shulman – MSU Med School – PBL Approach (late 60s – early 70s); Stanford University, Past President of the Carnegie Foundation for the Advancement of College Teaching

Shulman, Lee S. 1999. Taking learning seriously. Change, 31 (4), 11-17.

Pedago-pathologies

CAP - 10

Shulman, Lee S. 1999. Taking learning seriously. Change, 31 (4), 11-17.

What do we do about these pathologies? - Lee Shulman

Activity

Reflection

Collaboration

Passion

CAP - 11CAP - 11

CAP - 12CAP - 12

Pedagogies of Engagement

CAP - 13CAP - 13

MIT & Harvard – Engaged Pedagogy

January 2, 2009—Science, Vol. 323 www.sciencemag.org

January 13, 2009—New York Timeshttp://www.nytimes.com/2009/01/13/us/13physics.html?em

Calls for evidence-based teaching practices

CAP - 14CAP - 14

http://web.mit.edu/edtech/casestudies/teal.html#video

CAP - 15http://www.ncsu.edu/PER/scaleup.html

CAP - 16

Cooperative Learning

Positive Interdependence

Individual and Group Accountability

Face-to-Face Promotive Interaction

Teamwork Skills

Group Processing

CAP - 17CAP - 17

Cooperative Learning (cont.)

CAP - 18

Cooperative Learning Research Support

Over 300 Experimental StudiesFirst study conducted in 1924High GeneralizabilityMultiple Outcomes

Johnson, D.W., Johnson, R.T., & Smith, K.A. 1998. Cooperative learning returns to college: What evidence is there that it works? Change, 30 (4), 26-35.

January 2005 March 2007

CAP - 19

Outcomes

1. Achievement and retention

2. Critical thinking and higher-level

reasoning

3. Differentiated views of others

4. Accurate understanding of others' perspectives

5. Liking for classmates and teacher

6. Liking for subject areas

7. Teamwork skills

CAP - 20

Faculty interest in higher levels of inquiry in engineering education

Level 0 TeacherTeach as taught

Level 1 Effective TeacherTeach using accepted teaching theories and practices

Level 2 Scholarly TeacherAssesses performance and makes improvements

Level 3 Scholar of Teaching and LearningEngages in educational experimentation, shares results

Level 4 Engineering Education ResearcherConducts educational research, publishes archival papers

Source: Streveler, R., Borrego, M. and Smith, K.A. 2007. Moving from the “Scholarship of Teaching and Learning” to “Educational Research:” An Example from Engineering. To Improve the Academy, Vol. 25, 139-149.

CAP - 21

Active Learning: Cooperation in the College Classroom

Informal Cooperative Learning Groups

Formal Cooperative Learning Groups

Cooperative BaseGroups

See Cooperative Learning Handout (CL College-804.doc)

CAP - 22

Cooperative Learning

Positive interdependence(all members must cooperate to complete the task)

Individual and group accountability (each member is accountable for the complete final outcome)

People working in teams to accomplish a common goal

CAP - 23

Key Concepts

Positive Interdependence

Individual and Group Accountability

Face-to-Face Promotive Interaction

Teamwork Skills

Group Processing

CAP - 24

Individual & Group Accountability

CAP - 25CAP - 25

http://www.ce.umn.edu/~smith/docs/Smith-CL%20Handout%2008.pdf

CAP - 26

Book Ends on a Class Session

CAP - 27

Advance Organizer

“The most important single factor influencing learning is what the learner already knows. Ascertain this and teach him accordingly.”

David Ausubel - Educational psychology: A cognitive approach, 1968.

CAP - 28

Book Ends on a Class Session

1. Advance Organizer

2. Formulate-Share-Listen-Create (Turn-to-your-neighbor) -- repeated every 10-12 minutes

3. Session Summary (Minute Paper)

1. What was the most useful or meaningful thing you learned during this session?

2. What question(s) remain uppermost in your mind as we end this session?

3. What was the “muddiest” point in this session?

CAP - 29

Advance Organizer

“The most important single factor influencing learning is what the learner already knows. Ascertain this and teach him accordingly.”

David Ausubel - Educational psychology: A cognitive approach, 1968.

CAP - 30

Quick Thinks

Reorder the stepsParaphrase the ideaCorrect the errorSupport a statementSelect the response

Johnston, S. & Cooper,J. 1997. Quick thinks: Active- thinking in lecture classes and televised instruction. Cooperative learning and college teaching, 8(1), 2-7

CAP - 31

Formulate-Share-Listen-Create

FOCUS QUESTION

1. Formulate your response to the question individually2. Share your answer with a partner3. Listen carefully to your partner's answer4. Work together to create a new answer through

discussion

Informal Cooperative Learning GroupIntroductory Pair Discussion of a

CAP - 32

Minute Paper

What was the most useful or meaningful thing you learned during this session?

What question(s) remain uppermost in your mind as we end this session?

What was the “muddiest” point in this session?

Give an example or application

Explain in your own words . . .

Angelo, T.A. & Cross, K.P. 1993. Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey Bass.

CAP - 33

Session Summary (Minute Paper)

Reflect on the session

1. Most interesting, valuable, useful thing you learned.

2. Things that helped you learn.3. Question, comments, suggestions.4. Pace: Too slow 1 . . . . 5 Too fast5. Relevance: Little 1 . . . 5 Lots6. Instructional Format: Ugh 1 . . . 5 Ah

CAP - 34

MOT 8221 – Spring 2009 – Session 1

Q4 – Pace: Too slow 1 . . . . 5 Too fast (3.3)Q5 – Relevance: Little 1 . . . 5 Lots (4.2)Q6 – Format: Ugh 1 . . . 5 Ah (4.4)

0

5

10

15

20

25

Q4 Q5 Q6

12345

CAP - 35

Informal CL (Book Ends on a Class Session) with Concept Tests

PhysicsPeer InstructionEric Mazur - Harvard – http://galileo.harvard.eduPeer Instruction – www.prenhall.comRichard Hake – http://www.physics.indiana.edu/~hake/

ChemistryChemistry ConcepTests - UW Madison

www.chem.wisc.edu/~conceptVideo: Making Lectures Interactive with ConcepTestsModularChem Consortium –

http://mc2.cchem.berkeley.edu/

CAP - 36

Informal CL (Book Ends on a Class Session) with Concept Tests

STEMTECVideo: How Change Happens: Breaking the

“Teach as You Were Taught” Cycle – Films for the Humanities & Sciences –www.films.com

HarvardThinking Together & From Questions to

Concepts Interactive Teaching in Physics: Derek Bok Center –www.fas.harvard.edu/~bok_cen/

CAP - 37

<g> = Concept Inventory Gain/Total

CAP - 38

CAP - 39

The “Hake” Plot of FCI

Pretest (Percent)

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

20.00 30.00 40.00 50.00 60.00 70.00 80.00

ALS

SDI

WP

PI(HU)

ASU(nc)

ASU(c)HU

WP*

UMn Traditional

XUMn Cooperative Groups

XUMn-CL+PS

CAP - 40

Physics (Mechanics) Concepts:The Force Concept Inventory (FCI)

A 30 item multiple choice test to probe student's understanding of basic concepts in mechanics.

The choice of topics is based on careful thought about what the fundamental issues and concepts are in Newtonian dynamics.

Uses common speech rather than cueing specific physics principles.

The distractors (wrong answers) are based on students' common inferences.

CAP - 41

Informal Cooperative Learning Groups

Can be used at any timeCan be short term and ad hocMay be used to break up a long lectureProvides an opportunity for students to process material they have been listening to (Cognitive Rehearsal)Are especially effective in large lecturesInclude "book ends" procedureAre not as effective as Formal Cooperative Learning or Cooperative Base Groups

CAP - 42

Active Learning: Cooperation in the College Classroom

Informal Cooperative Learning Groups

Formal Cooperative Learning Groups

Cooperative BaseGroups

See Cooperative Learning Handout (CL College-804.doc)

CAP - 43CAP - 43

Formal Cooperative Learning Task Groups

CAP - 44CAP - 44http://www.aacu.org/advocacy/leap/documents/Re8097abcombined.pdf

CAP - 45

Top Three Main Engineering Work Activities

Engineering TotalDesign – 36%Computer applications – 31%Management – 29%

Civil/ArchitecturalManagement – 45%Design – 39%Computer applications –20%

Burton, L., Parker, L, & LeBold, W. 1998. U.S. engineering career trends. ASEE Prism, 7(9), 18-21.

CAP - 46

Teamwork Skills

Communication

Listening and Persuading

Decision Making

Conflict Management

Leadership

Trust and Loyalty

CAP - 47

Teamwork Skills

Design team failure is usually due to failed team dynamics(Leifer, Koseff & Lenshow, 1995).

It’s the soft stuff that’s hard, the hard stuff is easy(Doug Wilde, quoted in Leifer, 1997)

Professional Skills(Shuman, L., Besterfield-Sacre, M., and McGourty, J., “TheABET Professional Skills-Can They Be Taught? Can They Be Assessed?” Journal of Engineering Education, Vo. 94, No. 1, 2005, pp. 41–55.)

CAP - 48CAP - 48

Teamwork

CAP - 49CAP - 49

Characteristics of Effective Teams

CAP - 50

Characteristics of Effective Teams

A small number of people with complementary skills who are committed to a common purpose, performance goals, and approach for which they hold themselves mutually accountable

SMALL NUMBER

COMPLEMENTARY SKILLS

COMMON PURPOSE & PERFORMANCE GOALS

COMMON APPROACH

MUTUAL ACCOUNTABILITY

--Katzenbach & Smith (1993) The Wisdom of Teams

CAP - 51

Hackman – Leading Teams

Real TeamCompelling DirectionEnabling StructureSupportive Organizational ContextAvailable Expert Coaching

https://research.wjh.harvard.edu/TDS/Team Diagnostic Survey (TDS)

CAP - 52

Professor's Role inFormal Cooperative Learning

1. Specifying Objectives

2. Making Decisions

3. Explaining Task, Positive Interdependence, and Individual Accountability

4. Monitoring and Intervening to Teach Skills

5. Evaluating Students' Achievement and Group Effectiveness

CAP - 53

Formal Cooperative LearningTypes of Tasks

1. Jigsaw – Learning new conceptual/procedural material

2. Peer Composition or Editing

3. Reading Comprehension/Interpretation

4. Problem Solving, Project, or Presentation

5. Review/Correct Homework

6. Constructive Academic Controversy

7. Group Tests

CAP - 54

Challenged-Based Learning

Problem-based learning

Case-based learning

Project-based learning

Learning by design

Inquiry learning

Anchored instruction

John Bransford, Nancy Vye and Helen Bateman. Creating High-Quality Learning Environments: Guidelines from Research on How People Learn

CAP - 55

Challenge-Based Instruction with the Legacy Cycle

LegacyCycle

The Challenges

Generate Ideas

Multiple Perspectives

Research & Revise

Test Your Mettle

Go Public

https://repo.vanth.org/portal/public-content/star-legacy-cycle/star-legacy-cycle

CAP - 56

Kolb’s Experiential Learning Cycle

CAP - 57

5 E Learning Cycle Model

Engage

Explore

Explain

Elaborate

Evaluate

http://faculty.mwsu.edu/west/maryann.coe/coe/inquire/inquiry.htm

CAP - 58

Problem-Based Learning

STARTSTART

CAP - 59

Problem Based Cooperative Learning Format

TASK: Solve the problem(s) or Complete the project.

INDIVIDUAL: Estimate answer. Note strategy.

COOPERATIVE: One set of answers from the group, strive for agreement, make sure everyone is able to explain the strategies used to solve each problem.

EXPECTED CRITERIA FOR SUCCESS: Everyone must be able to explain the strategies used to solve each problem.

EVALUATION: Best answer within available resources or constraints

CAP - 60

Problem Based Cooperative Learning Format

INDIVIDUAL ACCOUNTABILITY: One member from your group may be randomly chosen to explain (a) the answer and (b) how to solve each problem.

EXPECTED BEHAVIORS: Active participating, checking, encouraging, and elaborating by all members.

INTERGROUP COOPERATION: Whenever it is helpful, check procedures, answers, and strategies with another group.

CAP - 61

http://www.udel.edu/pbl/

CAP - 62

Cooperative Base Groups

Are Heterogeneous

Are Long Term (at least one quarter or semester)

Are Small (3-5 members)

Are for support

May meet at the beginning of each session or may meet between sessions

Review for quizzes, tests, etc. together

Share resources, references, etc. for individual projects

Provide a means for covering for absentees

CAP - 63

Exercise

What kind of learning environments will produce the outcomes you are assessing?

Complete your worksheet using the questions on the next page.

CAP - 64

Backward Design

Plan Learning Experiences & Instruction

What enabling knowledge (facts, concepts, and principles) and skills (procedures) will students need to perform effectively and achieve desired results?

What activities will equip students with the needed knowledge and skills?

What will need to be taught and coached, and how should it be taught, in light of performance goals?

What materials and resources are best suited to accomplish thesegoals?

Is the overall design coherent and effective?CAP - 64

CAP - 65

Ways of Assessing Actual Teaching-Learning Helpful Resources:

Learning Goals for Course/Session/Learning Module:

This Kind of Learning: Activities: (e.g., people, things)

1.

2.

3.

4.

5.

6.

Worksheet for Designing a Course/Class Session/Learning Module

CAP - 66

Next Steps

What questions do you still have about the next steps?

What information do you need to explain your plan into a year-long project?