Learning Outcomes Staff Development Day February 2003 Gavilan Community College

Learning OutcomesLearning Outcomes

Staff Development DayFebruary 2003

Gavilan Community College

Expected OutcomesExpected Outcomes

Describe learning outcomes Explain why we must attend to learning

outcomes List and compare assessment methods Outline Gavilan’s approach Create a learning outcome Identify resources for further research Submit to the will of WASC

This is a subliminal message to promote Bike To Work and School Day

on May 15th

“Pedal Peace, not war”

PerspectivePerspective In the past, year-end reports demonstrated proper

management but not whether the programs directly aided student learning

Colleges measured enrollments, participation rates, budget expenditures

More recently have expanded to measure success, retention, persistence, graduation, and transfer

Focus has also been on instructional delivery versus student learning

Means have been mistaken for the ends (Barr & Tagg 1995)

Learning outcomes now required by WASC

What are Learning What are Learning Outcomes?Outcomes? “Learning outcomes” not a new concept

– Used in California in K-12 and vocational programs– Used in other college systems (i.e. Maryland,

England)– Barr and Tagg called for learning outcomes in 1995

Focuses on what and how much students learn not on how we instruct or support them; outcome based

Emphasis on higher more complex level of learning

Clearly states educational intentions for students

Sounds like behavioral Sounds like behavioral objectives!objectives!

Learning outcomes differ from behavioral objectives by – Being broadly stated– Having instructors assess gains in skills rather

than creating detailed list of specific topics and abilities to be mastered (Harden 2002)

Some authors do not perceive that the difference between learning outcomes and behavioral objectives is important (Prideaux 2000)

The definition of insanity is doing the same thing over and over again and expecting different results so if at first you don’t succeed try, try again…

New PerspectiveNew Perspective Shift from instruction as “core of the

college” towards learning as central focus Potentially lessens threat to faculty by

shifting focus of evaluation from teaching to learning

“We must treat the college as a learning laboratory” – John Baker, VP SS Gavilan

Some areas may be easier to evaluate than others such as Vocational programs

Many Student Services will be more difficult to assess

Levels of AnalysisLevels of Analysis

InstitutionalDepartment or ProgramClass or ServiceAssignment or Student Interaction

Gavilan Institutional Gavilan Institutional Learning OutcomesLearning Outcomes

CommunicationCognitionInformation CompetencySocial InteractionsAesthetic ResponsivenessPersonal Development and

Responsibility

Gavilan Institutional Gavilan Institutional Learning OutcomesLearning Outcomes

Communication– Listening– Reading– Writing

Gavilan Instutional Gavilan Instutional Learning OutcomesLearning Outcomes

Cognition– Analysis & Synthesis– Problem Solving– Creative Thinking– Quantitative Reasoning– Transfer of Knowledge & Skills

to a New Context


Information Competency– Research– Technological


Social Interaction– Teamwork– Effective Citizenship


Aesthetic Responsiveness– Differentiate between people who

can sing and people who can’t e.g. Brittney Spears vs. Norah Jones

– Distinguish between art that offends you, art that is cheesy, art you can’t understand, and art that is too expensive


Personal Development and Responsibility– Students will develop individual responsibility, personal

integrity, and respect for diverse people and cultures– Self-management: Students will demonstrate habits of

intellectual exploration, personal responsibility and physical well being

– Ethics and Values: Students will demonstrate an understanding of ethical issues that will enhance their capacity for making sound judgments and decisions

– Respect for Diverse People and Cultures: Students will respect and work with diverse people including those with different cultural and linguistic backgrounds and different abilities

Learning Outcome Learning Outcome Example: Introduction to Example: Introduction to

StatisticsStatisticsLearning Outcome ILO

Demonstrate proficiency with calculations of basic probabilities for simple events.

cog

Utilize statistical software to analyze and interpret data.

Com, cog, info comp

Demonstrate a synthesis of concepts learned throughout the semester by a group project, which will require students to formulate a survey, collect data, analyze data, and derive inferences from the data.

Com, cog, info comp, soc

Service →Learning Outcome ↓

Communication

Student understanding and explaining goals/ plans; use advice from counselor

Student discussing and understanding requirements with staff

Student explaining what is desired: registering, transcripts, adding, etc.

CognitionUnderstand GE patterns; Develop ed plan

Calculating income, need, and unit loads

Calculating units and fees

Information Competency

Using web to gather program information

Comprehension of forms and instructions

Comprehension of forms and instructions

Social Interaction

Working with college personnel

Working with college personnel/ mentoring

Working with college personnel/ mentoring

Aesthetic Responsiveness

Form design, web page layout

Form design, web page layout

Form design, catalog, schedule web pages layout

Personal Development

And Responsibility

Understand concept of balancing school, work, and family

Provide accurate information; meet deadlines

Provide accurate information; meet deadlines

Counseling Financial Aid Admissions and Records

Assessment Assessment ConsiderationsConsiderations

Assessment has a role but is not an end in itself

Assessments must feed back into practices All data collection must be preceded with

consideration of how it will be analyzed and how well it can answer the question of service efficacy and desired learning outcomes

Collect a few well-chosen pieces of evidence rather than obtaining all possible data

Data ConsiderationsData Considerations Quantitative and Qualitative Direct and Indirect As students are exposed to many influences

on learning, separating the effects of a service from other factors proves difficult

How long does it take for learning to manifest itself in observable behavior? Must assess in many places and stages.

Not everything that can be counted counts and not everything that counts can be counted -Einstein

Example Method

Randomly assign

students to service

Randomly assign

students to receive

information about services

Correlate use with

performance or skills

measures

SurveysCase study and

journals

Strength of Evidence

Can claim causality

Weaker causality claim

Causality cannot be claimed, useful in conjunction with other information

Causality cannot be claimed, useful in conjunction with other information

Causality cannot be claimed but complex and difficult to measure effects can be noted

Ethical Consideration

Denies access to a service that may or may not be effective for some students

All students have access but some receive less information

No restriction of access or information

Use student time to complete survey

Confidentiality

Common Assessment Common Assessment MethodsMethods Tests

– Locally developed or Standardized Performances

– Recital, Presentation, or Demonstration Cumulative

– Portfolios, Capstone Projects Surveys

– Attitudes and perceptions of students, staff, employers Database Tracked Academic Behavior

– Grades, Graduation, Lab Usage, Persistence Narrative

– Staff and student journals, essays, and interviews Embedded Assessment

– Using grading process to measure ILO

Embedded Assessment Embedded Assessment ExampleExampleCriteria Joe Jane Liam Celeste Avg

Arithmetic with fractions

3 4 5 4 4

Calculating basic areas

3 4 3 5 3.75

Order of operations

3 4 5 5 4.25

Right triangles

3 4 4 4 3.75

Total 12 16 17 18

Student Grade

C B B A

Down for grades, across for outcomes assessment

–after Nichols

Assessment StrategiesAssessment Strategies

Track outcomes for every student all the time

Randomly select a set of classes to assess once per year

Gavilan’s ApproachGavilan’s Approach Educational Master Plan calls for each

department to identify learning outcomes for at least one course

Student Services held a retreat to address the application of learning outcomes

Instruction is revising selected course outlines to include learning outcomes

Next step is to develop more comprehensive and formalized communication among departments and between Student Service and Instructional divisions

What is learned needs to be systemically integrated into ongoing services and evaluation

TeamworkTeamwork

The task of evidence compilation is an institutional responsibility and must be distributed across the institution

There is no “I” in team but there is a “me” if you take out the “a” and switch the “m” and the “e”

Writing Learning Writing Learning OutcomesOutcomes

Should be based on our Mission…In an environment that nurtures

creativity and intellectual curiosity, Gavilan College serves the community by providing a high quality learning experience which prepares students for transfer, technical and public service careers, life-long learning, and participation in a diverse global society

Writing Learning Writing Learning OutcomesOutcomes

…and our Values… Value 1: We value excellence in and promotion

of comprehensive programs, services, and activities.

Because we value excellence, our objective is to develop learning outcomes and measures at the services and program level.

The measure we will use to determine whether we have met our objective is that students will demonstrate their learning in all programs according to institutional learning outcomes.

From Harriett Robles

Writing Learning Writing Learning OutcomesOutcomes Outcomes must be observable so you can

measure or record them Could be quantitative

– Play all major scales in eighth notes at 120 beats per minute (comm, cog, aesthetic responsiveness)

Or not– Compose in sonata-allegro form (comm, cog, info

comp, aesthetic responsiveness)– Accurately complete a FAFSA (comm, cog)

Course level learning outcomes sound like behavioral objectives (or really are the same) but LO tie into institutional learning outcomes

Writing Learning Writing Learning OutcomesOutcomes Good verbs (Blooms’ Taxonomy):

– Compile Arrange Classify– Analyze Identify Operate– Design Solve Write– Apply Differentiate Calculate– Demonstrate Formulate Compose– Explain Predict Assess– Compare Estimate Critique

Bad verbs– Know Understand Appreciate Learn

Good verbs are clear and observable, bad verbs are vague states of mind

Your Turn! 5 Minute Your Turn! 5 Minute ExerciseExercise

Gather into groups with people you already know so you don’t have to spend time asking their name

Write 2 learning outcomes for a class and 2 for a service

Some of you will be randomly selected to share your outcomes

5 minute exercise 5 minute exercise suggestionssuggestions

Class Service

MATH 154: Elementary Algebra

Math Lab

PSYC 1A: Introduction to Psychology

Counseling

AH 16: Intravenous Therapy/Blood Withdrawal

Health Services

MUS 4A: Beginning Piano Financial Aid

ECOL 1: Ecology: Conservation of Natural Resources

Admissions and Records

Institutional ConcernsInstitutional Concerns Another education fad that will invoke hypnotic

tendrils of smoke and little flame Sounds kindergarten Students have a fundamental right to services

and if these services are not or cannot be related to learning outcomes, their existence could be unfairly jeopardized (Collins 2002)

Uncritical application of business models and concepts to education

Fear all this measuring will be a waste of effort and not enhance student achievement – Luna, Gavilan College Academic Senate

Do students really care about learning? – Borden

Learning Outcome Learning Outcome ResourcesResources

Authors– Trudy Banta– Patricia Cross– Thomas Angelo– James Nichols– Victor Bordon

Learning Outcome Learning Outcome ResourcesResources WASC

– www.wasc.org American Association for Higher Education

– www.aahe.org/teaching/portfolio_db.htm Palomar College

– www.palomar.edu/alp/ Novia Scotia Community College

– www.nscc.ns.ca/studentportfolio Cascadia Community College

– www.cascadia.ctc.edu/LearningForTheFuture/vanguard.asp

Raymond Walters College– www.rwc.uc.edu/phillips/index_assess.html

Alverno Colleg– www.alverno.edu

Gavilan College Curriculum and Research– www.gavilan.edu/curriculum– www.gavilan.edu/research

Learning Outcome Learning Outcome ResourcesResources

Curriculum CommitteeAcademic SenateStudent Services CouncilResearcherReference Librarian

ConclusionConclusion

The learning outcomes challenge can be another opportunity to improve our efforts and to better integrate with instruction

We must take control of the learning outcomes mandate and create a process that benefits our students

We must work together Learning outcomes is a plot concocted by

Texas CEO’s to distract us from the budget crisis

AcknowledgementsAcknowledgements Gavilan Student Services Division:

– John Baker– Margery Regalado

Gavilan Instructional Division– Marty Johnson– Marlene Dwyer & her Curriculum Committee– Department Chairs– Academic Senate, Enrique Luna

Research and Planning Group– Harriett Robles, Brad Philips, Fred Trapp

WASC– Darlene Pacheco, Barbara Beno

RiddleRiddle

Made from sheetsNot meant for sleep

It's bound to make you thinkWith stories of smearing ink

Thank YouThank You

Documents

Learning Outcomes Staff Development Day February 2003 Gavilan Community College