20
USING THE COLLEGIATE LEARNING ASSESSMENT: IMPRESSIONS AND ADVICE Presentation to the Association of Institutional Research and Planning Officers Buffalo, New York June 11, 2009

Using The Collegiate Learning Assessment: Impressions and Advice

Embed Size (px)

DESCRIPTION

Using The Collegiate Learning Assessment: Impressions and Advice. Presentation to the Association of Institutional Research and Planning Officers Buffalo, New York June 11, 2009. Presenters. Patricia Francis Associate Provost for Institutional Assessment and Effectiveness SUNY Oneonta - PowerPoint PPT Presentation

Citation preview

USING THE COLLEGIATE LEARNING

ASSESSMENT: IMPRESSIONS AND

ADVICE

Presentation to the Association of Institutional Research and

Planning OfficersBuffalo, New York

June 11, 2009

PRESENTERS

Patricia FrancisAssociate Provost for Institutional

Assessment and EffectivenessSUNY Oneonta

Rosalyn LindnerAssociate Vice President

Buffalo State College

SESSION TOPICS

CLA Structure, Scoring, and Reporting Mapping the CLA to SUNY GE Outcomes Advantages and Challenges in Using the

CLA Institutional Strategies for Overcoming

Challenges

DESCRIBING THE CLA:

Structure, Scoring, and Reporting

WHAT DOES THE CLA MEASURE?

Key Higher Order SkillsCritical thinkingAnalytical reasoningProblem solvingWritten communication

Types of TasksPerformance TaskAnalytical Writing

Make-an-ArgumentCritique-an-Argument

ADMINISTRATION DETAILS

CLA Administered Over Computers for a 60-90 Minute Session

Two ApproachesCross-sectional, comparing 100 first-

semester students and 100 second-semester seniors in same academic year

Longitudinal, comparing same students as freshmen (n=300) and seniors

ADMINISTRATION DETAILS (CONT.)

The CLA is Not a Multiple Choice Test Students Respond to One of Three

PromptsAnalyze Complex, Realistic ScenariosWrite Persuasive, Analytic EssayCritique Written Arguments

WHAT DO STUDENTS SEE?

Split ScreenLeft side: Directions, questions and

response boxRight side: Document library with pull down

menu

EXAMPLES Dyna Tech “Truth” in Media Child Obesity Study

CLA SCORES Unadjusted Performance Score for

Absolute Comparisons Deviation Score for Controlled

Comparisons

EXPECTED SCORE Uses SAT/ACT Scores for Measures of

Academic Ability of Students Prior to Matriculation (Entering Academic Ability, or EAA, Score)

Estimates Linear Relationship between CLA Scores and EAA

Reports in Standard Error Units

MAPPING CLA SCORES TO SUNY GE

LEARNING OUTCOMES:

Critical Thinking and Written Communication

GEAR REVIEW, SPRING 2009

Focused Exclusively on SCBA Outcomes for Written Communication and Critical Thinking

Primary QuestionsIs there reasonable face validity between

CLA measures and SUNY outcomes?Does the CLA report provide campuses with

sub-scores for individual outcomes?

CORRESPONDENCE BETWEEN CLA AND SUNY

OUTCOMES CLA Measures Map Well to (and Provide

Sub-scores for):Critical Thinking Outcome #1 (“Identify,

analyze, and evaluate arguments….”Critical Thinking Outcome #2 (“Develop

well-reasoned arguments”)Written Communication Outcome #1

(“Produce coherent texts within common college-level written forms”)

Implications for Information Management?

ADVANTAGES AND CHALLENGES IN USING THE CLA:

And, Campus Responses to Challenges

ADVANTAGES Student Artifacts Much Richer and More

Complex Inherently Value-Added Nature of CLA Controls for Students’ Incoming Ability Provides Information on Multiple

Learning Outcomes Implications for Faculty Workload (for

Administration and Scoring) Student Engagement/Interest

CHALLENGES Recruitment of Students (and

Implications for Representativeness of Sample)

Student Participation and Motivation Relevance to Classroom Scheduling of Computer Labs Staff Time and Effort

CAMPUS RESPONSES: BUFFALO STATE COLLEGE

Recruitment of Students and SamplingFreshmen and seniorsDirect recruiting compared to faculty

recruiting Student Participation and Motivation

Scheduling$$$

Relevance to ClassroomClosing the loop

Scheduling of Computer Labs

CAMPUS RESPONSES: SUNY ONEONTA

Student Recruitment and SamplingPreparation of sampling plan for CAEAssuring comparability between sample and

other students Student Participation and Motivation

Reliance on academic departmentsEmphasis to students on benefits to institution,

programs, and themselves Relevance to Classroom

Provision of individual student results to programs

“CLA in the Classroom” initiative

USING THE COLLEGIATE LEARNING

ASSESSMENT: IMPRESSIONS AND

ADVICE

Presentation to the Association of Institutional Research and

Planning OfficersBuffalo, New York

June 11, 2009