Upload
zane-neal
View
35
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Assessment. Is like a dancer’s mirror. It improves one’s ability to see and improve one’s performance. Alexander Astin 1993. ASSESSMENT . . . “ a rich conversation about student learning informed by data.” -- Ted Marchese -- AAHE. - PowerPoint PPT Presentation
Citation preview
AssessmentIs like a dancer’s mirror.
It improves one’s ability to see and improve one’s performance.
Alexander Astin
1993
ASSESSMENT . . . “a rich conversation
about student learninginformed by data.”
-- Ted Marchese -- AAHE
Assessment of Individual Student Development
Assessment of basic skills for use in advising• Placement• Counseling
Periodic review of performance with detailed feedback
End-of-program certification of competence• Licensing exams• External examiners• CLAST
Key Results of Individual Assessment
Faculty can assign grades
Students learn their own strengths and weaknesses
Students become self-assessors
A Second Look
Across students
Across sections
Across courses
Where is learning satisfactory?
What needs to be retaught?
Which approaches produce the most learning for which students?
Group Assessment Activities
• Classroom assignments, tests, projects
• Questionnaires for students, graduates, employers
• Interviews, focus groups• Program completion and placement• Awards/recognition for graduates• Monitoring of success in graduate
school• Monitoring of success on the job
Use of Results of Group Assessment
• Program improvement
• Institutional and / or state peer review
• Regional and / or national accreditation
Outcomes AssessmentThe process of providing credibleevidence of the processesand outcomes of higher education undertaken for the purpose of improving programs and services within the institution.
Banta, T. W.
Some Purposes of Assessment
1. Students learn content
2. Students assess own strengths
3. Faculty improve instruction
4. Institutions improve programs/services
5. Institutions demonstrate accountability
Assessment of Learning
1. Faculty agreement on outcomes and performance standards
2. Evidence in syllabi and assignments that outcomes are taught
3. Collective faculty review of group performance
4. Use of findings to improve instruction and curriculum
Most Faculty Are Not Trained as Teachers
Faculty Development
Can Help Instructors: Write clear objectives for student learning in
courses and curricula Individualize instruction using a variety of
methods and materials Ask questions that make students active
learners Develop assessment tools that test higher
order intellectual skills
Taxonomy of Educational Objectives
(Bloom and Others, 1956)Cognitive domain
categories
KnowledgeComprehensionApplicationAnalysisSynthesisEvaluation
Sample verbs for outcomes
Identifies, defines, describesExplains, summarizes, classifiesDemonstrates, computes, solvesDifferentiates, diagrams, estimatesCreates, formulates, revisesCriticizes, compares, concludes
Organizing for Assessment
Goal Course Measure Findings Uses
Write Portfolio
Speak Speech
Think Test
Find Information
Project
Planning for Learning and Assessment
1. Whatgeneraloutcomeare youseeking?
2. Howwould youknow it(theoutcome)if you sawit? (Whatwill thestudentknow orbe able todo?)
3. How willyou helpstudentslearn it?(in classor out ofclass)
4. How couldyoumeasureeach of thedesiredbehaviorslisted in #2?
5. What aretheassess-mentfindings?
6. Whatimprove-mentsmight bebased onassess-mentfindings?
Faculty and Staff Development
Focus faculty and student affairs professionals on improving learning in and outside class
Attend conferences together Study literature on student learning Provide workshops on teaching and learning Provide resources (e.g., grants, summer
salary, release time)
Direct Measures of LearningAssignments, exams, projects, papers
Indirect MeasuresQuestionnaires, inventories, interviews- Did the course cover these objectives?- How much did your knowledge increase?- Did the teaching method(s) help you learn?- Did the assignments help you learn?
Select or DesignAssessment Methods
1. Match with goals2. Use multiple methods3. Combine direct and indirect measures4. Combine qualitative and quantitative
measures5. Consider pre - post design to assess
gains6. Use built-in points of contact with
students
Outcomes Assessment Requires Collaboration
In setting expected program outcomes In developing sequence of learning
experiences (curriculum) In choosing measures In interpreting assessment findings In making responsive improvements
Barriers to Collaboration in the Academy
1 Graduate schools prepare specialists2 Departments hire specialists3 Much of our scholarship is conducted
alone4 Promotion and tenure favor individual
achievements -- interdisciplinary work is harder to evaluate
Change the Focusfrom
TEACHING
to
LEARNINGBarr & Tagg
1995
Functions of the Work of the Professoriate
Scholarship of Discovery
Scholarship of Integration
Scholarship of Application
Scholarship of Teaching
Ernest Boyer - 1990
Good assessment is good research . . .
An important question An approach to answer the
question Data collection Analysis Report
-Gary R. Pike (2000)
Campus Interest in Assessment
WHAT WORKS in….
increasing student retention? general education? use of technology in instruction? curriculum in the major?
Measures of Critical Thinking1. Academic Profile (ETS)2. Collegiate Assessment of Academic Proficiency
(ACT)3. California CT Dispositions Inventory (Faciones)4. California CT Skills Test (P. Facione)5. Cornell CT Test (Ennis & Millman)6. Tasks in CT (ETS)7. Reflective Judgment Inventory (King & Kitchener)8. Watson Glaser CT Appraisal (Psych Corp)
Collegiate Assessment of Academic Proficiency
(CAAP from ACT)
Individual ModulesReadingWriting (MC and Essay)MathematicsScience ReasoningCritical Thinking
COLLEGE BASE
InterpretiveReasoning
StrategicReasoning
AdaptiveReasoning
Competences
SUBJECTS
English
Math
Science
SocialStudies
EnglishSubject Clusters
InterpretiveReasoning
Writing Skills
Reading/Literature SkillsReading Critically
Reading AnalyticallyUnderstanding Literature
Are Standardized Tests the Answer?
Not available in many fieldsDo not measure all that is taughtUsually assess knowledge, not
performanceMay be standardized on
unrepresentative norm groupProvide few, if any, subscoresDo not indicate why scores are low
Start with Measures You Have
Assignments in coursesCourse examsWork performanceRecords of progress through
the curriculum
Primary Trait ScoringAssigns scores to attributes (traits) of a task
STEPS Identify traits necessary for success in
assignment Compose scale or rubric giving clear
definition to each point Grade using the rubric
Can Develop a Research Paper
1. Narrows and defines topic
2. Produces bibliography3. Develops outline4. Produces first draft5. Produces final draft6. Presents oral defense
Out-standing
Accept-able
Unaccept-able
BibliographyOutstanding – References current,
appropriately cited, representative, relevantAcceptable – References mostly current,
few citation errors, coverage adequate, mostly relevant
Unacceptable – No references or containing many errors in citation format, inadequate coverage or irrelevant
Mapping Course Outcomes to Program Outcomes
Outcomes Course 1 Course 2 Course 3
1 2
3
4
5
6
7
Sophomore Competence in Mathematics(Multiple choice responses & supporting work)
Score3
2
1
0
CriterionClear conceptual understanding, consistent
notation, logical formulation, complete solution
Adequate understanding, careless errors, some logic missing, incomplete solution
Inadequate understanding, procedural errors, logical steps missing, poor or no response
Problem not attempted or conceptual understanding totally lacking
Ball State University
Journal Evaluation1. Entries accurately and vividly
record objective observations of site experiences (events, people, actions, setting)
2. Entries convincingly record subjective responses to site experience (thoughts, emotions, values, judgments)
3. Entries effectively analyze/ evaluate your experiences (find insights, patterns, meaning, causes, effects)
Well done
Satisfac-tory
Unsatis-factory
Assessment in Sociology and Anthropology
Focus groups of graduating students Given a scenario appropriate to the discipline, a
faculty facilitator asks questions related to outcomes faculty have identified in 3 areas: concepts, theory, methods.
2 faculty observers use 0-3 scale to rate each student on each question
GROUP scores are discussed by all faculty Murphy & Goreham
North Dakota State University
Methods of Assessment Paper and pencil tests Individual or group projects Portfolios Observation of practice Observation of simulated practice Analysis of case studies Attitude or belief inventories Interviews and focus groups Surveys
Assessing Student GrowthThe Portfolio - Some Examples of Content
Course assignmentsResearch papersMaterials from group projectsArtistic productionsSelf-reflective essays (self-assessment)CorrespondenceTaped presentations
Student Electronic PortfolioStudents take responsibility for
demonstrating core skillsUnique individual skills and
achievements can be emphasizedMulti-media opportunities extend
possibilitiesMetacognitive thinking is enhanced
through reflection on contents- Sharon J. Hamilton IUPUI
PRINCIPLES OF UNDERGRADUATE LEARNING (PULs)
1. Core communication and quantitative skills
2. Critical thinking3. Integration and application of knowledge4. Intellectual depth, breadth, and
adaptiveness5. Understanding society and culture6. Values and ethics
Approved by IUPUI Faculty Council May 1998
ePort Goals
Promote clearer understanding among faculty and students of how the curriculum supports increasing mastery of the PULs.
Contribute to assessment of student learning of the PULs at multiple levels of aggregation.
Support student engagement with the PULs over their entire undergraduate experience.
PULs Levels of Competence Introductory: What all undergraduate students
should know and be able to do within the first 26 credit hours.
Intermediate: What all undergraduate students should know and be able to do within the first 56 credit hours.
Advanced: What all baccalaureate recipients should know and be able to do in their major or profession or academic program.
Experiential: Connecting curricular and co-curricular learning
Background Knowledge Probe(Pre-Test – Indirect Measure)
1. ARCHAEOLOGYA. Have never heard of thisB. Have heard of it, but don’t really know what it meansC. Have some idea what it means, but not too clearD. Have a clear idea what this means and can explain it- Classroom Assessment Angelo and Cross
Fast Feedback(at end of every class)
Most important thing learned Muddiest point Helpfulness of advance reading
assignments for day’s work in class Suggestions for improving class /
assignmentsBateman & Roberts
Graduate School of BusinessUniversity of Chicago
Student Suggestions for Improvement
Install a portable microphone Increase type size on
transparencies Leave lights on when using
projector Don’t cover assigned reading in
detail Provide more examples in class
Outcomes of Classroom Assessment
1. Classes more student centered2. Focus on learning, not teaching3. Students take ownership for learning4. Faculty look for data to make
improvements continuously
- Jann Freed Assessment Update - 1999
In a Comprehensive Assessment Program...
INVOLVE Students Faculty Student Affairs Staff Administrators Graduates Employers
Involve Students1. Set learning expectations in recruiting2. Communicate learning outcomes in
orientation3. Involve student leaders in promoting learning4. Involve students in evaluating
courses/curricula5. Let students know their recommendations
are used.
Student Advisory Council at Montevallo
A way to provide continuous student assessment
Student Recommendations1 Develop a statement of expected
ethical behaviors for students2 Add a second research course with lab3 Increase comparative psychology4 Add terminals for statistics lab5 Increase opportunities for research,
writing, and speaking
Guidance from Alumni Alumni surveys emphasized that graduates
valued skills in writing, speaking, working collaboratively, and information literacy
Now the Faculty Senate’s General Education Committee has developed 5 learning elements, at least 3 of which must be integrated in any course approved for general education-Michael Dooris Penn State University
Junior Course in Professional CommunicationTeams devise promotional event for non-
profit agency Campaign theme and rationale Logo and other visual materials Event outline
Faculty, agency representative, peers evaluate the plan- A. J. Johnson Alverno College
Assessment in Fine Arts— Portfolio review and/or audition
for every student every semester by
Panel of faculty, students, community representatives, staff or faculty outside fine arts
Results - Students creativity, conceptualization, technique have improved
— Alec Testa— Eastern New Mexico University
Involving EmployersCombination of survey and focus groups
for employers of business graduates Identified skills, knowledge, personality attributes
sought by employers Encouraged faculty to make curriculum changes Motivated student to develop needed skills Strengthened ties among faculty, students, employers
- Kretovics & McCambridge Colorado State University
Colorado State UniversityCollege of Business
Curriculum changes based on employer suggestions: 1 credit added to Business Communications for team
training and more presentations Ethics & social responsibility now discussed in intro courses New Intro to Business course emphasizing career decision-
making More teamwork, oral & written communication, problem-
solving in Management survey courses- Kretovics & McCambridge
Authentic AssessmentAt
Southern Illinois University - Edwardsville
Business - Case Study Analysis with Memo Education - Professional Portfolio Psychology - Poster on Research Project Engineering - Senior Design Project Nursing - Plan of Care for Patient
Responses to AssessmentAt
Southern Illinois University - EdwardsvilleBusiness - More case studies and research
Education - More practice in classroom management
Psychology - Curriculum change in statistics Engineering - More practice in writing and
speaking Nursing - Simulation lab with computerized
patients
Virginia Commonwealth University
1. First-year English students wrote 1 response / week for 15 weeks
2. Faculty-student affairs teams read essays
3. Sample of writers interviewed 18 months later
Responses to findings: Central advising center, new advising handbook, multicultural workshops
2004-2005
The Year of Accountability
. . . Peter Ewell Assessment Update
September 2005
National Center for Public Policy in
Higher Education
Measuring Up 2004-5-State Pilot:•National Adult Literacy Survey•Licensure and grad admissions scores•College Learning Assessment
(or ACT Work Keys)
3 Themes Common to 4 Reports
1. Accountability is now about the outcomes of student learning
2. Accountability is more focused on serving the public interest
3. Accountability now emphasizes public disclosure. . . Peter Ewell Assessment Update September 2005
External Accountability Measures
are Summative rather than Formative
and Lead to Compliance
or Creative LOCAL Initiatives
The Future Need for evidence of accountability will increase More faculty will recognize benefits of
assessment More electronic assessment methods will be
developed More sharing of assessment methods will take
place Faculty will learn more about learning and
student learning will improve