View
214
Download
0
Category
Tags:
Preview:
Citation preview
2
On Planning…
“Whatever failures I have known, whatever errors I have committed, whatever follies I have witnessed in private and public life have been the consequence of action without thought.” Bernard M. Baruch 1870-1965, American Financier
“To be prepared is half the victory.”Miguel De Cervantes 1547-1616, Spanish Novelist, Dramatist, Poet
“Before beginning, plan carefully.” Marcus T. Cicero c. 106-43 BC, Great Roman Orator, Politician
3
Purpose of the plan Gives us direction Establishes accountability Helps measure progress Keeps us in track Facilitates execution Increases probability of achieving goals Demonstrates commitment to institutionalize
assessment.
at least…it Should!!!
4
Levels of Assessment of Learning
Institutional Learning Goals
Course Goals
Lesson Objectives
Program Learning Goals
AACSB’s Focus
5
Writing a Program Assessment Plan: Steps 1-5
1. Revisit Vision and Mission Statements2. Express rationale and purpose3. Identify audiences you wish to
address4. Set assessable learning outcomes5. Identify (or design) enabling
experiences
6
6. Select/design assessment instruments and establish performance criteria
7. Set schedule for assessing each goal.
8. Assess learning outcomes
9. Interpret and report results
10. Introduce change strategies
The Plan: Steps 6-10
7
Must take into consideration external and internal reality
Every effort must be based on your program’s mission
Mission must be owned by faculty, students and administrators
Must be revisited periodically
Step #1: Revisit Program’s Vision and Mission
8
Is it aligned with your institution’s mission? Is it inspiring? Does it delineate the scope of academic
programs? Does it define your geographic market? Does it profile your completing students and
faculty? Is it unique?
Step #1: Revisit Program’s Vision and Mission
9
Step #2: Express Rationale and Purpose for Assessment
Rationale: Why are you writing this plan?“As a liberal arts college, we seek to enhance the
personal development of our students.”
Purpose: Who will be assessed? What will you assess? When will assessment takes place?
“The primary purpose of the assessment program is to describe the cognitive and personal development of
our students from matriculation to registration.”
10
Let’s practice Step #2: Answer the following:
Why are you doing this plan? ________________________________________________________________________________________________________________________
What main purpose will it serve?________________________________________________________________________________________________________________________
11
Step #3: Identify Audience(s) This section addresses:
To whom you plan to communicate your assessment results.The message that you want to convey.The media through which you will communicate results.
Example: Who: Prospective StudentsMessage: “We regularly assess students’
performance, and provide them feedback and guidance to improve.”
Media: Program brochure and recruiting materials.
12
Let’s Practice Step #3:Identify at least two audiences…
Audience #1- _______________________
Message - ________________________________________
________________________________________
________________________________________
Media -
__________________________________________________________________________
________________________________________
13
Step #4: Establish Program Learning Outcomes
State the broad educational expectations for each degree program.
General, not specific. general knowledge and skills management-specific learning goals
Specify the intellectual and behavioral competencies a program is intended to instill.
Clarify how we intend for graduates to be different as a result of their completion of the program.
14
Learning Outcomes…
Need to be defined operationally in order to be assessable.
Normally, four to ten learning goals will be specified for each degree program.
Express what students will be able to do when they complete your program requirements.
Must be focused on the student’s learning outcomes
Specify (or suggest) level of learning.
15
Hierarchy of Learning Outcomes
Institutional
College (Dept)
Program
Course
Session
“Apply scientific inquiry methods.”
“Think logically and critically.”
“…basic and applied research.”
“…apply data collection techniques.”
“list steps in the scientific process.”
16
The focus! The student’s learning.
Focused on Teaching Focuses on the
professor’s effort, intent, and interests
or
Focused on Learning Focuses on the student’s
achievement
Example: “Expose the student to recent
research in organizational behavior”.
Example: “After completing this course the
student should be able to evaluate recent research in organizational behavior”.
Objective #4
17
Types of Learning (According to Bloom)
What is to be learned? Cognitive: mental skills (Knowledge)
“…list three major EEO laws…” Affective: growth in feelings or emotional
areas (Attitude) “…value diversity…”
Psychomotor: manual or physical skills (Skills) “… use the keyboard…”
Objective #4
18
Levels of learning…
Know
Comprehend
Apply
Analyze
Synthesize
Evaluate
Define, describe, list, reproduce, enumerate
Classify, explain, discuss, give examples, summarize
Determine, develop, compute, utilize, conduct
Correlate, diagram, distinguish, outline, infer
Adapt, combine, compare, contrast, design, generate
Compare & contrast,
critique, justify
Objective #4
Bloom’s Taxonomy (Cognitive Domain)
19
Dealing with “understand”…
Always ask yourself… How do you know someone understands?
He/she can: Give examples of … Determine the “correct” method to use… Discuss pros and cons… Identify elements in a given case…etc.
20
Let’s Practice Step #4:Write at least 2 Program Learning Outcomes
“Graduates of the Business Administration Program at UPR-___________ will be able to:
1. (Do what?) (At which level?)
2.
21
Step #5: Identify/Design Enabling Experiences
Curriculum Focus on Core Courses
Taken by all students
Extra curriculum Student associations Seminars Guest speakers Forums Internship/coop
22
Program Goal MatrixGoal #2 - “The students will use the appropriate software packages and hardware as a working tool in the daily operations of a business”.
Course/Experience Level of Learning Application of Learning
ADMI 3007 - Intro. to Data Processing
Application
Business related problems
ESTA 3001- 3002 - Business Statistics
Application
Business related problems
CONT 3005-3006 - Elementary Accounting
Application
Accounting problems
Internship/COOP
Application
Real work environment
Computer-assisted courses
Application
Business problems
Assigned projects in several courses.
Application
Objective #4
23
Lets Practice Step #5: Choose 1 Goal and complete the following table:
Goal: ________________________________________
Courses/Experiences that help students achieve this goal
Level of Learning (Bloom’s Taxonomy)
24
Step #6: Select Assessment Instruments and Establish Performance Criteria
For each learning outcome choose an instrument
Entry Level Midpoint Exit Level Assessment Assessment Assessment
Essay included Paper written Senior Essay in admission in 2nd yr. course package
Criteria: 50% 70% 85%
Diagnostic Test Repeat Repeat
Criteria: 44% 70% 85%
25
Program Assessment Instruments
Direct Measures of Learning Diagnostic test to incoming
students on learning goals Course-embedded
assessment Capstone courses Projects Seminar Writing assignments
Internships/Coop Proficiency exams Professional exams Student Portfolio
Indirect Measures of Learning Employer Reports on Alumni Exit Surveys Focus groups Interviews Student reports on
Internships/Coop Employer performance
evaluation of interns/coop Student evaluation of courses
26
Examples of Measures of Achievement
Example 1 School A has defined a learning goal in ethical reasoning for each
of its four undergraduate majors. Student achievement on this goal is relevant to demonstrating satisfaction of Standard 16. The school’s faculty has defined the goal.
Learning Goal “Each student can recognize and analyze ethical problems and
choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing.”
Demonstration of Achievement The school uses course-embedded exercises in three required
introductory-level courses. Faculty in the three disciplines have developed different methods for instructing and assessing achievement toward this learning goal. AACSB
27
In Accounting A two-week module near the end of the introductory
course is devoted to “Ethical standards and fraud in accounting.”
A topic outline has been developed by faculty members to structure an exam on the materials of this module, and a standard set of expectations has been created for grading the exam.
In addition to this exam’s contribution to the course grade, it provides a pass/fail indication on the learning goal.
Examples of Measures of Achievement
AACSB
28
In Human Resource Management Students must provide four written analyses of problem
situations during the course. On three of these analyses (on the topics of selection, reward systems, and job design), students are asked to respond to ethical issues.
A standard scoring key on the ethical component provides evaluation toward the course grade and a pass/fail indication on the learning goal.
Examples of Measures of Achievement
AACSB
29
In Marketing Each student must compose a term paper analyzing a
current national or international marketing campaign. The analysis must include a specified set of components,
and ethical issues that have been presented in lectures are among the required components.
In addition to the overall grade of the paper, each student receives a pass/fail indicator on the ethics component.
Examples of Measures of Achievement
AACSB
30
Let’s practice Step #6:
Choose 1 learning goal and brainstorm with colleagues on assessment instruments that may be used to directly measure student achievement of the goal.
Establish criteria (“passing grade”)
31
Step #7: Create a Calendar for assessing each goal
Learning Goal Year
1
Year
2
Year
3
Year
4
Year
5
Writing X X
Speech X X
Teamwork X
Information skills X X
Ethics X X
33
Step #8: Assess Learning Outcomes
Conduct assessment according to schedule
Remember you can use samples
34
Step #9:Interpret and Report results
Collect, process and analyze results. Envision sitting before the curriculum
committee with the assessment results in your hand.
Engage them in analyzing and reflecting on the meaning of these results and implications to them.
KISS Descriptive statistics, graphs…
35
Reporting results
Use your plan in Step #3 Audience Message Media
Change the culture!Communicate, communicate,
communicate!
36
Step #10: “Close the loop”Introduce Change to Improve
In curriculum Faculty development New learning experiences
37
Determine the gap and narrow it!
Assessment
Reveals the Gap
Intended Outcomes
1. Act ethically
2. Quantitative Analysis
3. Apply models
4. Decision making skills
5. Communication skills
Actual Outcomes
1.
2.
3.
4.
5.
Objective #1
Recommended