Upload
jocelyn-gibbs
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Fred TrappBob Pacheco
Mary Allen
Ensure continuity in assessment when passing the baton
Orient new faculty to program assessment Provide a historical record of changes and
their rationale Inform subsequent assessments of the
same outcomes, such as using the same rubric or calibration exemplars
Accessible warehouse of information to use for program review or accreditation
Review of the currency of the outcomes Review of the curriculum map Review of annual assessment findings and
associated program changes Discussion of the evidence of impact of
previous program changes
Students understand economic theories. Students can write and speak effectively. Students complete a dissertation. Students pass the licensure exam. Students can conduct literature reviews.
Course PLO1 PLO 2 PLO3 PLO4
100 X X X X
200 X X X X
300 X X X X
310 X X X X
312 X X X X
340 X X X X
400 X X X X
410 X X X X
425 X X X X
430 X X X X
490 X X X X
The department chair assessed three outcomes this year using exit interviews with graduating students. She found that students satisfactorily mastered all three outcomes.
We assessed students’ mastery of research methods by calculating the average grade in the Research Methods class. The average grade was 2.96. This is good.
We selected 50 capstone projects at random, calibrated 10 faculty on the use of an analytic rubric with four dimensions, and assessed the quality of students’ writing skills. Inter-rater reliability estimates ranged from .87 to .93.
We decided that we’d be satisfied if at least 80% of the students scored at level 3 or higher on each dimension of the rubric. We were satisfied with students’ control of syntax & mechanics and their use of sources, but we were disappointed with their synthesis of ideas and the overall organization of their writing.
We learned that students are not explaining theories at the level we expected. We decided to close the loop by expanding the emphasis on theory in each survey course (310, 312, 315, 316, and 321) and by placing more emphasis on theory in exams in these courses. Faculty will devote more time to theory during class meetings, and at least 20% of students’ grades will now reflect their ability to explain theories.
Does closing the loop require additional budget for the program?
Have a procedure for allocating funds for well-based budget requests.
Identify common outcomes with weak results to identify the need for a campus solution, such as the need to establish a Writing Center, to expand ESL assistance, or to offer faculty a WAC workshop.
Warehoused annual reports and program reviews are easily accessed for integration into WASC self studies and for visiting teams to review prior to their visit. Annual reports demonstrate sustainable, effective assessment is in place.
15
How can the report writing experience: ◦ Help faculty explore the student learning process?◦ Determine the extent to which the curriculum is
working?◦ Where can time, energy and/or money be allocated
for continuous improvement in learning?◦ Exploit the writing process and dialogue about
results to gain broader institutional learning experiences? **
◦ Help meet our quality assurance pledge to the community?
**Adriana Kezar ed. Organizational Learning in Higher Education New Directions for Higher Education. No. 131, Fall 2005. Jossey-Bass.
1. Assessment focus- course, program, general ed, etc.
2. What outcomes were assessed?3. How and when were they assessed?4. Who was assessed?5. What were the results?6. Who reviewed the results, made sense of the
them and what conclusions were reached?7. What are the implications for practice and/or
policy or future assessment work?
Promising Vehicles for Expanding Information to the Public◦Brief narrative report from annual
assessment reports◦Simple statistical reports on learning
outcomes or surveys◦Best practices stories supported by
assessmentPeter Ewell. Accreditation & the Provision of Additional Information to the
Public about Institutional and Program Performance, CHEA, May 2004
National Institute for Learning Outcomes Assessment (NILOA)◦ 2010 Webscan report Exploring the Landscape: What
Institutional Websites Reveal About Student learning Outcomes Assessment Activities
◦ 2010 Connecting State Policies on Assessment with Institutional Assessment Activity
◦ 2011 Providing Evidence of Student Learning: A Transparency Framework
Ease dropping◦ www.learningoutcomeassessment.org
◦ Projects are at least three semesters long◦ Individual and high-impact courses (all sections)
included◦ Project proposal by a faculty group◦ Course improvements based on data analysis◦ Reassessment expected◦ Results/report shared across the college and web
posted
Ease dropping◦ http://www.ccbcmd.edu/loa/CrseAssess.html◦ Two-page executive summaries available
CHEM 108◦ An initial “failure” turned to success and
collaboration with a four-year school HLTH 101
◦ Addressing an achievement gap with professional development and increased communication with students
CRJU 101 and 202◦ Statewide group assessment development effort
and creativity in the interventions used
Program learning outcomes data collected in a student E-portfolio◦ Directing internal and external evidence (1 to 10
measures)◦ Indirect evidence (1 to 4 measures)
Evidence drawn from samples of student work for faculty to apply an agreed upon holistic rubric◦ Eight general education outcomes (student
success skills)◦ Discipline-specific exit competencies or outcomes
Hocking College example reports and analysis◦ Culinary Arts Technology (on cloud)◦ Forestry Management Technology (on cloud &
college web)◦ Nursing Technology (on cloud)
JFK University example report◦ Counseling Psychology (on cloud)
General education studies completed 2007-08; 2005-06◦ Numeracy◦ Scientific inquiry◦ Problem solving/critical thinking (2008-09)◦ Information literacy◦ Workplace skills (CTE) (2009-10)
General education studies completed 2006-07; 2004-05◦ Arts & humanities◦ Cultural diversity◦ Oral communication◦ Written communication
Ease dropping◦ http://www.mesacc.edu/about/orp/assessment/
index.html
◦Annual reports and summaries available Nine years of history and experience 14 years of assessment work
Ease Dropping◦ Assessment work began in 1970◦ http://assessment.truman.edu/
Assessment Almanac- A compilation of results from each year’s assessment work (versions from 1997 to 2009 are posted)
General Education outcomes are assessed in the context of the major field of study Portfolio Project- required of all seniors to show best work assessed
by faculty for the nature & quality of the liberal arts and sciences learning outcomes (versions from 1997 to 2008 are posted)
Why archive?◦ Compliance vs. institutional learning
Where to keep the completed report?◦ Decentralized vs. central office
Options for how to keep the reports?◦ Paper◦ Electronic templates◦ Interactive database
To all affected participants Campus committees
◦ Curriculum, assessment, resource allocation group, unit (department) leadership, general academic and college leadership
Campus fairs, brown-bag lunches, poster sessions for information sharing
Faculty professional development programs Accreditation self-study committee work groups Local governing board presentation College web site for the public
Feedback & recognition◦ Feedback rubric for annual assessment
reports Conversations and action Collection and analysis of evidence Implementation of findings
◦ Recognition (achievement & excellence)
Ease droppinghttps://www4.nau.edu/assessment
Seal of Assessment Achievement Academic programs earning this recognition
have demonstrated in their annual report that
• learning outcomes have been assessed through
two or more methods, and • findings have been discussed
among the faculty.
Seal of Assessment Excellence Academic programs earning this
recognition have demonstrated ◦ a thorough implementation of assessment
plan(s) ◦ the reporting of meaningful assessment data ◦ the discussion of findings among faculty
and perhaps students◦ the use of findings to showcase
student achievements and to make curricular adjustments.
Voluntary System of Accountability◦ APLU & AASCU (520 public institutions, award 70% of
bachelor’s degrees in the US each year)
◦ College Profile (includes learning outcomes & links to campus) Proactive initiative to document learning gains and average
institutional scores (choice of 3 national instruments) Proactive initiative to illustrate unique campus learning outcomes
assessment work Promoting a learning institution
◦ Ease Dropping http://www.collegeportraits.org/ CSUPomona
http://www.collegeportraits.org/CA/CPP/learning_outcomes
National Association of Independent Colleges and Universities◦ Assessment programs on campus tied to
institution’s mission
◦ Ease Dropping http://www.naicu.edu/special_initiatives/
accountability/Student_Assessment/id.514/default.asp
Pepperdine University http://services.pepperdine.edu/oie/learning-outcomes/
learning-outcomes-overview.aspx
Filesanywhere.com
http://www.filesanywhere.com/fs/v.aspx?v=8a69668b5c6773a96f6d
Mary Allen (independent consultant)◦ [email protected]
Robert Pacheco◦ [email protected]
Fred Trapp (Cambridge West Partnership)◦[email protected]
Questions and Comments