Embed Size (px)
Citation preview
Enhancing Excellence in Assessment: Being Successful with MSCHE Standards 7 & 14
Deborah MoeckelNancy Willie-Schiff
SUNY Council on AssessmentNovember 2012 Workshops
2
Campuses expected to meet or exceed MSCHE’s standards by assessing:• general education (based on common SUNY SLOs)• academic programs at all levels (on a 5-7 year cycle, or with programmatic accreditation)• institutional effectiveness.
http://www.suny.edu/provost/academic_affairs/assessment.cfm
SUNY Assessment Policy - 2010
3
Middle States Commission on Higher Education
Characteristics of Excellence in Higher Education: Requirements of Affiliation and Standards for Accreditation (Updated 2011)
http://www.msche.org/publications/CHX-2011-WEB.pdf
4
Crosswalk of Standardsto Federal Regulations
• One each for Standards 7 and 14.
• Explains the source of each standard’s fundamental
elements.
• Produced by Middle States staff.
5
MSCHE Standard 7Institutional Assessment
The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and itsCompliance with accreditation standards.
FUNDAMENTAL ELEMENTS• documented, organized, and sustained assessment process to evaluate and improve the total range of programs and services; achievement of institutional mission, goals and plans; and compliance with accreditation standards that meets the following criteria:o a foundation in the institution’s mission and clearly articulated institutional,
unit-level, and program-level goals that encompass all programs, services and initiatives and are appropriately integrated with one another;
o systematic, sustained, and thorough use of multiple qualitative and/or quantitative measures that:
maximize the use of existing data and information;clearly and purposefully relate to the goals they are assessing;
6
MSCHE Standard 7Institutional Assessment
FUNDAMENTAL ELEMENTS (continued)are of sufficient quality that results can be used with confidence to
inform decisions;o support and collaboration of faculty and administration;o clear realistic guidelines and a timetable, supported by appropriate
investment of institutional resources;o sufficient simplicity, practicality, detail, and ownership to be sustainable;o periodic evaluation of the effectiveness and comprehensiveness of the
institution’s student learning assessment processes;• evidence that assessment results are shared and discussed with appropriate constituents and used in institutional planning, resource allocation, and renewal…to improve and gain efficiencies in programs, services and processes…• written institutional (strategic) plan(s) that reflect(s) consideration of assessment results.
7
MSCHE Standard 14Assessment of Student Learning
Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills and competencies consistent with institutional and appropriate higher education goals.
FUNDAMENTAL ELEMENTS• clearly articulated statements of expected learning outcomes, at all levels (institution, degree/program, course), and for all programs that aim to foster student learning and development…• documented, organized, and sustained assessment process to evaluate and improve student learning that meets the following criteria:o systematic, sustained, and thorough use of multiple qualitative and/or
quantitative measures that: maximize the use of existing data and information; clearly and purposefully relate to the goals they are assessing; are of sufficient quality that results can be used with confidence to
inform decisions;
8
MSCHE Standard 14Assessment of Student Learning
FUNDAMENTAL ELEMENTS (continued) include direct evidence of student learning;
o support and collaboration of faculty;o clear, realistic guidelines and timetable, supported by appropriate
investment of institutional resources;o support and collaboration of faculty;o sufficient simplicity, practicality, detail, and ownership to be sustainable;o periodic evaluation of the effectiveness and comprehensiveness of the
institution’s student learning assessment processes;• assessment results that provide sufficient, convincing evidence that students are achieving key institutional and programmatic learning outcomes;• evidence that student learning assessment information is shared and discussed with appropriate constituents and is used to improve teaching and learning; and• documented use of student learning assessment information as part of institutional assessment.
9
Helpful Middle States Publications
•Team Visits: Conducting and Hosting an Evaluation Visit (Appendix 1)
• Assessing Student Learning and Institutional Effectiveness
•Evaluating Student Learning Assessment processes (rubric)
•Examples of Evidence of Student Learning
•Follow up Reports and Visits
•Summary of Actions a Team May Take
10
Resources for
Campuses:Advantages
of “Systemnes
s”
11
http://www.suny.edu/provost/academic_affairs/assessment.cfm
AND MUCH MORE…
12
http://www.suny.edu/provost/academic_affairs/RegAccred.cfm
AND MUCH MORE…
13
Campus Assessment Contacts
[email protected](STATE-OPERATED, INSTANT)
[email protected] (COMMUNITY COLLEGES, INSTANT)
[email protected] (ALL CAMPUSES, WEEKLY DIGEST)
Campus Accreditation Liaison Officers
[email protected](STATE-OPERATED, INSTANT)
[email protected](COMMUNITY COLLEGES, INSTANT)
[email protected](ALL CAMPUSES, INSTANT)
SUNY Listservs
14
SUNY Council on Assessment (SCoA)
• Enhances the assessment of institutional effectiveness and student learning by building, leading, and supporting a SUNY-wide assessment community that fosters collaboration, reduces duplication of effort and provides guidance and resources to campuses
• Plans and leads professional development opportunities for faculty, staff and administrators in the area of assessment
http://www.sunyassess.org/index.html
15
Assessment Network of New York (ANNY)• Founded in 2010, as a private, non-profit professional association
• Over 200 members by fall 2011, more than 60% from SUNY
• Stimulates dialogue and the exchange of ideas related to assessment across MSCHE-accredited institutions and provides support and resources to those committed to meaningful assessment of student learning and institutional effectiveness
• First Executive Board elected in June 2012 (11 members, 6 from SUNY, including 3 State Ops and 3 Community Colleges)
• SERVICES: Listserv, Newsletter, Conference, Regional Workshops
• Free, charter memberships still available
http://www.oneonta.edu/anny/
16
SUNY’s MSCHE StudyPurpose• Assess SUNY progress toward goals• Inform campuses for improvement
Data Sources• Documents from 41 MSCHE reviews in 2010 – 2012• Decennial visits, follow ups & PRRs• Commission actions or team reports
Method• Identify and classify each finding
Caveats • No documents for a few additional reviews• Team reports not definitive
17
Decennial, 63%
Follow Up, 17%
PRR, 20%
Distribution of 41 MSCHE Reviews in 2010 - 2012 by Type
18
Commendations, 55%
Recommendations, 41%
Requirements, 4%
Distribution of MSCHE's 621 Findings in 2010 - 2012 by Type
19
Middle States Commission on Higher Education - Definitions
Commendation: Significant accomplishments, significant progress, or exemplary/innovative practices.
Suggestion: Non-binding Findings for Improvement.
Recommendation: Institutional action needed for the institution to continue to meet the standards in Characteristicsof Excellence in Higher Education. (Must be reported on in PRR or Monitoring Reports.)
Requirement: Institutional actions needed to achieve compliance with the standard; “requirements” necessitate Commission action of postponement, warning, probation, or show cause. (Followed by additional monitoring reports and small team follow-up visit.)
20
0%
2%
4%
6%
8%
10%
12%
14%
Distribution of MSCHE's 341 Commendations in 2010 - 2012
by Standard
21
0
5
10
15
20
25
Number of Campuses with Commendations in 2010 - 2012 by Standard
22
0%
5%
10%
15%
20%
25%
Distribution of MSCHE's 254 Recommendations in 2010 - 2012
by Standard
23
0
5
10
15
20
25
30
Number of Campuses with Recommendations in 2010 - 2012 by Standard
24
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Distribution of MSCHE's 26 Requirements in 2010 - 2012
by Standard
25
0
1
2
3
4
5
6
7
8
9
Number of Campuses with Requirements in 2010 - 2012 by Standard
26
Areas for Improvement for Standards 7 and 14
SETTING GOALS● Use of SLOs for all programs and courses including graduate programs ● Goals for all functional units ● Incorporation of student learning outcomes in institutional effectiveness assessment
PLANNING● Formal assessment plan document ● Use of quantitative measures, direct measures, and data ● Alignment of all assessments (curriculum mapping) ● Staff and institutional support for assessment
IMPLEMENTATION● Culture of assessment/faculty engagement ● Communication of assessment results ● Closing the Loop ● Linking assessment to planning and budgeting ● Sustainability of assessment efforts and degree and consistency of implementation
27
Omnibus Requirements (10Q) and Recommendations (2R)
•Appear most frequently in situations where: 1. assessment efforts appear random and
inconsistent2. evidence of assessment is lacking3. the institution has been cited previously for
assessment, and progress seems insufficient
•Are most helpful in delineating all of the elements that an assessment plan and process need to indicate.
28
Sustainability of efforts, degree or consistency of implementation (18R)
These recommendations are used primarily when assessment is underway, but:
• the process is so complex or labor intensive that it cannot be sustained long term
• the process is applied incompletely or inconsistently
29
Linking assessment to planning and budgeting (13R) andclosing the loop (12R, 1Q)
These recommendations tend to appear when:
• assessment does not seem to be used to create institutional or departmental budgets
• assessment does not appear to be used in strategic or unit planning
The process is also especially important for academic planning:
• implementation of new programs
• elimination of programs
30
Alignment of all assessments,including curriculum mapping (12R, 5Q)
Curricular and assessment alignment: clear linkages among student learning goals at the program, general education and course level.
Some teams have looked for:
• evidence that assessment of student learning outcomes specifically informs faculty grading policy
• evidence that course syllabi contain this information
31
Development of SLOs for all programs and courses including graduate programs (6R)
• ALL programs and courses must have clearly identified and published student learning outcomes (SLOs).
• Program-level information must be available in catalogs and on websites.
32
Staff and institutional support for assessment (6R)
These recommendations are primarily evident where:
• there is no formal ownership of the assessment process in a particular organizational structure
• various campus leaders are unresponsive to the need for their participation in assessment processes
• appropriate assessment processes lack sufficient institutional research support
• processes for assessment are particularly labor intensive
33
Use of quantitative measures, direct measures, and data (6R)
• Use of multiple measures is recommended.
• Quantitative measures in sufficient quantity must be used.
• Direct measures of student learning must be included (as opposed to use of grades or surveys alone).
34
Communication of assessment results (6R)
• These recommendations appear in situations where assessment results appear to be owned by a few
people.
• The need for access to these results for Boards of Trustees and College Councils has also been
noted.
• Communication of results is seen as an important part of linking assessment with planning and budgeting.
35
Goals for all functional units – administrative assessment (5R)
• Implement assessment of all college services.
• All offices including Business Offices, Financial Aid, Public Safety, Human Resources, among others,
should have documented unit goals and assessment plans.
36
Culture of assessment/faculty engagement (5R, 1Q)
• Some recommendations have indicated a need for all faculty to be actively engaged in
assessment activities.
37
Incorporation of student learning outcomes in institutional effectiveness assessment (5R)
• The primary purpose of Standard 7 is to assess whether or not an institution is meeting its mission and goals. Since the primary mission of any institution of higher education is student learning, these outcomes are considered to be appropriate to this standard.
38
Formal assessment plan document (3R, 1Q)
The assumption that ‘everybody knows’ what needs to be done for assessment and what the processes are isn’t enough.
Need a document which outlines:• all of the processes and how they fit together, • who is responsible, • what measures are being used, • what the timeline is, and • how the results have been used for improvement.
See the omnibus recommendations and requirements for more detail on what these plans should include. Also helpful is the SUNY Memorandum to Presidents 2010-02 on the SUNY Website.
