ROBBIE TEAHEN

ROBERT VON DER OSTEN

KIM WILBER

TracDat Overview

One Approach to Outcomes

http://www.youtube.com/watch?v=DRBW8eJGTVs

Purpose of Assessment

Simply: To advance the quality of student learning through careful elaboration of

intended learning, meaningful measures of students’ learning achievements, and

systematic collection of data that informs instructional and other improvements – at the

level of courses, programs, colleges, and institutions. Assessment involves going

beyond the evaluation of individual student performance (Teahen, 2008).

The Learning Design Cycle

Assess performance

Determine learning Needs Analyze learner

needs

Specify learning outcomes

Plan learning activities

Curriculum DevelopmentCurriculum

Development

Session Outcomes

As a result of participating in today’s session, you will: Demonstrate your ability to log in to the system Change your password Describe the meaning of relevant terms, such as “reporting” or

“assessment” units and “learning outcomes” Create an appropriate learning outcome Enter assessment unit (e.g. program) information (if unit admin) Create at least one learning outcome entry Know the tentative implementation plans/timelines – pending

further refinement Be prepared to continue to enter target outcomes/goals,

assessment plans, and your results on an ongoing basis

Goals of the Implementation

Create a repository for conserving assessment information

Stimulate cross-discipline sharing of learning and institutional effectiveness outcomes

Focus unit-level efforts on specifying and monitoring assessment outcomes

Encourage course-, program-, and institutional-level learning enhancements

Streamline reporting requirements for academic units

Topics to be Included Today

User rolesAccess Changing the passwordEntering data – outcomes, plans, and resultsAttaching documentsLinking assessment unit outcomes to

reporting unit standards or plansProducing reportsResources (Documents, “power users,” and

Oversight team, Kim)

Roles and Security

Roles: User Reports-Only Program-level admin Ferris Administrator (Kim Wilber)

Access and Security levels and permissions: Form (see Academic Affairs assessment website) Approvals by Deans

Attempting to Allay Common Concerns

Comparing instructor performance . . . Outcomes and measures are expected to be highly unique, so it will not

be possible to compare “performance” across programs or disciplines Instructor Evaluation

Instructors will continue to be evaluated by existing methods. If they want to use this information to demonstrate their effectiveness, it’s their option.

What Information is Reported Inclusion is decided at the appropriate level. . . What the program

personnel put in is what they decide to enter. Most reporting will be at the program level, and course-level reported is expected to evolve at the

This Too Shall Pass. . . Assessment of student learning has been required since 1995.

Expectations have heightened. I Don’t Have Time (It’s too much work) . . .

Assessment is integral to the work of a professional educator. It is a part of the cycle of instructional design and the continuous improvement approach of “plan, do, check, act”

Assessment Unit Information

Show/Tell/Do (using the Demo Site) Unit Name Mission Sites Offered Accreditation Body Date of next accreditation visit Advisory Board? Next program review data

Assessment Plan Information

Show/Tell/Do (using Demo) Outcome name Outcome statement Assessment Method Criterion for Success Related Goals Additional assessment methods

Outcome Checklist

describe one of the major skills that is an intended outcome for a course or program

represent a skill that a competent individual would use outside the context of the course

begin with an action verb describing what the learner will be able to do upon completion of this course/program

are measurable and observable

require application of skill, knowledge, or attitude/value – at a level of application or above on Bloom’s taxonomy

present a clear, concise, and precise statement describing the action

specify a single performance/outcome, not a combination

describe learner performance, not the instructor's activities, learning plans, or instructional strategies

Sample Program-Level Outcome and Measure(s)

Outcome: Technology UseOutcome Statement: Learners will demonstrate their

use of common functions associated with software relevant to the discipline (e.g. MS Office, SPSS, CAD, etc.)

Measure(s): 1. Capstone project assignments will incorporate the utilization of

common software applications associated with the field. Rubrics will be provided for each that address learner performance in technology use.

2. Exams in the second-year major course will incorporate timed tests utilizing identified software to produce documents appropriate to meet external performance requirements. Standards of the profession will be utilized to assess learners’ performance.

3. Throughout the program, individual course requirements will incorporate and report on technology-use performance by students, as appropriate

Components to Enter in Plan (Program)

Outcome NameOutcomeMeasuresOutcome TypeStatus (active or not)Star

Don’t Forget!

SAVE CHANGES (button at bottom)

“Means of Assessment”

Relating to a particular outcome, specify: Method Category

list still being refined Offer your suggestions

Method (Description) Criterion

What will success look like for this program or course? Schedule

When will assessment methods be implemented? Frequency?

Sample Results and Action Plan

Learners will demonstrate their use of common functions associated with software relevant to the discipline (e.g. MS Office, SPSS, CAD, etc.)

Measure(s): 1. Capstone project assignments will incorporate the utilization of common software

applications associated with the field. Rubrics will be provided for each that address learner performance in technology use.

Results: Review of 32 capstone projects for students in the X program during the spring of

2008 revealed that 95% of the learners were able to perform all specified functions within MS Word and Powerpoint, but just 62% could demonstrate their abilities to perform specified functions within Access. Further, AutoCAD design capabilities were rated to be at an average level of 3 on a scale of 1 to 5, with 10% of the soon-to-graduate students not meeting minimum standards for the profession.

Action Plan: Faculty within the major will meet in August 2008 to examine the curriculum to

determine where and how Access and CAD are introduced and reinforced and development supplemental units to assist students to more adequately achieve intended outcomes. Faculty meetings will address this performance concern and monitor curricular changes and end-of-year performance of students in spring 2009. Results from the Spring 2009 will be reported and determination made about whether additional curricular reform is required.

Linking Documents and to Goals

Attach documents. . . Examples:

Rubrics Standards Assignments

Link to standards or criteria Such as accreditation standards Program outcomes General education outcomes

Tentative Timelines

Enter Unit information immediately (mission, accreditor, etc.)

Enter Program Outcomes and Measures before end of semester

Enter at least one program outcome by end of spring

Enter multi-section and general education course assessment plans during spring 2009

Enter outcomes as available for course-level assessments

Regarding the Learning Paradigm:

"The result of this paradigm shift is a college where faculty are the designers of powerful learning environments, where curriculum design is based on an analysis of what a student needs to know to function in a complex world rather than on what the teacher knows how to teach, where the college is judged, not on the quality of the entering class, but on the quality of aggregate learning growth possessed by its graduates, where compartmentalized departments are replaced by cross-disciplinary cooperatives, and where every employee has a role to play and a contribution to make in maintaining a learner-centered environment (p. 5).”

-Bill Flynn, Palomar College, 1998