AssessingAssessing StudentStudent

LearningLearning

Section 5Section 5

CourseCourseAssessmentAssessment

Course AssessmentThe materials in this course are designed to equip faculty to begin doing assessment where they have the most expertise, in their favorite course, and then develop skills for assessment at the program and institutional level. This section addresses assessment in courses; however the initial focus is on your own classes and sections - not the course as described in the curriculum document which may be taught by numerous faculty. The strategy is to develop SLOs and assessment practices in your sections and through dialogue, determine shared SLOs and assessment strategies for the course of record. The core SLOs for courses can then be examined to determine program SLOs. Of course program outcomes may suggest curriculum modifications, which means going back to courses and sections and modifying those SLOs and assessments; the end result is an iterative cycle of improvement.

"All the curriculum reform in the world is ultimately useless if students do not learn what faculty teach. Assessment is a response to the realization that—curricular reform, new teaching technologies, testing and grading, and ever-higher tuition notwithstanding—college graduates do not seem to be learning at a level that matches the expectations of employers, parents, or the general public. If assessment is to improve undergraduate education, it must be a faculty-defined, faculty-controlled activity." Wright, 1999

Of course there is concern that in writing individual SLOs a course or program may become fragmented. However, my colleagues and I have found that individual SLOs overlapped more than we initially assumed between sections. In fact, many faculty wanted to borrow and use other faculty member's SLO statements and assessment strategies. We also found that within programs, course outcomes logically overlapped between courses (e.g. chemistry and biology outcomes).  This makes intuitive sense if a program is really going to work. We had anticipated great debates; instead the dialogue was

Section

SLOs

 

SLOs

That

Overlap

Become

Course

SLOs Section

                     SLOs

Section SLOs

Section SLOs

Section SLOs

2

rich and beneficial.

Most faculty have experience with curriculum and program review; outcomes assessment simply builds on these review processes, while incorporating the shift from what we will do as teachers - to what the students can do as a result of the course. The challenge is to articulate the desired outcomes and document how well the outcomes were achieved with the primary purpose of improving teaching and learning. At this point take out your course SLOs developed in Section 3 and think about the potential assessment tools discussed in section 4.

The goal is to focus on, and assess, only one or two SLOs per year. At this rate, by the time you reach the six year cycle of accreditation, or your program review cycle, you should have assessed your entire set of SLOs.

DO NOT assess every SLO, every semester, or even every year!

3

Course Assessment - What do you assess?

"This is a riddle: What assessment method has nearly universal faculty participation, enjoys superb student participation, is never accused of violating academic freedom, provides detailed diagnostic assessment of student learning, is tightly linked to teaching objectives, has a tight feedback loop into classroom learning and into teacher planning, and is cheap to implement? Answer: Grading, when it is well done." Walvoord & Anderson, 1995

(There are several excellent books and articles that specifically address course assessment as noted in the reference section.)

To improve learning requires more than just assessing the product or learning outcomes. Learning is the consequence of numerous factors, including: pedagogy, curriculum, student metacognition, prior learning and misconceptions. Since this plan is focused on improvement and not just accountability, these areas are also assessed to improve outcomes. The course assessment scheme in these materials involves the use of simple assessment tools in four major areas:

Curriculum Pedagogy

Student Prior Knowledge and Misconceptions

Student Metacognition

Student Learning Outcomes

The course assessment strategy used in this material is faculty-led and embedded in class activities. When assessing curriculum, pedagogy, metacognition, prior knowledge and misconceptions, short and simple tools called Classroom Assessment Techniques (CATs) are very useful. CATs were developed by Angelo and Cross and have been used successfully to improve many aspects of teaching and learning. There will be more describing these techniques in the following sections, including links to websites and samples.

In the strategies used in this material, assessment of course student learning outcomes (SLOs) has been incorporated into the grading process using PTA and rubrics to create validated assessment tools. (Refer to Section 4 for a review of PTA and rubrics.)

4

The following flowchart represents an overview of this process.

5

Generalized Classroom Assessment Plan

Assess course comparability and pre-requisites

6.      Introduction and course establishment

Assess entering knowledge & relevant misconceptions

5.     Align Course Activities with SLOs. Create a matrix of the SLOs and course activities; eliminate those things that don’t support the outcomes

Assess activities using Primary Trait Analysis and scoring rubrics for course assignments & exams

Final course assessment and report summarizing modifications to course, SLOs, etc

2.    Develop Student Learning Outcomes

3.     Check SLOs with faculty, other schools, and professional expectations

4.   Review pre-requisites

8. Pedagogy

9. Student metacognition

Use CATs to assess effectiveness of pedagogy

Use CATs and reflective papers to have student assess their own learning

Prior to Course - Assess Curriculum

Beginning of Academic Term - Assess Previous Knowledge

Throughout Term - Formative Outcomes Assessment, Pedagogy, and Metacognition

7. Teaching and learning styles Assess learning styles

Review teaching goals

10. Terminal assessment using graded activities

11. Summary assessment of outcomes and modifications

End of Term

1. Review course curriculum and activities

Assess your teaching goals and activities using the online Teaching Goals Inventory

6

Course Assessment GuidelinesThere are benefits to assessment that are readily palpable in a classroom, but there are also concerns. A brief review of these issues, available at the link to the right, help provide a rationale for fair and manageable assessment in courses.Table 2 Benefits and Concerns of Classroom AssessmentStakeholders Benefits ConcernsStudents Explicitly describes expectations

(Students know expectations for each piece of work)

Allows students to prioritize goals

Focuses student efforts on what is necessary

Enables students to select schools and courses based on clearly defined outcomes

Provides immediate, diagnostic feedback

Extra work beyond course or program material

Impact on final grades

Frequency of assessments

Anonymity of results

 

Individual Faculty

Clearly defines the student performance expectations

Helps define content coverage

Directs assignments and assessment techniques

Provides structure to integrate multidisciplinary issues & delineates role of GE outcomes

Defines the full array of competencies (cognitive, affective, and psychomotor)

Ensures incorporation of skills and attitudes, not just cognitive outcomes

Incorporates more critical thinking outcomes and deep learning

Training

Extra time  to develop initially SLOs and assessment tools

Intrusiveness from administration or external agencies

Increases workload logging data and writing reports

Challenges academic freedom

Intellectual reductionism exhibited by national over-reliance on standardized multiple choice tests

7

Focuses attention on direct learning outcomes

Provides immediate feedback on effectiveness of pedagogy and curriculum

Improves student learning

Initiates faculty dialogue and interaction

8

Do's Don'ts

Do define expectations and criteria explicitly, hand out SLOs and rubrics.

Don't norm or rank students based on their hard work or participation, assessment is based on competence.

Do describe which assessments are part of the grading process and which are anonymous and for feedback only.

Don't be embarrassed when needs for improvement become evident - no one is perfect

Do focus on the appropriate level of Bloom's taxonomy and the three domains.

Don't focus on effortless recall, simplistic thinking skills, or factoids.

Do reflect on the cognitive, psychomotor, and affective outcomes.

Don't neglect important outcomes because they appear difficult to assess.

Do make assignments and grading criteria public.Don't ignore the public demand for accountability - you have reasons for doing things the way you do, just articulate these.

Do create multiple methods to assess students' ability.

Don't depend upon a very few assessments that are all identical in nature, allowing only certain students to display what they can do.

Do provide adequate opportunity for formative assessment.

Don't create high stakes assessments without opportunities to improve.

Do provide detailed and diagnostic feedback. Don't allow assigning grades or scoring to take precedence over providing meaningful feedback.

Do openly discuss and critically review one another’s assessments with the goal of enhancing classroom instruments.

Don't develop graded elements last or in a hurry, independent of desired outcomes or lacking in scoring criteria.

Do use assessment as a teaching tool to prompt learning.

Don't assume that assessment is an add-on; use it to improve learning as a strong motivational force.

Do pay attention to confidentiality. Don't share or report data that may reveal individual student's performance.

Do consider workload and use realistic and effective assessment plans.

Don't try to do everything at once without closing the loop and improving anything.

Do use student feedback to adjust SLOs, assignments, rubrics, and pedagogy

Don't be afraid to change and use collegial dialogue to validate changes.

Think about the way you will collect the data before you create you finalize your assessment. Scantron.com has some very useful survey forms that allow you to see how many students selected each answer. They also have automated assessment equipment for classrooms. Formsite.com provides a tool for online assessment tools.

9

Course Assessment - Assessing Curriculum

Prior to Course - Assess Curriculum

    Assess your teaching goals and activities using the online TGI

                                                                 

 

 

 1. Select your favorite course from among the courses you will teach next semester. Review the curriculum and activities and then take the teaching goals inventory. The purpose is to self assess what you want to do and target in your course, with what you do in your course activities. In the analysis section, you will notice in the results that the goals you rated as "essential" are bolded, so you can compare this to the top 3 goals as rated by other faculty in your field. (The survey takes less than 15 minutes and automatically scores the results and compares it to other faculty responses, http://itsnt12.its.uiowa.edu/cft/tgi/FMPro?-db=tgi.data_.fp5&-format=tgi.data.entry.html&-view.) Assessing faculty goals with relation to a specific course is an important step that enables faculty to define student learning outcomes more clearly and provides a tableau to initiate dialogue with other colleagues.

2. You should have developed your SLOs in section 3.  What you assess in a course is dependent upon the SLOs because SLOs direct the type of assessment and the tools you will use. This emphasizes the importance of clearly stated, substantive outcomes, developed through dialogue. The assessment process is

1. Review the course curriculum and activities

2.   Develop Student Learning Outcomes

3.   Check SLOs with faculty, other schools, and professional expectations

4. Review pre-requisites

10

Assess course comparability and pre-requisites

5.   Align Course Activities with SLOs. Create a matrix of the SLOs and course activities; eliminate those things that don’t support the outcomes

Assess activities using Primary Trait Analysis and scoring rubrics for course assignments & exams

initiated by SLOs, so the quality of the SLOs is important. Reread those SLOs using the SLO checklist.   Have you discussed them with other faculty in your discipline? Would modify anything? This is a reflective and iterative process; modify things SLOs as improvements become apparent. (This takes about 10 minutes.)

3. Review and compare your SLOs to professional expectations and SLOs from other campuses. Go to the web to check out professional standards or expectations relevant to your course. Do a search on your type of course and select 2 or 3 courses at other institutions to review content and SLOs or course objectives. (While this may seem as though it takes a lot of time, the search and review can be done in 15-20 minutes)

4. Review pre-requisite courses related to your course, if any, and the courses your course feeds. Do the expectations and SLOs align? Are the students entering your course with the requisite knowledge? Do the students that pass your course meet the standards of the next course for entering skills, knowledge, and ability? If your course satisfies the General Education requirements for your institution, do you address those SLOs in your course activities? (More on this in Section 6.)

*Note: I developed my Microbiology SLOs from the original objectives for General Microbiology Courses as defined by the American Society of Microbiology Educator's. As I changed morphed these into outcomes I made them more relevant to my students' needs and our department goals. I also checked with the local CSU Bakersfield Microbiology Professor (because the course transfers) and she asked me to add one.

5. The next step involves mapping out how the SLOs fit with your course work. It aligns expectations with the student activities and your workload. I found that I had some very cool activities, but they did not support my SLOs. I eliminated them regaining some class time and several prep hours. For instance, one activity used an entire lab session of 3 hours and was a lot of prep and clean-up but did not support an SLO. The activity was really fun (a simulated cruise ship with students assigned identities when an epidemic breaks out), the students learned a lot, but the information was not essential to my SLOs. Aligning SLOs and curriculum activities takes a little time, but contributes to a more effective and efficient organization; examples are included on the next page.

11

Creating an SLO Course Matrix

In this process each activity or graded item in the course is compared to your SLOs. A simple matrix is constructed. Some faculty have suggested creating this matrix using an F for formative assessments and an S for summative assessments to clarify the purpose of the assessment.

Hypothetical Matrix of Course Activities and SLOs

SLO 1 SLO 2 SLO 3 SLO 4 SLO 5 SLO 6

Quiz 1 F F

Quiz 2 F F F

Exam 1 F F F F F

Exam 2 F

Final S S S

Team Project

S

Essay

Oral Report S

Check the matrix for multiple opportunities for the student to receive formative feedback to improve, as well as variety in the types of assessment. Do you see any problems? Problems: 1) In the matrix above SLO 5 is never summatively assessed. 2) The first exam is the only formative assessment indicated for SLO6 and then it is summatively assessed on the final. This may not be adequate to help students

5.      Create a matrix of the SLOs and course activities; eliminate those things that don’t support the outcomes

12

Assess activities using Primary Trait Analysis (PTA) and create scoring rubrics for course assignments & exams

develop unless the exam results are used in a significant way that will be remembered for the final.3) The essay does not seem to serve a purpose related to the SLOs. This may mean: 1) that the essay assignment needs to be redefined, or 2) there is an important SLO that relates to the essay activity that has been left out, or 3) one of the SLOs needs to be modified. Checking your course goals and expectations should help resolve this situation.(Examples of an actual course matrix and SLOs are included in the Appendix)

Course Assessment - Assessing Pedagogy

The challenge is measuring the gap between

the planned curriculum,

the taught curriculum

and the learned curriculum

(Peter Ewell).

This gets back to the shift in perspective from what faculty are teaching, to what students are learning.

One way to assess the taught curriculum is to get regular anonymous feedback from students. It will come as no surprise to faculty that students often do not GET what we think we have GIVEN. Cross and Angelo developed a resource text called Classroom Assessment Techniques that describes the use of learner-centered, flexible assessments to monitor the information students are processing in class. I like to look at it as analogous to taking a pulse or blood pressure on a patient. Everything may look fine, but you need to get a reading on the vital signs in order to operate on evidence and not just intuition.

Student feedback using classroom assessment techniques provides important data to effectively modify teaching strategies and there is evidence that it enhances student engagement.  Classroom Assessment Techniques (CATs) are faculty-developed tools directed at assessing classroom strategies and activities from the student's perspective. CATs are short anonymous surveys embedded in everyday course activities. Acknowledging the effectiveness of CATs, the National Institute

13

of Science Education sponsored a website that provides a library of CATs, organized by goals and topics that can be downloaded and modified for use in any class. This and other CAT sources are noted in the reference section.

This type of course assessment is for the faculty member's own use. It would not be necessary to officially record the data, do a statistical analysis, or report details on the student feedback. The feedback on pedagogy ultimately contributes to improved student learning. However, four things are useful when assessing pedagogy:

1. Create a folder and use a notebook or file for storing and summarizing your written observations of the feedback.

2. Review the responses as soon as possible- right after class is best. Write down your overall impression of the feedback, this usually only takes 10-15 minutes.

3. Think about how you might use the feedback to modify teaching strategies and write down any ideas or plans with a short rationale.

4. Talk to colleagues about the feedback and reflect on any ideas they may suggest. Take a moment to add these thoughts to your other feedback summary.

Sample CATs from Classroom Assessment Techniques by Angelo and Cross

Muddiest Point - In this assessment technique a class or topic session is concluded by handing out an index card. Students are asked to anonymously write down the muddiest point in the class that day. The cards are collected and the faculty member reviews them immediately after class. Topics that occur frequently are then reviewed at the onset of the next class meeting.

Minute Paper - Here the students anonymously respond to two questions: 1) What is the most important thing you learned today? 2) What questions remain uppermost in your mind as we conclude the class session?

Application Cards - This assessment asks students to write down a practical application of the material they learned in the everyday world. It forces students to

14

transfer their new knowledge into a useful form and helps the faculty member see how the students are integrating class concepts.

There are many other examples in the Angelo and Cross book on Classroom Assessment Techniques and available online at the Field-tested Learning Assessment Guide at http://www.flaguide.org/intro/intro.htm

15

6. Introduction and course establishment

 

Assess entering knowledge & relevant misconceptions

Assessing Prior KnowledgeAssessing the curriculum and the pedagogy are important, however we know that even the best teacher and best circumstances do not always result in student learning. The next two sections focus on the students, their abilities, engagement, and responsibilities.

Assessing Entering KnowledgeWhen reviewing the curriculum in the first step you were asked to look at any pre-requisites for the course you were assessing. In addition to the pre-requisites you should look at other entrance factors. Are any pre-admission assessments applicable to the course, for example, is there a reading level requirement. Implementing a pre-requisite can be daunting due to the justification requirements and statistical analyses. Institutional Researchers can help simplify this process and the analysis. The resources include a reference to the Academic Senate of California Community Colleges (ASCCC) pre-requisite establishment document. This issue is more important than ever with the increasing number of under prepared students. Should you establish a pre-requisite? If there is a pre-requisite established, how do you know that the students entering your class have the essential skills and knowledge you expected them to have from that pre-requisite experience?It is relatively easy to create an assessment to measure your students entering knowledge, skills, and abilities. In the Advanced Composition and Critical Thinking class, students write an ungraded, two-page essay for homework after the first class. This reveals the student's baseline skills at MLA format, ability to create a thesis, and expertise in constructing and organizing an essay. At the end of the term this essay is used as a comparative work for the student's final reflective essay and self-assessment. In Microbiology a multiple choice test is used.Advantages to assessing entering knowledge: students are notified that prior knowledge is important and linked class time is more focused having diagnosed entering weaknesses and strengths students who do not remember, or never mastered pre-requisite skills and

knowledge, can now take specific responsibility to repair shortfalls class activities can be based upon knowledge of student ability, rather than on

what the faculty member assumes students can do

16

Assessing MisconceptionsStudents enter our classes with knowledge that pertain to our course from life experiences and courses in other disciplines. Research suggests that it is essential to link outside or past knowledge to coursework to produce deep learning and transferable knowledge (NRC, 2000). Unfortunately, some of the information students arrive with is inaccurate or based on misconceptions, and this can not be simply overlooked. A study done at Harvard University looked at a major misconception held by many "educated" graduates, even after discipline specific courses should have provided the correct information. The implications are that if the misconceptions aren't specifically identified and corrected, students will revert back to the misconception even after passing exams on the "correct" material. A video was produced displaying this research called A Private Universe. This 20 minute video is available at Annenberg for $39.95 or can be watched online for free. I encourage every faculty member to watch it and every institution to invest in a copy; the website is found in the reference section.

It is quite easy, and somewhat enjoyable, to develop a list of major misconceptions held by your students at your institution by talking with faculty. Some professional associations (Math, Physics, Engineering, Biology, and Astronomy) have identified major misconceptions held by students in that field. The reference section lists websites dealing with discipline specific misconceptions and assessment techniques to identify and correct them, as well as a Stanford reference addressing student service area misconceptions.

Unfortunately, research has shown that simply making the correct information available will not correct, or even modify, the misconceptions. Diagnosing the misconception and making people aware of the inaccuracies requires diagnostic assessment. I have created an anonymous survey for my Microbiology course. It is linked in the resource section. This survey given the first day of class to assess elementary pre-requisite knowledge (pre-requisites include chemistry and a biology course) and self-assessed lab skills. Below the survey link are two years' of data and some interpretive comments to help you determine whether this would be helpful in your course. Included in the Appendix are the Microbiology Background and Misconceptions Survey and Data from the first year.

The next step looks at helping students to look at their own learning strategies.

17

Course Assessment - Assessing Student Metacognition

"Self-assessment is a method that allows -indeed forces- students to take stock of and analyze their own learning. As such, it can be not only an evaluative tool but an educational process in its own right." Wright 1999

Recent learning research has re-emphasized the significance of students taking responsibility for their own learning, because self-monitored learning prompts and improves student metacognition. Zull (2003) advocated that students’ knowledge about their own learning was the most significant force in improving learning. Assessment tools that provide an opportunity for students to contextualize learning, and link new material to background knowledge and experiences, enhances deep learning (NRC 2000, 2001a, & 2001b). Effective assessment strategies should examine a wide variety of learning styles and include student self-assessment of their own metacognition (Lowe, 1994; Zull 2003). 

The National Resource Council describes metacognition as one of the top three strategies that produce useable in-depth learning. The NRC emphasizes the importance of incorporating self-learning skills into the curriculum in order to actively target student metacognition and profoundly influence learning outcomes (2000). In summary, metacognitive activities embedded in courses leads to:

improved learning increased ability to transfer knowledge to real life situations, and self-dependence necessary for establishing habits essential to lifelong

learning.

CATs can be used to stimulate students to think about their own learning and thinking processes. CATs related to metacognitive assessment include study-time logs to self-assess engagement and effort, diagnostic learning logs to analyze preparation, and exam post-mortems to assess and plan for specific exam learning strategies.   In addition to using CATs to enhance student metacognition, many faculty advocated the use of student learning style analyses to alert students to their own natural preferences.

18

Begin by Diagnosing Learning Styles

Learning style analysis has been historically used for basic skills or college readiness programs, but new research on learning indicates that is useful for all students. Several institutions provide access to online learning styles analysis (North Carolina State, Georgia Southern University, University of Tennessee, Penn State University, University of California, Berkeley and University of California, Davis).

Learning style tests differ in their classification methods, but most suggest study strategies based upon the diagnosis, saving the student time and energy. The two self-assessment websites below quickly determine student learning preferences and suggest strategies in less than 15 minutes.

VARK (Visual, Auditory, Read/Write, and Kinesthetic) by Fleming & Bonwell, 1998 found at http://www.vark-learn.com/english/index.asp

Learning Style Analysis by Solomon and Felder, 1999 found at http://www.engr.ncsu.edu/learningstyles/ilsweb.html

One approach is to require students to do the VARK analysis as homework and turn in a specific study plan based upon the syllabus and course outcomes as homework after the first class meeting. The following class meeting students can be temporarily grouped around the classroom based upon learning styles. Grouping by learning styles, allows them to develop study teams by selecting participants with a variety of learning strengths besides their own.

Inevitably there are very few students that learn best through auditory styles, the teaching modality they have been most exposed to in college. Examining learning styles helps students to become aware of and broaden their ability to learn from various teaching styles (Felder, 1993). *The large number of kinesthetic learners always reminds me of the importance of hands-on activities, even though they take more time.

Understanding learning styles is an important faculty consideration when choosing assessment methodologies. Anderson (2000) advocated that, particularly in open admission environments common in community colleges, there is an increased need to adapt assessment methods to a variety of learning styles.

19

Course Assessment - Assessing Discipline Learning

In section 4 we discussed assessing student learning and the grading process. Using graded assignments as assessment tools provides many benefits to students and faculty members. It requires work up front to align grading practices with assessment guidelines for validity and reliability, but has almost immediate tangible payoffs. Remember the classic grading quote?

Paul Dressel (1976) defined a grade as "an inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite material.” Miller, Imrie, & Cox 1998, p. 24. Miller, Imrie and Cox argued that,

"To counter Dressel’s criticisms we would need to ensure that the following five expectations are met:

1.  the assessment is based on clearly defined material

2. the relative contribution of each assessment is weighted according to its relative importance

3.  the same assessment consistently results in the same grade being assigned

4.  the grading procedure conforms to institutional grading policy

5   it is compatible with standards applied by colleagues." (p. 24)

Classroom-embedded assessment techniques, particularly those that align graded assignments with learning outcomes, motivate faculty because graded assignments become effective assessment tools and provide reliable alternatives to external intrusion and standardized testing (Boud, 1995a; Nichols, 1995; Watson & Klassen, 2003; Wiggins, 1993b). 

Techniques, such as Primary Trait Analysis (PTA) and the development of rubrics, as discussed in Section 4 of this material, link assignments to clearly defined material and student learning outcomes (knowledge, skills and abilities) and identified performance standards shifting grading from a norm-referenced to a criterion-referenced activity. Proper weighting of both the criteria and the assessment in relation to the class require reflective thought by the faculty member.

20

Reliance upon course embedded tools provides assessment data closest to the hub of learning with a vigorous capability to improve learning outcomes. Acceptance of classroom generated data is always coupled with the warning that assessment methods in the classroom must be designed using valid and reliable methods that target significant learning (Brookhart, 1999; Huba and Freed, 2000). *Use the draft checklist on the next page to determine if your assignment has considered aspects that convert it to a reliable and valid assessment tool.

When grades were tied to student learning outcomes and valid assessment measures they provide several distinct advantages:

The frequency of course embedded elements provide reiterative, formative feedback; a factor lacking in standardized testing.

Students are more motivated to perform their best for a grade.

The variety and flexibility of graded elements represents a fair and broad-based opportunity for students to display what they know or can do.

Embedded course assessment provides a viable, affordable alternative to standardized summative testing.

Classroom embedded assessment makes authentic assessment logistically feasible.

Benefits to students and faculty are often apparent within a semester, producing palpable results, increased faculty adoption and buy-in (Maki, 2002c).

Linking student learning outcomes to grading criteria improves, but does not completely solve, the problem of generalizing grades within multiple sections of single course offerings and between courses (Eder, Walvoord & Anderson, 1998). In order to compare similar sections of one course taught by the same or different professors, or to assess student learning outcomes across a program, additional strategies and collaboration are required. Simply collating individual course data at the institutional level is unmanageable, for this reason, course data should remain the property of the faculty member. Data are aggregated and interpreted as a single page report for incorporation into program assessment. The aggregated data are more useful to institutional level decision-making, and serve as a buffer to protect individual faculty or students by

21

maintaining the confidentiality of individual results. This will be discussed in Section 6.

22

Draft Checklist for Assessment Tools to Assess SLOs √Embedded in everyday course activities.Authentically tests student skills, knowledge, or ability.Assess appropriate level of learning.Based upon clearly defined material.Evaluated by explicit criteria.Collects valid data concerning a specific SLO.Provides reliable results or reproducible evaluative results.Assessment criteria and expectations have been validated through discussion with colleagues.Systematically administered to students.Conforms to institutional mission and values.Reflects outcomes consistent with department or program outcomes.Cost effective concerning supplies and administration.

23

Course Assessment and Curriculum Review Processes

The official curriculum document of record must represent what the institution has guaranteed the public about a given course. It is an outline that lists the important components, criteria, and expectations for each course. WASC accreditation standards include the specific requirement that the outcomes for courses be explicitly stated in the curriculum document of record. This would include those core SLOs collegially approved by the discipline faculty. The syllabus for each course and section must include all relevant SLOs; those particular to that instructor in addition to the core SLOs those on the document of record.

The curriculum committee may need to review the official curriculum document of record to be sure it provides information relevant to the new accreditation standards. Faculty may need to update their syllabi to include explicit SLOs. Discussion about these issues and any process changes should involve broad campus dialogue, careful consideration of the WASC standards, and be consistent with campus culture, missions, and values. It is helpful for the leadership to draft a timeline for both modifying the curriculum review process and reviewing updates on existing curriculum. A realistic timeline would project this updating over the next 3-5 years depending upon the campus and number of courses. Serious evaluation and careful process modifications are essential in good assessment practices and sustainable assessment practices.

The Academic Senate of California Community Colleges (ASCCC) website has several models useful for curriculum review prior to the WASC changes (see http://www.academicsenate.cc.ca.us/Publications/Papers/Model_outline.htm and http://www.academicsenate.cc.ca.us/Publications/Papers/RoleFacultyCurriculum.htm ). Refer to these historic documents and update them in conjunction with WASC standards, particularly those found in Standard 2.A.2.a-i found at http://www.accjc.org

Using the ASCCC curriculum document of record format and new WASC standards, a good starting point for components of new curriculum documents would include:

A description of how the course addresses the key theory, skills, knowledge, and abilities associated with the discipline.

24

A list of core Student Learning Outcomes (SLOs) for the course. (DO not include SLOs specific to instructors, only those SLOs agreed to as the core course SLOs by discipline faculty.)

A very brief explanation of the process used to determine the core SLOs.

A record of the course topics or concepts and teaching objectives associated with each topic and activity.

Identification of the various types of instruction used to address the course content and objectives. (These can be generally addressed with a checklist and may vary among instructors.)

A description of how the outcomes will be assessed. (Use the same general terms used in reporting the types of instruction. DO not specify assessment tools; this provides freedom within each section to test various methods and adapt tools that work best for each instructor. It is more important to address the use of single tools or embedded components when dealing with program assessment.)

This also suggests that the program review processes may need to be reviewed and updated; this will be discussed section 6.

What should the faculty do with course assessment data reports? As previously stated, these reports are of value to the instructor and students. Aggregated data and results are useful to departments or programs and protect student and faculty confidentiality. The next section describes what these reports might look like and suggests a reporting template for use with aggregated course data.

Sample Course Assessment Plans and Reports

In the Appendix is the sample assessment plan for Microbiology in two formats: 1) showing a three year timeline to implement the assessments and 2) showing how each SLO will be assessed by the end of the three year timeline. Each year has four assessment activities, two can be implemented the first semester and two the second semester. Although I am getting data on many of my SLOs I will focus my reporting on one or two of the SLOs each year.

25

The most difficult aspect of assessment is taking the TIME to write down thoughts and conclusions concerning the data, simply because it is a new step, but a valuable one allowing improvement to be made based on data, rather than intuition. Most practitioners suggest reviewing the data and writing thoughts and concerns down immediately.

Review assessment results immediately - it only takes 15 - 30 minutes, depending upon the data.

Sort index cards into piles by topics and write a few summary lines based upon the major topics on the cards.

Store the student data and conclusions in a dated file folder.

Collect digital data in a spreadsheet with summary statements typed directly into a section of the sheet.

Review data with colleagues immediately via email or phone or make a note to share it with faculty at the next department meeting.

Share important student feedback (being careful to retain student anonymity) at the next class meeting. *The English 2 (Advanced Composition and Critical Thinking) decided to post the aggregated class data on the class website. This produced an interesting response; students anticipated the data and become even more engaged in the course.

Assessment ReportThe assessment report should be simple and to the point, no more than a single page, and most reports (excluding SLO statements) are only half a page. Check out the University of Washington Assessment Website which has numerous course assessment reports available http://www.washington.edu/oea/0103.htm#Drama.

Sample Assessment Report Template

Course:

SLOs:

Methods of Assessment:

Assessment Results Summary and Interpretation:

26

Plans for Improvement:

27

A sample Microbiology report is included below summarizing assessment results.

Bakersfield College Course Assessment Report 2003-2004Course: Microbiology B16SLOs: (SLOs focused on this year *)

1. Use examples of infections, treatment, and epidemiologic control to compare and contrast the characteristics of prions, viruses, bacteria, protozoans, and multicellular parasites.

2. Explain the dynamics of commensal and pathological relationships that occur between microbes and humans.

3. Evaluate methods of microbial control and apply the proper methods necessary when given a scenario. 4. Briefly describe sample metabolic pathways found in microorganisms and their implications for food

production and human disease.5. Summarize basic bacterial genetic principles and analyze implications for mutation, genetic recombination,

and bacterial control.6. Articulate and diagram the role of the immune system in maintaining homeostasis, challenging infections,

and fighting cancer. *7. Apply the scientific method by stating a question; researching the topic; determining appropriate tests;

performing tests; collecting, analyzing, and presenting data; and finally proposing new questions about the topic. *

8. Correctly perform microbiologic lab skills and display a habit of good lab practices which extends to relevant situations in the student’s homes.

9. Retrieve, evaluate, and use microbiologic information regarding contemporary issues in the world and relevant to their everyday lives. *

Methods of Assessment:CATs for student feedback on pedagogy and curriculum.Projects, Oral reports, skills tests and exams for learning outcomes.Survey for background knowledge and terminal learning objectives.

Assessment Results Summary and Interpretation:Pedagogy & Curriculum: Students provided useful feedback on pedagogy and use of the hybrid course. The lab manual was entirely online for free student use and downloading. Technical difficulties were corrected and improvement can still be made in downloading options.Student Metacognition: 40% Visual, 8% Auditory, 12% Reading, and 44% Kinesthetic learners.Student learning: 100% of the students successfully created a pamphlet or 3D project satisfying information competency, research skills, and written and oral communication outcomes. 98% of the students were computer competent in word processing and publication and 75% were able to create a graph using a software program. Every student improved in their ability to give an oral presentation and create a word processed report, as well as information competency related to research on an emerging health topic.Areas of student learning weakness included basic organic chemistry, enzymes, DNA, transcription and translation, immunology, transmission of infections. Areas of student strengths appeared to be chemical bonding, cell anatomy, tissues, and microscope skills. Upon completion of the course student competence improved in cell anatomy, biochemical topics, DNA, immunology, infections and diagnosis, and use of antibiotics. Student achievement was disappointing in the area of organic chemistry (an important foundational area for microbiology) particularly 3-dimensional protein shape and interaction, as well as HIV transmission and epidemiologic implications.Final projects and exam scores revealed nearly every student achieved competence in the major areas as presently weighted. New strategies using immunology flow charting to meet this group of learners was very effective.

Plans for Improvement:Review weighting of team versus individual projects. Consider curricular changes for chronic chemistry problems – talk with chemistry department for ideas. Reduce lab and lecture time for hand washing; students seem to already know this information. Improve activities associated with HIV diagnosis and transmission. Continue to refine Immunology flow charting strategies. Re-evaluate SLOs next semester.

28

These reports go to the department chair and are aggregated for a program assessment report and used as resources for department planning.

Discuss, evaluate, and act on the assessment data. Participate in cross disciplinary dialogue about assessment results,

outcomes, potential modifications and improvements. Evaluate the changes needed in a course or program following thorough

reflection and discussion of the data. Implement modifications to improve student learning, student

metacognition, curriculum or pedagogy based on evidence. Create a public report to communicate the strengths and weakness in a

course or program based upon student learning outcome data. Close the loop using the data from assessment to modify and improve

activities and learning in the classroom.

The next section addresses program assessment. Program assessment of instructional areas builds upon the aggregated data available from the courses.

Please take the time to fill out the survey and provide feedback concerning these materials and any comments that might improve the training.

29

Develop, modify, or review a curriculum, course, program, or service.

Develop Student Learning Outcomes (SLOs)

Design & Measure Student Learning as a result of the Curriculum, Course, or Program

Collect, discuss, and analyze data.

Determine refinements based on outcomes data.   Closing the Assessment Loop

Surveying Progress Thus Far Section 5

1) Which of the following criteria are important when doing assessment? (Check all that apply.)

The depth or complexity of learning

Complexity of discipline language

Domains represented by the outcomes

Evidence of adequate memorization

Assessibility (Can you qualitatively or quantitatively measure this outcome?)

Availability of a standardized assessment

Consistency with external requirements

Alignment with any existing standards

Student's ability to thoroughly understand SLOs on the first day of class

Ease in generating statistical yet irrelevant data

The ability for all students to adequately perform the outcome

Alignment with program and institutional outcomes and goals

Importance of topics to overall course

2) Which of the following components of the training were helpful? (Check any that apply.)

History of Assessment

WASC standards

Assessment Plan Flowchart

Course Assessment Guidelines

Do’s and Don’ts table

Table of Assessment Tools

Introduction to CATs

Sample Assessment Tools

SLOs Matrix

Step-by-step Instructions Planning Assessment

Assessment Tool Checklist

Assessing Misconceptions

Learning Styles and Metacognition Discussion

References

30

3) Evaluate your own competence at doing the following:

4 3 2 1

Expert Experienced Beginner Novice

Matching assessments with SLOs

Discussing assessment tools with other faculty

Selecting an Assessment Tool

Mentoring a faculty member to implement assessment

Implementing Assessment in one of your courses

4) Do you have any suggestions that would improve the training?

31

Resources for Section 5

Academic Senate of California Community Colleges ( ASCCC), 1993. Model District Policy for Pre-requisites, Co-requisites Advisories on Recommended Preparation and Other Limitations on Enrollment. From the ASCCC website at http://www.academicsenate.cc.ca.us/Publications/Papers/Model_prerequisites.html

Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey-Bass.

Banta, T.W. (Ed). (1988). Implementing outcomes assessment: Promise and perils. New directions for institutional research, no.59. Vol. XV, No.3. San Francisco, CA: Jossey-Bass.

Benander, R., Denton, J., Page, D., & Skinner, C. (2000). Primary trait analysis: Anchoring assessment in the classroom. The journal of general education vol 9 (4). University Park, PA; The Pennsylvania State University.

Brookhart, S. M. (1999). The art and science of classroom assessment: The missing part of pedagogy. (ASHE-ERIC Higher Education Report, Vol. 27, No.1).

Boud, D. (1995b). Developing a typology for learner self-assessment practices. University of Technology Sydney Web site: http://www.education.uts.edu.au/ostaff/staff/publications/db_15_boud_brew.pdf

California Assessment Institute at http://cai.cc.ca.us/

Comins, N.F. (2000). An in-your-face approach about student misconceptions in astronomy. Available at http://kramer.ume.maine.edu/~panda/comins/miscon.html

Darden, A. G. (2001). The assessment-teaching link: Using student perceptions of teaching strategies to determine the value of those strategies. Focus on Microbiology Education, 7(1), 4-6.

Dressel, P. (1976). In Miller, A.H., Imrie, B.W., & Cox, K. (1998). Student assessment in higher education: A handbook for assessing performance. London; Kogan Page.

Eder, D. J. (2003). Primary trait analysis. Southern Illinois University, Edwardsville (SIUE) Undergraduate Assessment and Program Review Web site: http://www.siue.edu/~deder/assess/cats/pta.html

Ellis, A. B., & Mathieu, R. D. (n.d.). Field-tested learning assessment guide. Field-Tested Learning Assessment Guide for Math, Science and Technology Instructors Web site: http://www.flaguide.org/intro/intro.htm

32

Harvard-Smithsonian Center for Astrophysics, Science Education Department, Science Media Group. 1987. A Private Universe (Videotape Documentary). Accessible at the Annenberg website at http://www.learner.org/resources/series28.html

Huba, M. E., & Freed, J. E. (2000). Learner-centered assessment on college campuses: Shifting the focus from teaching to learning. Needham Heights, MA: Allyn and Bacon

Lowe, J. P. (1994). Assessment that promotes learning. Pennsylvania State University, Center for Teaching and Learning Excellence Web site: http://www.psu.edu/celt/Lowe.html

Mestre, J. (2000). Hispanic and Anglo Student Misconceptions in Math. Available at http://www.ericfacility.net/databases/ERIC_Digests/ed313192.html (Eric document ED313192)

Middle States Commission on Higher Education [MSACHE]. (2003). Student learning assessment; Options and resources. Philadelphia, PA: Middle States Commission on Higher Education

National Research Council. (2000). How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Nichols, J.O. (1995b). A practitioner’s handbook for institutional effectiveness and student outcomes assessment implementation. Flemington, NJ; Agathon Press.

Rodrigues, R. J. (2002). Want campus buy-in for your assessment efforts? American Association for Higher Education, Bulletin Archives Web site: http://www.aahebulletin.com/member/articles/2002-10-feature02_1.asp

Stanford University Bridge Project (2003), Student Misconceptions about Preparing for an Attending College. Available at http://www.stanford.edu/group/bridgeproject/10+Misconceptions.pdf

Udovic,D. (n.d.) Confronting Student Misconceptions in a Large Class. Available at http://www.wcer.wisc.edu/nise/CL1/CL/story/udovicda/TSDUA.htm

University of Washington, Office of Educational Assessment at http://www.washington.edu/oea

Walvoord, B. E., & Anderson, V. (1995, November-December ). An assessment riddle: Guidelines from research and practice. In T. W. Banta (Ed.), Assessment Update, 7, 8-11.

Walvoord, B. E., & Anderson, V. (1998). Effective grading: A tool for learning and assessment. San Francisco: Jossey-Bass.

33

Watson, R. J., & Klassen, P. T. (2003, February). Outcomes assessment: A faculty project. Advocate, 20(3), 1-5. Washington, DC: National Education Association Higher Education

Wiggins, G. (1990). The Case for Authentic Testing. at http://pareonline.net/getvn.asp?v=2&n=2

Wiggins, G. P. (1993a). Assessing student performance: Exploring the limits of testing. San Francisco, CA: Jossey-Bass.

Wiggins, G.P. (1993b). Assessment: authenticity, context, and validity. Phi Delta Kappan, 75, 200-208.

Wright, B. D. (1999). Evaluating learning in individual courses. California Assessment Institute Web site: http://www.cai.cc.ca.us/Resources/Wright2.doc

Zull, J. E. (2003). The art of changing the brain: Enriching the practice of teaching by exploring the biology of learning. Sterling, VA: Stylus.

34

Appendix

35

Microbiology Course Matrix Identifying Course Learning Outcomes with Course Activities

Assignments

Cell Theory

 

Microbial Interactions

Microbial Control

Microbial Metabolism

Microbial Genetics

Immune Response

Scientific Method

Application

Lab SafetySkills

Appraisal of info

Exam 1 X X         X X  

Exam 2X X X X       X X

Exam 3X X X X X X X X X

FinalX

X X X X X X X X

Teams Case Studies   X X       X   X

Gram Stain &Isolation Test X     X     X X X

Lab   Microscope X             X  Lab  Culturing X             X  Lab  Staining X   X X       X  Lab  Aseptic Technique X X X         X  

Lab  Survey of Microbes X X   X X       X

Lab  MediaX X X X X   X X  

Lab  Metabolism X   X X X     X  

Lab  DNA X     X X     X XLab  PCR and Genetics X     X X   X X X

Lab Antiseptics X   X X     X X X

Lab Disinfectant X   X       X X X

36

Lab Antibiotics X   X X X   X X X

Three D or Pamphlet project   X X X X X     X

Domain

Student Learning Outcomes for Microbiology B16

Specific OutcomesSummative Assessment

Method

Knowledge/ Cognitive

Following Completion of the Microbiology Course (B16) students will be able to:

 

Cell Theory

 Use examples of infections, treatment, and epidemiologic control to compare and contrast the characteristics of prions, viruses, bacteria, protozoans, and multicellular parasites.

Final exam essay

 question

Microbial Interactions

Explain the dynamics of commensal and pathological relationships that occur between microbes and humans.

Take home case study question for final exam

Microbial Control Evaluate methods of microbial control and apply the proper methods necessary when given a scenario. 

Multiple choice questions

on final exam

Microbial Metabolism

Briefly describe sample metabolic pathways found in microorganisms and their implications for food production and human disease.

Diagram labeled on final

exam

Microbial Genetics Summarize basic bacterial genetic principles and analyze implications for mutation, genetic recombination, and bacterial control.

Table completion on

final exam

Immune Response Articulate and diagram the role of the immune system in maintaining homeostasis, challenging infections, and fighting cancer.

Flow chart created by

student on the final exam

Skills/ Psychomotor Following Completion of the Microbiology Course (B16) students will be able to:

 

Scientific Method Application

Apply the scientific method by stating a question; researching the topic; determining appropriate tests; performing tests; collecting, analyzing, and presenting data; and finally proposing new questions about the topic.

One team & one individual

Senior Picnic & Unknown lab

Lab Safety Skills Correctly perform microbiologic lab skills and display a habit of good lab practices which extends to relevant situations in

Components of labs used

37

the student’s homes. above

Attitudes and behavior/ Affective

Following Completion of the Microbiology Course (B16) students will be able to:

 

Appraisal of microbiologic information

Retrieve, evaluate, and use microbiologic information regarding contemporary issues in the world and relevant to their everyday lives.

Project - Microbiologic

Pamphlet or 3 D project

38

Microbiology Background Knowledge SurveyUse the following pictures to answer the next three questions:

A

B C D

1. Typical of a bacterial cell.2. Typical of eukaryotic cell.3. Typical of a tissue.4. Typical of fungi.

5. Which is usually true of Bacteria?a. They are found as rods, spheres, or spirals.b. They reproduce by binary fission.c. They are prokaryotic cells and have no nucleus.d. All of the above.

6. Watera. is a polar molecule.b. is referred to as the universal solvent.c. is often a product or reactant in chemical reactions.d. All of the above.

7. pHa. is a measure of the hydrogen ion concentration.b. Utilizes a scale from 5 to 8.c. Is a geometric scale.d. Is an abbreviation for “power of hydrogen.”

8. Organic chemistry is the study ofa. molecules containing carbon.b. molecules such as carbohydrates, proteins, lipids and nucleic acids.c. chemicals important to living organismsd. All of the above

9. The important class of chemicals that limit radical pH changes in the body, medications, food, and also bacterial media are referred to as

a. enzymes.b. bases.c. buffers.d. acids.

39

10. ATPa. is an important molecule for energy storage.b. contains three high-energy phosphate bonds.c. Is classified with other organic molecules that contain nucleotides.d. All of the above.

11. Equal sharing of electrons form a(n)a. hydrogen bond.b. ionic bond.c. covalent bond.d. All of the above.

12. The negatively charged component of the atom is the a. proton.b. nucleus.c. neutron.d. electron.

13. DNA is composed of a. sugar.b. protein.c. sugar, phosphate and nucleotide bases.d. lipids and nucleotide bases.

14. In biological systems, the most important feature of a protein is itsa. primary structure (amino acid sequence).b. reactive side groups.c. three dimensional or tertiary structure.d. Electric charge.

15. Which of the following is a phagocytic cell found in the human body?a. Erythrocyte.b. Neutrophilsc. B celld. T cell

16. Worldwide, which of the following is responsible for the majority of illness and death?a. Infectious diseasesb. Heart diseasec. Cancerd. Toxic poisoning

17. Some common diseases, such as ulcers and cardiovascular disease,a. Are solely due to lifestyle.b. Are the result of a bacterial infection.c. Are solely due to genetics.d. Are due to new mutations in bacteria.

40

18. Microorganisms are involved ina. the production of medicinal products.b. food production.c. pollution cleanup.d. All of the above.

19. The single most important measure to prevent the spread of disease isa. antibiotics.b. quarantines.c. pasteurizationd. hand washing.

20. Coldsa. are caused by prolonged exposure to a cold environment.b. can be cured by antibiotics.c. Are being caused by viruses and basically untreatable.d. All of the above.

21. Which of the following is true about HIV?a. There is a mutated strain of HIV that is transmitted through the air.b. HIV can be easily contracted through saliva, contact with a soiled feminine

napkin, or other sources of dried blood.c. HIV infects T-lymphocytes and macrophages and requires close contact with

blood or body fluid.d. All of the above.

22. STD’s (sexually transmitted diseases)a. are preventable through the use of birth control pills.b. are the most common infectious diseases in the United States.c. are easy to diagnose with easily identifiable symptoms in both males and females.d. All of the above.

23. Antibiotics help in the treatment ofa. bacterial infections.b. viral infectionsc. allergies.d. All of the above.

24. I would evaluate my overall laboratory skills as:a. Excellentb. Goodc. Needing improvementd. Non-existent

25. I would evaluate my microscope skills asa. Excellentb. Adequate and ready to go in the microbiology classc. Rusty needing practice and guidanced. Very poor to non-existent – What’s a microscope!

41

Microbiology Misconceptions & Cognitive Pre and post testThis multiple choice test was given in the fall 2003 on the first day of class to all attendees (including those wanting to crash or add the class without prerequisites checks). The test was again given in the spring and again students self-selected.

# Topic

Pretest only Fall 2003n=53

Pretest Spring 2004n=52

Change from

previous semester

PosttestSpring2004n=42

Change within

semesterNotes

1 Bacterial stain 68% 19% -49% 45% +26% Unclear photo? See #4, difficult to differentiate

2 Tissue diagram

91% 98% +7% 95% -3% Confusing picture? Students performed slightly ( not significantly) worse

3 Eukaryotic cell

85% 77% -8% 98% +21% Significant Δ improvement SLO1

4 Fungal slide 60% 21% -39% 48% +27% Unclear photo? See #1, difficult to differentiate

5 Bacteria descriptors

72% 69% -3% 93% +24% Significant Δ improvementSLO1

6 H2O Chemical Characteristics

74% 83% +9% 95% +12% Chem. faculty

7 Definition pH 74% 60% -14% 81% +21% Significant Δ improvement

8 Definition organic chem

57% 56% -1% 60% +4% Needs curricular Δ attention

9 Function buffers

64% 62% -2% 86% +24% Significant Δ improvement

10 Function ATP 58% 50% -8% 69% +19% Significant Δ stillNeeds curricular Δ 60% is inadequate

11 Functioncovalent bonds

23% 46% +23% 70% +24% Significant ΔNeeds curricular Δ 70% is inadequate

12 Electron charge

64% 67% +3% 81% +14% Significant ΔNeeds curricular Δ should be 100% Test question checked; it is good? What’s up?

42

# Topic

Pretest only Fall 2003n=53

Pretest Spring 2004n=52

Change from

previous semester

PosttestSpring2004n=42

Change within

semesterNotes

13 DNA composition

- 65% - 95% +30% Significant ΔCurriculum good; Discuss with Physio & anatomy instructors

14 Protein 3D shape

23% 10% -13% 35% +25% Significant Δ But still terrible outcome- Needs curricular Δ & discussion

15 Neutrophils phagocyte f(x)

72% 35% -37% 84% +49% Significant ΔCurriculum good

16 Worldwide illness causes

26% 12%71%

- 26%65%

+8% Answer not well defined; change question

17 New bacterial Dx heart/ulcers

32% 21% -11% 79% +58% Significant ΔImprove pedagogy

18 Bacterial roles 55% 81% +26% 95% +14% Okay19 Handwashing - 98% - 98% No

changeReduce content time. Students arrive with understanding

20 Cause & Rx of colds

62% 60% -2% 91% 31% Significant ΔCurriculum good

21 HIV transmission

- 67% - 71% +4% Insignificant ΔNeeds curricular ΔNeed >80%

22 STD Facts 49% 87% +38% 93% +6% Strong curriculum throughout program

23 Antibiotics 40% 67% +27% 90% +23% Significant ΔCurriculum good

24 Self-assessed lab skills

50% 78% +28% 91% +13% Combined responses for competencyGood job in pre-requisites and improved in micro

25 Self Assessed Microscope skills

52% 62% +10% 100% +38%

43

Assessment Activities Year OneActivity Timing Format Purpose Data Access and useFacultyTeaching Inventory

Prior to class or curriculum development

Online survey @http://www.uiowa.edu/~centeach/tgi/

Used to identify and clarify personal goals, and sharpen student learning outcomes. Faculty self-assessment.

Results restricted to faculty member’s personal use

Student Learning Style Index

Within the first week of class.

Online survey @http://www.ncsu.edu/felder-public/ILSdir/ilsweb.html

Not graded

Introduces student ownership of learning and metacognition. Allows faculty to match teaching and learning styles. Provides institutional knowledge of student population learning preferences.Students self-assess and report learning preference and strategies to instructor.

Data made available:privately & immediately to students online per survey site design to instructor via student responseanonymously to class for group informationaggregated data to faculty & administration

Primary Trait Analysis (PTA) and rubric for one class activity

Prior to assignment usually prior to semester beginning.

Analysis of alignment between SLOs and assignments and creation of criteria and weighting for rubrics.Graded

Links and aligns assignment grading to student learning outcomes creating a summative student learning assessmentFaculty member assesses student learning

Data made available:to students individually anonymously to class for group informationaggregated data to faculty & administration

Postmortem on Exam 2

Created prior to Exam. Students respond after receiving graded exam.

Short series of open-ended questions on student success, difficulties, & strategies for improvementNot graded

Provides anonymous feedback to faculty member on exam, pedagogy, & student learning.Provides information to students on metacognition and success strategies. Student self-assessment & feedback

Data made available:to students individually anonymously to class for group informationaggregated data to discipline faculty & administration

CAT on one class activity

Distributed for one content areas with historically low student success

http://www.flaguide.org/cat/cat.htm

Not graded

Provides immediate, formative, anonymous feedback to faculty member concerning pedagogy and student learning.

Data made available:to students individually anonymously to class aggregated data to discipline faculty & administration

Review data immediately after each assessment and make summary comments.Review after each semester and make summary comments.Create final assessment report at the ends of the year.

Assessment Activities Year Two

44

Activity Timing Format Purpose Data Access and use

Prerequisite knowledge and preconceived misconceptions inventory

First day of class Multiple choice test and scantronDeveloped by faculty member apropos to course Not graded

Identifies previous knowledgeand misconceptions allowing appropriate strategies to focus specific content teaching, provides direction for inaccuracy correction strategies.

Data made available:by student name to instructorAnonymously to class for group informationaggregated data to faculty & administration

PTA on two additional assignments

Prior to assignments

Analysis of assignment objectives with SLOs creating criteria and weighting for rubricsGraded

Links and aligns assignment grading to student learning outcomes creating a summative student learning assessment.

Data made available:to students individually anonymously to class for group informationaggregated data to faculty & administration

PTA and post mortem on Exam 1

Prior to exam using any relevant input from post mortems on exam 2

Short series of open-ended questions on student success, difficulties, & strategies for improvement Not graded

Links assignment grading to student learning outcomes.

Data made available:to students individually anonymously to class for group informationaggregated data to faculty & administration

CATS Additional CATs on 2 additional areas

Distributed throughout the term immediately following the activity or lecture when difficult course content is covered

http://www.flaguide.org/cat/cat.htm

Not graded

Provides immediate anonymous feedback to faculty member concerning pedagogy and student learning in content areas with historically low student success.

Data made available:to students individually anonymously to class aggregated data to discipline faculty & administration

Final Exam PTA and development

Prior to exam using any relevant input from other post mortems

Analysis of assignment objectives with SLOs

Graded

This analysis uses the SLOs to determine and weight the type of questions used in the final to aptly structure the exam as a true measurement of SLOs and learning success. This analysis validates the use of the final exam as an accurate assessment tool for learning outcomes success.

Data made available:to students individually anonymously to class aggregated data to discipline faculty & administration

Review data immediately after each assessment and make summary comments.Review after each semester and make summary comments.Compare to previous year’s report.Create final assessment report at the ends of the year.Assessment Activities Year Three

45

Activity Timing Format Purpose Data Access and usePTA on all remaining class exams & activities

Prior to assignments

Analysis of assignment objectives with SLOs creating criteria and weighting for rubrics

Graded

Links and aligns assignment grading to student learning outcomes creating a summative student learning assessment.

Data made available:to students individually anonymously to class for group informationaggregated data to faculty & administration

CATS As necessary

Distributed throughout the term immediately following the activity or lecture with difficult course content .

http://www.flaguide.org/cat/cat.htm

Not graded

Provides immediate anonymous feedback to faculty member concerning pedagogy and student learning in content areas with historically low student success.

Data made available:to students individually anonymously to class aggregated data to discipline faculty & administration

Final Grade as Summative SLO Assessment

End of Semester Final grade A-F format

Graded

Summative student learning success Data made available:to students individuallyto department chair and dean aggregated data to discipline faculty & administration

Overall Summative Course Assessment Survey

Final anonymous student course assessment completed at the end of the course.

Student self reporting on-line multiple choice and open ended survey

Not graded

Summative pedagogy assessment Data made available:1) specifically to faculty member2) aggregated data to faculty & administration

Review data immediately after each assessment and make summary comments.Review after each semester and make summary comments.Compare to previous year’s report.Create final assessment report at the ends of the year.

Create Final Three Year Summative assessment report

End of the year (Can be maintained throughout the year by adding additional data as it is collected)

To be determined To summarize and evaluate final outcomes success. To serve as and accountability report. And to provide data for decisions making and modifications. To serve as a source of data for faculty dialogue.NOT used for faculty evaluation or personnel files.

Data made available:1) results restricted to faculty member when attached to faculty name and course identification2) can be distributed anonymously to faculty & administration

46

Student Learning Outcome

Description of goals to achieve outcomes

Evidence of Success Measurement & Documentation

Knowledge1. Comprehension of

the extent and implications of the Cell Theory

Compare and contrast the acellular, prokaryotic, and eukaryotic single-celled life forms with multicellular organ-system life forms.

Students will be able to compare and contrast the requirements of various cells on written & lab exams (or drawings)

Pre-test beginning of semesterClass examsFinal exam table to show synthetic understandingPost test – final

2. Formulate explanations of microbial interactions

Integrate physiological and ecological principles to explain the commensal and pathological relationships occurring between microbes and humans.

Students will be able to name and describe the role of normal flora on an exam. Students attain a basic knowledge of common (indicated on the course list) pathogenic viruses, bacteria, fungi, and multicellular parasites –in case studies, exams and lab activities.

Pre-test beginning of semesterCase Studies and Oral reportsPost test -Final examUCLA Micro lab field trip report

3. Microbial control Categorize methods of microbial control and appraise the proper applications of each category of control

Students will select and use appropriate methods of control when using laboratory organisms as exemplified in lab, on exams and in case study applications; when applying microbial control in their homes, in scenarios and discussions.

Pre-test beginning of semesterClass examLab Assignments & quizzesPost test -Final exam

4. Immune Response Articulate the role of the immune system in maintaining homeostasis and challenging infections and cancer.

Students will be able to draw a flow chart or concept map of the components of the immune system and explain the pathways used with challenges to homeostasis, in a team project, on exams and in case studies

Pre-test beginning of semesterImmune flow chart on final examPost test – final

47

Student Learning Outcome

Description of goals to achieve outcomes

Evidence of Success Measurement & Documentation

Skills5. Scientific

Method Application

Apply the scientific method by stating a question; researching the topic; determining a test (experiment); performing the test (experiment); collecting, analyzing and presenting the data; and finally proposing new questions about the topic.

Given a problem the students will be able to work as teams and produce a data-supported answer using the scientific method.

Lab – Senior Picnic & Lab 3Integrate data reports in lab exercises and oral reporting by groupsFinal Unknown Lab Project

6. Lab Safety Skills

Practice and develop the habit of using good safety and aseptic skills necessary for microbiology in the lab as well as with people and within their homes.

Daily labs will exhibit good techniqueStudents will be able to answer question concerning infection control and epidemiologyStudents will be able to explain instances where they have used microbiologic safety principles in their home

Lab Test – Safety, Gram Stain & Isolation Streak & Asepsis Quizzes

Team project on safety levels with Outbreak video

Attitudes7. Appraisal of

microbiologic information

Develop an ability to seek out and appraise microbiologic information and use it to analyze and evaluate everyday issues in the world.

Evidence of skill in seeking, selecting and using resources.References correctly cited in reports. Ability to analyze case studies from current topics. Limited understanding of bioterrorism threats and safety responses.

Pamphlet Production and Oral report.Final exam questions on 1) relevance and quality of data 2) interpretation of a recent outbreak 3) limitations to a news article

48