Multi-Section Assessment of One Course: English 407A Ed Nagelhout Julie Staggers Department of English

Multi-Section Assessment of One Course: English 407A

Ed Nagelhout

Julie Staggers

Department of English

Presentation Goals

Describe steps for assessing multiple sections of a single course

Use English 407A: Business Writing as an extended example

Offer recommendations for other multi-section courses

Answer Questions

Eight Steps

1. Think Programmatically

2. Select Outcomes

3. Develop Tools

4. Simplify Evaluation Measurements

5. Construct Report Template

6. Examine Findings

7. Improve Teacher Support Structure

8. Review/Evaluate Assessment

1. Think Programmatically

Every student should have basically the same experience in every (multi-section) course regardless of teacher or platform.

CoherenceConsistencyProject-CenteredPedagogical Emphasis

English 407A: Business Writing

70-75 Sections per year

Taught primarily by PTIs, GAs, and Adjuncts

Business Writing Course Delivery Platforms:Traditional Face-to-Face sectionsHybrid sectionsDistance Ed sections


To Meet Programmatic GoalsCommon SyllabusCommon OutcomesCommon ProjectsMentoring ProgramTeacher SupportTeacher Choice

2. Select Outcomes

How much can we legitimately do in 16 weeks?Determine what’s important

What do you want them to know?What do you want them to do?

Promote primary and secondary objectivesBe explicitMaintain focus


Common Outcomes Write to multiple audiences, for various purposes Plan and manage short- and long-term writing projects Draft, design, revise, and edit documents Design and implement appropriate research strategies Write collaboratively (e.g., co-authoring, peer revising) Follow and adjust to business writing conventions Design documents for both content and visual appeal Use writing to help prepare and deliver oral

presentations Write ethically and responsibly

3. Develop Tools

Tools should be designed to determine the achievement of outcomes as effectively and as efficiently as possible

Think programmaticallyAlign explicitly with outcomesUse a range of tools


QuizzesExercisesIntroductory Project (2)Case Project (4)Employment ProjectMajor Collaborative Project (3)

NOTE: Each project includes multiple deliverables and a project assessment memo


1. Primary Criteria

2. Project-Specific Criteria

Format

Audience

Access

Information

Rhetorical Awareness

Organization

Argument

Business Style

Depth of Thought

Detail

Visual

Grammar and Correctness

4. Simplify Evaluation Measurements

Evaluation measurements provide the data to help you determine the success of the course

Take cut/paste mentalityAllow for multiple lensesProvide different ways into the data


Common Outcomes Write to multiple audiences, for various purposes Plan and manage short- and long-term writing projects Draft, design, revise, and edit documents Design and implement appropriate research strategies Write collaboratively (e.g., co-authoring, peer revising) Follow and adjust to business writing conventions Design documents for both content and visual appeal Use writing to help prepare and deliver oral

presentations Write ethically and responsibly


1. Primary Criteria

2. Project-Specific Criteria

Format

Audience

Access

Information

Rhetorical Awareness

Organization

Argument

Business Style

Depth of Thought

Detail

Visual

Grammar and Correctness







5. Construct Report Template

Reporting the data in a way that will allow you to improve your course is paramount, so you need to be proactive

How do you report the data?Where do you report the data?Why do you report the data?Create template for application over time


English 407A Annual Report

Department QueriesCollege QueriesUniversity Queries

6. Examine Findings

Examine findings through the lenses you find most applicable

Student LearningTeacher EffectivenessTeacher SupportProgram Effectiveness


Preliminary Findings

Analyze by OutcomeAnalyze by Criteria for a Single Project

and across ProjectsAnalyze by PlatformAnalyze by Section and across Sections


Sample Findings

Pilot Study – Spring 2008 – 7 SectionsOutcome: “Design documents for both content and visual appeal”

Criteria: Format: All projects; All sectionsOverall Average = 4.23Low Average = 3.98 (Case Project)High Average = 4.39 (Employment Project)

7. Improve Teacher Support Structure

Strong teacher support structures are imperative to successful course delivery and assessment

Teacher EffectivenessTeacher EfficiencyTeacher WorkshopsTeacher Materials


10-Hour per Week CommitmentIf four courses is considered full-time, then

no teacher should spend more than 10 hours per week per course.

This includes time in class, time in office hours, time preparing for class, time responding / evaluating student writing, and time in training.

As a program, our primary objective is to help teachers meet this expectation.


Teacher Workshops Classroom Activities Responding Effectively AND Efficiently to

Student Writing (Egg Timer Workshop) Norming Sessions Focus Groups Staff Meetings


Teacher MaterialsCourse Materials

Syllabus, Schedule, Evaluation Sheets

Handouts Project, Pedagogical, Process/Product

Classroom Materials PPT Slides, Samples, Activities, Mini-Lectures

8. Review/Evaluate Assessment

Why are we doing assessment?Are we achieving our goals?Are the outcomes appropriate?Are we improving our course?Are we improving our teaching?Are we looking at the right things?What can we do differently?What can we do better?


Program RecommendationsRevise Project Assessment MemosIncorporate Focus GroupsUpdate/Revise WebCampusReview/Revise OutcomesReview/Revise CriteriaImprove Methods

Assessment and Pedagogical

Conclusions/Recommendations

Design to fit your needsGo slowlyMake incremental progressThink long-term

Imagine reduced teacher workload and improved student learning

Questions?

Documents

Multi-Section Assessment of One Course: English 407A Ed Nagelhout Julie Staggers Department of English