Assessment@Dartmouth Resource Making Your Case: Presenting … · 2019. 12. 10. · Cone of Experience!10% of what we read!20% of what we hear!30% of what we see!50% of what we see

Assessment@Dartmouth

Copyright 2008

Making Your Case:

Presenting Results and

Writing ReportsGavin Henning

Student Affairs Planning, Evaluation and Research

Dartmouth College

[email protected]

Assessment

@

Dartmouth

Resource

! Bers, T. H. & Seybert, J. A. (1999). Effective reporting. Resources in institutional research 12. Talahassee, FL: Association for Institutional Research.

2

Assessment

@

Dartmouth

Session Outcomes

! Develop better reports

! Present data in a way that others understand it

3

Assessment

@

Dartmouth

Application

! Identify a report you will need to write or types of data you will need to present

4

Assessment

@

Dartmouth

Challenges

! What challenges do you face when presenting or reporting assessment results?

5

Assessment

@

Dartmouth

Reporting Process

! Determine clients, purpose, and audience for report

! Determine data and info available or need to gather for report

! Select type of report (i.e., survey report, factbook)

! Select format of report (i.e., written, oral, etc.)

! Decide how to depict data an information

! Produce the report

! Disseminate the report6

Assessment

@

Dartmouth

Types of Audiences

! Executives

! Board of trustee members

! Faculty members

! Experts

! Lay people

! Parents

! Combined audiences

! Students7

Assessment

@

Dartmouth

Audiences

! List your primary and secondary audiences

8

Assessment

@

Dartmouth

Considering Audiences

! What do my audiences need to know?

! What do my audiences want to know?

! What do I tell my audience about this?

! What decisions will or might they make based on this report?

! What other individuals might my audience send this report to?

! Who else might be interested and will they have access to relevant documents?

9

Assessment

@

Dartmouth

Considering Audiences

! Thinking about your primary audiences?

! What is going to have an impact on my audience• quantitative, qualitative, or both• conclusions, recommendations, supporting

evidence

! What is the knowledge level of the audience

10

Assessment

@

Dartmouth

Purposes of Reports

! Historical record

! Support for planning or decision making

! Public relations

! Information dissemination

! Compliance with external reporting requirements

11

Assessment

@

Dartmouth

Purpose

! What is the purpose of the report you need to write or data you need to present?

12

Assessment

@

Dartmouth

Types of Reports

! Survey or major projects reports

! Projections

! Accreditation self-studies

! Program review and evaluation reports

! Factbooks

! Planning reports

! Technical reports

! Financial reports

! Peer group comparisons

! Promotional materials 13

Assessment

@

Dartmouth

Report Formats

! What are the various formats that reports can take

14

Assessment

@

Dartmouth

Types and Formats

! What type of report would be most appropriate?

! What format of report would be most appropriate?

15

Assessment

@

Dartmouth

Audiences, Types, and Formats

! How do you decide which types and formats to use with which audiences?

16

Assessment

@

Dartmouth

Cone of Experience

! 10% of what we read

! 20% of what we hear

! 30% of what we see

! 50% of what we see and hear

! 70% of what we discuss

! 80% of what we experience

! 95% of what we teach

17Dale, E. (1969). Audiovisual methods in teaching. New York: Holt, Rinehart, and Winston.

Assessment

@

Dartmouth

Report Components! Meaningful title

! Executive summary

! Table of contents

! Introduction and purpose

! Methodology

! Findings

! Summary, conclusions, implications, and

recommendations

! References

! Glossary

! Appendices, exhibits, attachments 18

Assessment

@

Dartmouth

Quantitative Data and Reports

! Summarize key findings in both narrative and numeric forms

! Statistical significance may not be practical significance for decision makers

! Use tables and graphics judiciously in the body

! Provide narrative description of what statistical significance means

! Consider using a question-answer format

19

Assessment

@

Dartmouth

Quantitative Data and Reports

! Use 1-page briefs and/or Web pages referring readers to full report

! Use anecdotes and quotes to liven up the report

! Ask audience how to improve reports

20

Assessment

@

Dartmouth

When to Graph

! Use graphs when to illustrate relations among measurements

! Use a precise title to clarify message and data supporting it

! Organize the graph to help reader answer the question resulting from the graph

! Use concepts and displays that are familiar

21Kosslyn, S. M. (1994). Elements of graph design. New York: W. H. Freeman and Company

Assessment

@

Dartmouth

Good and Bad Reports

! What are characteristics of:

! Good reports

! Bad reports

22

Assessment

@

Dartmouth

Fonts

! Safe fonts include Times Roman, Bookman,

Century Schoolbook, Garamond, or Arial

! Use both CAPITAL and small letters

! Use italics sparingly

! Use underlining sparingly

! Use boldface for emphasis

! Mix fonts in the same document, but use only 2-3

23

Assessment

@

Dartmouth

Tables

! Order table rows and columns in a way that makes sense

! Round number to whole numbers if possible using one decimal place for more technical audiences.

! Use mean, median, or total for comparison

! Column headings should be about same length

24

Adapted from Wainer, H. (1992, January-February). Understanding graphs and tables.

Educational Researcher, 12-23.

Assessment

@

Dartmouth

Tables

Dartmouth College Enrollment by Race/Ethnicity for 2003-2008

25

Non-

resident

alien

Black,

non-

Hispanic

American

Indian/

Alaskan

Native

Asian or

Pacific

Islander

Hispanic White,

non-

Hispanic

Other/

Race-

ethnicity

unknown

2003-2004 5% 6% 3% 12% 6% 58% 10%

2004-2005 5% 6% 3% 13% 6% 57% 8%

2005-2006 5% 6% 3% 13% 6% 57% 8%

2006-2007 6% 6% 4% 13% 6% 57% 7%

2007-2008 7% 6% 4% 13% 6% 57% 6%

Data is from the Common Data Set years 2003-2008

Assessment

@

Dartmouth

Charts

! Pie charts show proportional size

26

Dartmouth Undergraduate Composition

by Race/Ethnicity 2007-2008

Non-resident

alien

7%

Asian or

Pacific

Islander

13%

Hispanic

6%

Other/Race/e

thnicity

unknown

6%

White, non-

Hispanic

57%

Black, non-

Hispanic

7% American

Indian or

Alaska

Native

2007-2008 Dartmouth Common Data Set

Assessment

@

Dartmouth

Charts

! Horizontal bar charts show proportional relationships

27


by Race/Ethnicity 2007-2008

0% 10% 20% 30% 40% 50% 60%

Black, non-Hispanic

American Indian or

Alaska Native

Asian or Pacific

Islander

Hispanic

White, non-Hispanic

Other/Race/ethnicity

unknown

Percen

tag

e

Racial/Ethnic Group

2007-2008 Dartmouth Common Data Set

Assessment

@

Dartmouth

Charts

! Cluster bar charts allow grouping of multiple variables for comparison over time

28Dartmouth Common Data Sets


by Race/Ethnicity by Year 2003-2006

0%

10%

20%

30%

40%

50%

60%

70%

Non-r

esid

ent

alien

Bla

ck,

non-H

ispanic

Am

eri

can I

ndia

n o

r

Ala

ska N

ative

Asia

n o

r Pacific

Isla

nder

His

panic

White,

non-H

ispanic

Oth

er/

Race/e

thnic

ity

unknow

n

Racial/Ethnic Group

Percen

tag

e

2003-2004

2004-2005

2005-2006

Assessment

@

Dartmouth

Charts! Cluster bar charts allow grouping of multiple

variables for comparison over time



by Race/Ethnicity by Year

0%

10%

20%

30%

40%

50%

60%

70%

2003-2004 2004-2005 2005-2006

Percen

tag

e

Non-resident alien

Black, non-Hispanic

American Indian or AlaskaNative

Asian or Pacific Islander

Hispanic

White, non-Hispanic

Other/Race/ethnicityunknown

Assessment

@

Dartmouth

Charts

! Line charts show trends

30

Dartmouth Common Data Sets

Dartmouth Degrees Awarded by Year

0

200

400

600

800

1000

1200

2003 2004 2005 2006 2007

Nu

mb

er

Bachelor's degrees

Master's degrees

Doctoral degrees

Assessment

@

Dartmouth

Charts

! One hundred percent bar charts display how proportional relationships change over time


0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2003-2004 2004-2005 2005-2006 2006-2007 2007-2008

Percen

tag

e

Year

Dartmouth Undergraduate Composition by Race/Ethnicity by Year

Other/Race/ethnicity unknown

White, non-Hispanic

Hispanic


American Indian or Alaska Native

Black, non-Hispanic

Non-resident alien

Assessment

@

Dartmouth

Charts

! Would charts be useful in your reports?

! If so, which ones might be most useful?

32

Assessment

@

Dartmouth

Guidelines for Presenting Charts

! Keep charts simple


2003-2004 2004-2005 2005-2006 2006-2007 2007-2008

Other/Race/ethnicity unknown 10% 8% 8% 7% 6%

White, non-Hispanic 58% 57% 57% 57% 57%

Hispanic 6% 6% 6% 6% 6%

Asian or Pacific Islander 12% 13% 13% 13% 13%

American Indian or Alaska Native 3% 3% 3% 4% 4%

Black, non-Hispanic 6% 7% 7% 7% 7%

Non-resident alien 5% 5% 5% 6% 7%

5% 5% 5% 6% 7% 6% 7% 7% 7% 7% 3% 3% 3% 4% 4% 12% 13% 13% 13% 13% 6% 6% 6% 6% 6%

58% 57% 57% 57% 57%

10% 8% 8% 7% 6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percen

tag

e

Year

Dartmouth Undergraduate Composition by Race/Ethnicity


White, non-Hispanic

Hispanic



Black, non-Hispanic

Non-resident alien

Assessment

@

Dartmouth


! Present variables according to logic



for 2003-2008

0%

10%

20%

30%

40%

50%

60%

70%

Non-r

esid

ent

alien

Bla

ck,

non-H

ispanic

Am

erican I

ndia

n o

r

Ala

ska N

ative

Asia

n o

r Pacific

Isla

nder

His

panic

White,

non-H

ispanic

Oth

er/

Race/e

thnic

ity

unknow

n

Racial/Ethnic Group

Percen

tag

e 2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

Assessment

@

Dartmouth


! When comparing charts, keep displays comparable

35


for 2003-2005

0%

10%

20%

30%

40%

50%

60%

Non-r

esid

ent

alien

Bla

ck,

non-H

ispanic

Am

erican I

ndia

n o

r

Ala

ska N

ative

Asia

n o

r Pacific

Isla

nder

His

panic

White,

non-H

ispanic

Oth

er/

Race/e

thnic

ity

unknow

n

Racial/Ethnic Group

Percen

tag

e

2003-2004

2004-2005


for 2006-2008

0%

25%

50%

75%

100%

Non

-res

iden

t alien

Black

, no

n-Hispa

nic

Amer

ican

Ind

ian or

Alask

a Native

Asian

or Pa

cific

Islan

der

Hispa

nic

White, no

n-Hispa

nic

Oth

er/R

ace/

ethn

icity

unk

nown

Racial/Ethnic Group

Percen

tag

e

2006-2007

2007-2008


Assessment

@

Dartmouth


! Use common sense for use of grid marks, tick marks, and labels


2003-2004 2004-2005 2005-2006 2006-2007 2007-2008

Other/Race/ethnicity unknown 10% 8% 8% 7% 6%

White, non-Hispanic 58% 57% 57% 57% 57%

Hispanic 6% 6% 6% 6% 6%

Asian or Pacific Islander 12% 13% 13% 13% 13%

American Indian or Alaska Native 3% 3% 3% 4% 4%

Black, non-Hispanic 6% 7% 7% 7% 7%

Non-resident alien 5% 5% 5% 6% 7%

5% 5% 5% 6% 7% 6% 7% 7% 7% 7% 3% 3% 3% 4% 4% 12% 13% 13% 13% 13% 6% 6% 6% 6% 6%

58% 57% 57% 57% 57%

10% 8% 8% 7% 6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percen

tag

e

Year



White, non-Hispanic

Hispanic



Black, non-Hispanic

Non-resident alien

Assessment

@

Dartmouth


! Avoid three dimensions


20032004

20052006

2007

Bachelor's degrees

Master's degrees

379 417 423 440441

1021 1061 1109

10291021

0

200

400

600

800

1000

1200

Number

Year

Dartmouth Degrees Awarded 2003-2007

Bachelor's degrees

Master's degrees

Assessment

@

Dartmouth




1021

379

67

1061

417

56

1109

423

70

1029

440

60

1021

441

73

0

200

400

600

800

1000

1200

Number

2003 2004 2005 2006 2007

Year


Bachelor's degrees

Master's degrees

Doctoral degrees

Assessment

@

Dartmouth





10211061

1109

1029 1021

379417 423 440 441

67 56 70 60 73

0

200

400

600

800

1000

1200

2003 2004 2005 2006 2007

Year

Number

Bachelor's degrees

Master's degrees

Doctoral degrees

Assessment

@

Dartmouth


! Use a maximum of six segments in a pie chart

40

Dartmouth Enrollment Fall 2007

Degree-seeking,first-time freshman

Other first-year,degree seeking

All other degree-seeking

All oftherundergraduatesenrolled

First time, first-professionalstudents

All other first-professionalstudents

Degree-seeking,first-time graduatestudents

All other degree-seeking graduatestudents

All other graduatesenrolled in creditcourses


Assessment

@

Dartmouth


! Use no more than five or six data series on any single line graph

41

Dartmouth Enrollment by Year 2004-2007

0

500

1000

1500

2000

2500

3000

3500

2004 2005 2006 2007

Year

Nu

mb

er

Degree-seeking, first-time

freshman

Other first-year, degree

seeking


All ofther undergraduates

enrolled

First time, first-

professional students

All other first-professional

students

Degree-seeking, first-time

graduate students


graduate students

All other graduates

enrolled in credit courses


Assessment

@

Dartmouth

PowerPoint - Don!ts

42

Assessment

@

Dartmouth

Most Useful Tips

! Which tips will be most useful to remember when presenting results and writing reports?

43

Assessment

@

Dartmouth

Conclusion

! Any final thoughts, questions, or comments?

44

Assessment

@

Dartmouth

Additional Resources

! Fraenkel, J. R. & Wallen, N. W. (2003). How to design and evaluate research in education (5th ed.). New York: McGraw-Hill. P. 610.

! Suskie, L. A. (1996). Questionnaire survey research: What works (2nd ed.). Tallahassee, FL: Association or Institutional Research.

! Upcraft, M. L. & Schuh, J. H. (1996). Assessment in student affairs: A guide for practitoners. San Francisco: Jossey-Bass. Pp. 282-286.

45

Documents

Assessment@Dartmouth Resource Making Your Case: Presenting … · 2019. 12. 10. · Cone of Experience!10% of what we read!20% of what we hear!30% of what we see!50% of what we see