Embed Size (px)
DESCRIPTION
Amy Hofer & Margot Hanson. From Loex-of-the-West 2010 conference program: "We need evidence of our value to justify the library budget, gain buy-in from faculty, and please accreditors, but the process of gathering meaningful evidence can be quite a challenge. After all, were you trained to do this kind of research? Neither were we, but the Assessment Coordinator that our university hired in preparation for a WASC visit guided us through our study design. Drawing on this relationship with a neutral expert outside the library, two business librarians at Golden Gate University designed an effective learning assessment of our instruction to our school’s intensive ESL program. We’ll use the valuable lessons we learned as a starting point for small-group exploration of some real-world assessment territory."
Citation preview
TERRA INCOGNITA WITH NO GPS:
A case study approach to mapping assessment
Amy Hofer, Distance Learning Librarian, Portland State UniversityMargot Hanson, Web Services Librarian, Golden Gate University
LOEX-of-the-West 2010, Mount Royal University, Calgary, Alberta
ABOUT US!‣Golden Gate University‣The IL landscape‣Our assessment needs
ABOUT YOU!‣Hello my name is...‣Your school‣Pressing assessment issues (past, present, or future)
PROGRAM EVALUATIONStudy your program to find out whether it’s meeting your goals
LEARNING ASSESSMENTStudy whether students gained knowledge, skills, and abilities as
a result of your instruction
PLEASE DISCUSS‣Think about an assessment project at your organization. ‣Learning assessment or program evaluation? ‣Right tool for the job?
LESSONS FROM LISA
‣ For what general purpose is the evaluation being conducted?
‣ What are the specific goals of the study?
‣ Who is the audience for the research findings?
‣ What kinds of information are needed to make the decisions you need to make?
‣ What data collection method will be appropriate?
OUR STUDY
POPULATIONS: FALL 2009
PLUS
‣ English language learners
‣ 8 librarian visits
‣ 2 librarians per visit
‣ 18.5 hours total
Undergraduate
‣ 43% ELL in our sample
‣ 3 librarian UG class visits out of 61 UG classes offered Fall 09 (none in our sample)
‣ 30-60 minutes per visit
METHODS
PLUS Students
‣ 43 completed tests before research instruction (paper)
‣ 40 completed tests after research instruction (paper)
‣ 39 writing samples
Undergraduate Students
‣ 51 completed tests (paper and online)
‣ Sample drawn from 9 classes with agreeable faculty
OUR AREAS OF INQUIRY
‣ A. Do students identify appropriate sources for their research need?
‣ B. Do students ask a great question that fits the scope of the assignment and requires research and analysis to answer?
‣ C. Do students demonstrate academic integrity by correctly citing sources?
‣ D. Do students use the library’s resources and services as part of the research/writing process?
‣ E. Qualitative info about students
http://tinyurl.com/lotw2010
Didn't Get It Got It Got It & Ran With It!
Appropriate Sources
•Uses only one format •Not relevant or authoritative•Heavy reliance on one or two sources
•Uses a range of formats, including reports, articles and websites•Uses only resources presented in class
•Uses a range of formats, including reports, articles and websites•Includes resources only available from the library, but went beyond resources presented in class•Sources are relevant and substantive
Topic Development
•Topic too narrow or too broad for assignment•Uses report format, as a simple statement of facts, missing original analysis
•Research question is of appropriate scope for assignment•Includes original analysis•Question can be answered with resources presented in class
•Uses critical thinking and creative approach to topic, which is of appropriate scope•Research question requires original analysis•Question requires additional sources beyond what was presented in class
Citation
•Errors in APA formatting•Incorrect use of paraphrasing, quotations, or summarizing•Quotes are dumped into paper and not incorporated into analysis•Originality report reveals problems with exact matches
•Consistent APA formatting in-text and reference list•Originality report reveals less than 10% exact match
•APA correctly and consistently used for in-text and reference list citations•Citations are distributed evenly throughout paper, contribute to analysis, and support conclusions
You don’t have to read this - it’s online!
FINDINGS
OMG, data!
Most Improved Players
0%
10%
20%
30%
40%
50%
60%
70%
60%66%
54%
PLUS pretest PLUS posttest Undergrad
Test
Sco
re
0
8
16
23
31
39
8810
2113
18
1018
11
What the PLUS work samples told us
Appropriate sources Topic development Citation
Didn’t get itGot itGot it & ran with it!
Stud
ent
Pape
rs
Undergrads: How many times during the semester do you get help from a librarian?
2%
46%
24%
22%
6%
More than 5 times2-5 timesOne timeNeverI don’t know
Do you think that the library and its services contribute to your academic success at GGU?
5%13%
83%
PLUS
2%10%
18%
20%
51%
Undergrad
Yes Somewhat Maybe No I don’t know
Do students know what they know?
0
5
10
15
20
0 1 2 3 4 5 6 7 8 9 10
I don’t knowMy skills are poorMy skills are fairMy skills are goodMy skills are excellent
Test Scores
Stud
ents
NEXT STEPS‣Connections to your own situation or small-group discussions?‣Strategic outreach - getting a foot in the door?‣Questions for us?
Image sources:
http://www.flickr.com/photos/telmo32/2967379555http://www.flickr.com/photos/rhiannonstone/3782706061http://www.wordle.net http://www.flickr.com/photos/princessashley/1546575032http://www.flickr.com/photos/gadl/320300354http://www.flickr.com/photos/mcaven/4319012152http://www.flickr.com/photos/huffstutterrobertl/3269126061/http://www.flickr.com/photos/quinnanya/3183174451/
