Embed Size (px)
DESCRIPTION
Guess what -- you don't need to do learning assessment on a 45-minute one-shot presentation. Instruction librarians at Golden Gate University learned this and much more when an Assessment Coordinator arrived to help prepare our school for WASC. Oakleaf & Hinchliffe (2008) identify lack of coordination as one of the barriers librarians face in conducting assessment, and we found that having a smart, committed, and trustworthy coordinator made all the difference to our research project. We leveraged the Assessment Coordinator's expertise to stay focused on a project that produced valid and useful results from an in-depth learning assessment to measure student learning in an English Language Learners program. Our presentation focuses on the people connections that made this assessment work: between librarian collaborators, with students and instructors in the ELL program, and all the way upstairs to our University-wide assessment coordinator. We'll talk about how we designed our assessment and - phew - let go of post-instruction evaluation forms. Participants will get a fresh look at how information literacy assessment can benefit from upstairs-downstairs collaboration (floor plan not included)!
Citation preview
UPSTAIRS-DOWNSTAIRS:Working with a campus assessment coordinator and
other allies for effective information literacy assessment
Amy Hofer, Research Instruction LibrarianMargot Hanson, Web Services Librarian
Golden Gate UniversityCARL Conference, Sacramento CA, April 9, 2010
Monday, April 12, 2010
ASSESSMENT CYCLE OR CIRCULAR FILE?
Monday, April 12, 2010
GOLDEN GATE
UNIVERSITYabout us!
Monday, April 12, 2010
RESEARCH INSTRUCTIONMonday, April 12, 2010
ACADEMIC CHAMPIONSMonday, April 12, 2010
PLUSPreparation in Language & University Studies
Monday, April 12, 2010
Monday, April 12, 2010
PROGRAM EVALUATIONStudy your program to find out whether it’s meeting your goals
Monday, April 12, 2010
LEARNING ASSESSMENTStudy whether students gained knowledge, skills, and abilities as
a result of your instruction
Monday, April 12, 2010
Monday, April 12, 2010
THE OLD SURVEY
It’ll be bigger in a minute!
Monday, April 12, 2010
1) Will this library instruction session help you with your research assignment for this class?
Yes Maybe No
Monday, April 12, 2010
2) What did you find valuable about the session? (mark all that apply)
Learning about library services Learning about developing keywords Learning how to select relevant resources Learning how to search databases Finding out about online research guides Receiving course-specific handouts Learning about when to cite sources Other, please specify:
Monday, April 12, 2010
3)What information received today is still unclear to you?
Monday, April 12, 2010
4) What would you have liked to learn that was not covered in this session?
Monday, April 12, 2010
THE OLD SURVEY
‣Timing - right after instruction
‣How can we use this information?
Monday, April 12, 2010
THE LITERATUREMonday, April 12, 2010
OFF TO THE RACESMonday, April 12, 2010
Monday, April 12, 2010
UPSTAIRS-DOWNSTAIRS
Program Evaluation Groupand other allies
Monday, April 12, 2010
LESSONS FROM LISA
‣ For what general purpose is the evaluation being conducted?
‣ What are the specific goals of the study?
‣ Who is the audience for research findings?
‣ What kinds of information are needed to make the decisions you need to make?
Identify Study Objectives
Monday, April 12, 2010
LESSONS FROM LISA
‣ Test/Quiz
‣ Systematic examination of work samples
‣ Survey
‣ Individual interviews
‣ Focus group interviews
‣ Field observations
‣ Secondary data analysis
‣ Mixed methods approaches
What data collection method is appropriate?
Monday, April 12, 2010
OUR AREAS OF INQUIRY
‣ A. Do students identify appropriate sources for their research need?
‣ B. Do students ask a great question that fits the scope of the assignment and requires research and analysis to answer?
‣ C. Do students demonstrate academic integrity by correctly citing sources?
‣ D. Do students use the library’s resources and services as part of the research/writing process?
‣ E. Qualitative info about students
Monday, April 12, 2010
FOR EXAMPLE:
If you have just found 5 sources for your research paper, what is the best next step to take?
a. Scan (pre-read) a little bit of what you already have to find out whether you need more information
b. Brainstorm new topics in case you want to change your mind
c. Search for 5-10 more articles
d. Print everything out
e. I don’t know
Monday, April 12, 2010
FOR EXAMPLE:
Which of the following topics is a great idea for a research paper?
a. How has the Cash for Clunkers stimulus program affected the economic well-being of single mothers who lost their homes in Hurricane Katrina?
b. Cancer
c. Is communication the key to good management skills?
d. Is it ethical to target advertising to very young children in order to build long-term brand loyalty?
e. I don’t know
Monday, April 12, 2010
FOR EXAMPLE:
Why should you cite your sources?
a. If you don’t cite, you’re stealing somebody else’s words or ideas
b. So that your professor can find the source of the information you used
c. Because it is the expectation of the academic community that scholars build on the ideas of others
d. All of the above
e. I don’t know
Monday, April 12, 2010
FOR EXAMPLE:What is the most important difference between searching for information on Google vs. in the library’s databases?
a. Google always gives you the full text of the resource
b. Library databases give you access to in-depth information not available on Google
c. Google is better because you will always get more hits
d. You don’t need to evaluate the information you find in library databases because it is always scholarly
e. I don’t know
Monday, April 12, 2010
Didn't Get It Got It Got It & Ran With It!
Appropriate Sources
•Uses only one format •Not relevant or authoritative•Heavy reliance on one or two sources
•Uses a range of formats, including reports, articles and websites•Uses only resources presented in class
•Uses a range of formats, including reports, articles and websites•Includes resources only available from the library, but went beyond resources presented in class•Sources are relevant and substantive
Topic Development
•Topic too narrow or too broad for assignment•Uses report format, as a simple statement of facts, missing original analysis
•Research question is of appropriate scope for assignment•Includes original analysis•Question can be answered with resources presented in class
•Uses critical thinking and creative approach to topic, which is of appropriate scope•Research question requires original analysis•Question requires additional sources beyond what was presented in class
Citation
•Errors in APA formatting•Incorrect use of paraphrasing, quotations, or summarizing•Quotes are dumped into paper and not incorporated into analysis•Originality report reveals problems with exact matches
•Consistent APA formatting in-text and reference list•Originality report reveals less than 10% exact match
•APA correctly and consistently used for in-text and reference list citations•Citations are distributed evenly throughout paper, contribute to analysis, and support conclusions
You don’t have to read this - it’s online!
Monday, April 12, 2010
OUR RUBRIC
Didn’t get it
Got it
Got it & ran with it!
Monday, April 12, 2010
FOR EXAMPLE:Did the student use appropriate sources?
‣ Didn’t get it: Uses only one format; not relevant or authoritative; heavy reliance on one or two sources
‣ Got it: Uses a range of formats, including reports, articles and websites; uses only resources presented in class
‣ Got it & ran with it!: Uses a range of formats, including reports, articles and websites; includes resources only available from the library, but went beyond resources presented in class; sources are relevant and substantive
Monday, April 12, 2010
PRELIMINARY FINDINGS
Monday, April 12, 2010
OMG, data!
Monday, April 12, 2010
Most Improved Players
0%
7%
14%
21%
28%
35%
42%
49%
56%
63%
70%
54%
66%
60%
PLUS pretest PLUS posttest Undergrad
Monday, April 12, 2010
0%
5%
10%
15%
20%
25%
30%
35%
40%
0 1 2 3 4 5 6 7 8 9 10
PLUS pretest PLUS posttest Undergrad
More Detail
Monday, April 12, 2010
CATEGORICALLY SPEAKING
Monday, April 12, 2010
0%
15%
30%
45%
60%
53% 55%60%
PLUS pretest PLUS posttest Undergrad
Do students identify appropriate sources for their research need?
Monday, April 12, 2010
0%
16%
32%
47%
63%
57%63%
53%
PLUS pretest PLUS posttest Undergrad
Do students ask a great question that fits the scope of the assignment and requires research and analysis to answer?
Monday, April 12, 2010
0%
20%
40%
60%
80%
48%
72%68%
PLUS pretest PLUS posttest UG
Do students demonstrate academic integrity by correctly citing sources?
Monday, April 12, 2010
0%
23%
45%
68%
90%
70%
90%
57%
PLUS pretest PLUS posttest UG
Do students use the library’s resources and services as part of the research/writing process?
Monday, April 12, 2010
1
2
3
4
5
6
7
8
9
10
11
2
5
2
7
6
6
2 3
Appropriate sources Topic development Citation
Didn’t get itGot itGot it & ran with it!
What the student papers told us
Monday, April 12, 2010
Monday, April 12, 2010
Do students know what they know?
0
5
10
15
20
0 1 2 3 4 5 6 7 8 9 10
I don’t knowMy skills are poorMy skills are fairMy skills are goodMy skills are excellent
Monday, April 12, 2010
ERROR ANALYSIS
Monday, April 12, 2010
Monday, April 12, 2010
Image sources:
http://www.flickr.com/photos/seattlemunicipalarchives/2680241972/http://www.flickr.com/photos/telmo32/2967379555/http://www.un.org/esa/socdev/unpfii/media/images/UN-LOGO%20copy.jpg http://www.flickr.com/photos/princessashley/1546575032/ http://www.wordle.net http://www.flickr.com/photos/kevinomara/415961040/http://www.flickr.com/photos/curtisnewton/4089464649/http://www.flickr.com/photos/gadl/320300354/http://www.flickr.com/photos/michaelcr/1425130599/http://www.flickr.com/photos/mcaven/4319012152/ http://www.flickr.com/photos/daikrieg/1294053038/ http://www.uscibooks.com/Taylorposter.jpg http://www.streetsigngenerator.com
Monday, April 12, 2010
DISCUSSION STARTERS1. A question from the ILI list: How do you evaluate evidence of critical
thinking?
2. If you feel that your current assessment efforts are ineffective, can your group identify institutional or methodological stumbling blocks that you could work on?
3.Do you have a spectacularly successful or unsuccessful assessment experience to share with your group?
4.Are you currently doing assessment that is: program evaluation; learning assessment; satisfaction surveys? Is it working for you?
5.Do you currently have partners in assessment at your institution? Are there individuals or groups you would like to collaborate with?
Monday, April 12, 2010
