42
ASSESSMENT OF LIBRARY INSTRUCTION @ Randall Library

Assessment of Library Instruction

  • Upload
    cale

  • View
    67

  • Download
    0

Embed Size (px)

DESCRIPTION

@ Randall Library. Assessment of Library Instruction. What is Assessment?. “Assessment is an ongoing process aimed at understanding and improving student learning.” 1 Main purposes: 2 Increase student learning Strengthen instructional program Answer calls for accountability. - PowerPoint PPT Presentation

Citation preview

Page 1: Assessment of Library Instruction

ASSESSMENT OFLIBRARY INSTRUCTION @ Randall Library

Page 2: Assessment of Library Instruction

What is Assessment?“Assessment is an ongoing

process aimed at understanding and improving student learning.”1

Main purposes: 2

Increase student learning Strengthen instructional programAnswer calls for accountability

http://www.acrl.org/ala/mgrps/divs/acrl/issues/infolit/resources/assess/issues.cfm

(1)Oakleaf & Kaske, 2009 (2)

Page 3: Assessment of Library Instruction

Why Assess Library Instruction?To increase student learningTo improve instructionTo inform future assessment effortsTo provide data for accountability

measuresTo collaborate with facultyTo collaborate with other librariansTo argue for increased resources

http://libraryassessment.org/bm~doc/oakleaf_assessment_cycle.ppsForward to Slide 9

Page 4: Assessment of Library Instruction

Why Assess Library Instruction?“Well done assessment is essential to

documenting the effects of information literacy programs and the degree to which they are meeting their objectives. The assessment process can also assist anyone involved in information literacy instruction by getting them actively involved in the process and engaged in classroom activities, by giving them feedback about students’ thinking and learning, and by helping them focus on goals and objectives …”

Assessing Student Learning Outcomes for Information Literacy Instruction in

Academic Institutions

Page 5: Assessment of Library Instruction

Resistance and Misperceptions“_____ is making us do this.”

“_____ is trying to tell us how to teach.”

“We have to get this over with.” (Once and done)

“I don’t have time for this.”

“I have to develop a pre-test and a post-test.”

“Assessment is a fad.”

Page 6: Assessment of Library Instruction

What Needs to be Assessed?Macro to micro

◦UNCW curriculum as a whole◦All UNCW programs / majors◦All UNCW courses◦Current “Basic Studies” program / New

“University Studies” program (General Education) Information Literacy (IL) component of

University Studies curriculum (as a whole) “Information Literacy courses” within IL component

Library instruction program Individual library instruction sessions

Page 7: Assessment of Library Instruction

Past Library Instruction AssessmentLibrary Instruction Sessions

◦Primary focus on teaching evaluation Online student evaluation forms Peer (librarian) observations Faculty evaluations of teaching (Attending instructor)

Focus has been on us◦Perceptions of teaching◦ In general, little measure of what students are

actually learning / can do Some measure of what they feel they are learning Indirect assessment

Page 8: Assessment of Library Instruction

Moving ForwardComprehensive library

instruction assessment plan

◦Instruction program (holistically)

◦Student learning outcomes for individual instruction sessions

Ongoing, continuous process

Page 9: Assessment of Library Instruction

Strategic Plan for Assessment Determine purpose(s) of assessment Examine mission of the university, library, and

instruction program Examine structure of library instruction program Examine and redefine program goals Collect and examine relevant data / statistics related to

program goals (e.g. How many students take ENG 201?) Examine program resources Examine / establish collaborations with other faculty Establish aggressive yet attainable timeline

◦ What will we assess?◦ How will we assess?◦ When will we assess?◦ What will we change because of assessment data?

Page 10: Assessment of Library Instruction

Purpose(s)Why are we engaging in assessment?What do we hope to gain from

assessment efforts?

Main purposes: Increase student learning Strengthen instructional programAnswer calls for accountabilitySee slide #3

Page 11: Assessment of Library Instruction

Other PurposesConnection between library instruction

assessment and UNCW assessment◦ Only area offering courses not represented in UNCW

assessment efforts (?)◦ No representation on Learning Assessment Council◦ Instruction sessions a component of UNCW courses

Promotion of library instruction program and information literacy

Internally-focused purposes:◦ Improve workload, workflow, organizational

structures in library, etc.

Page 12: Assessment of Library Instruction

Driving Forces (Randall Library)Formation of Information Literacy Strategic

Planning Team (Fall 2008)Instructional Services Coordinator training:

◦ ACRL Immersion Program (Summer 2008)◦ Master of Science in Instructional Technology (MIT)

MIT 531 (Assessment of Learning Outcomes) (Summer 2009)

◦ LOEX Conference (May 2010)◦ NCLA Bibliographic Instruction Group: Assessment &

Student Learning Outcomes Workshop (May 2010)◦ Georgia Conference on Information Literacy

(October 2010)◦ Library Assessment Conference (October 2010)

Page 13: Assessment of Library Instruction

ILSPT Mission Statement:

“The Information Literacy Strategic Planning Team will lead long-term, strategic planning to help transform the current set of instructional services activities into a well conceived, planned program that is integrated into both the basic studies curriculum and into each major. An information literacy program is one that utilizes well placed instruction sessions and tutorials/online modules to integrate information literacy skills into the curriculum of courses and builds skills progressively through a student’s college career.”

Add: Led by the Coordinator of Instructional Services, ILSPT will plan for and oversee the assessment of library instruction …

Page 14: Assessment of Library Instruction

Driving Forces (RL & UNCW)UNCW SACS accreditation

New/revised “University Studies” program which includes Information Literacy Component

Need for library to demonstrate importance, effectiveness, need for resources, etc.

Page 15: Assessment of Library Instruction

Driving Forces (UNCW)Highest priority: Student learning

UNCW shift to culture of assessment◦2005 campus assessment workshops (Susan

Hatfield, consultant)◦Learning Assessment Council◦ Institutional Research & Assessment◦Assessment directors ◦Assessment of Basic Studies

National Survey of Student Engagement (NSSE) Collegiate Learning Assessment (CLA) General Education Assessment Committee (Report)

Page 16: Assessment of Library Instruction

Assessment so Far (UNCW)UNCW Learning GoalsGeneral Education Report (March 2009)On Information Literacy

◦No librarians were consulted in the construction of this process / report

◦Instructional Services Coordinator contacted the Director of University Planning and sent the Information Literacy Competency Standards to Director through campus mail

◦Communication was not returnedhttp://uncw.edu/assessment/Documents/General%20Education/GenEdAssessmentCommitteeReportMarch2009.pdf

Page 17: Assessment of Library Instruction

General Education Report“All UNCW freshman seminar

participants undergo instruction by the Randall Library in locating and using information, although there is no current assessment of student’s information literacy proficiency either before or after that instruction.”

Page 18: Assessment of Library Instruction

General Education Report“Recent data indicate that college

students have difficulty developing a topic question and are often not aware of informational resources available to them. Even when aware of appropriate information resources, they may lack the ability to adequately evaluate and synthesize information. This can result in classroom work based on haphazard discussion groups or personal opinion.”Albitz, R.S. (2007) The what and who of information literacy and critical thinking in higher

education. Libraries and the Academy. 7, 1, 97‐109

Page 19: Assessment of Library Instruction

General Education Report“Of 37 common learning outcomes

established for the Basic Studies categories, only two (CMP3 and IP1A) fit into the category of information literacy. Perhaps there is not sufficient emphasis in the spectrum of Basic Studies courses on information literacy in a broad range of disciplines, or perhaps additional common learning outcomes should be developed to encompass efforts that are currently being made.”

CMP3: “Employ a writing process that includes finding, evaluating, analyzing, and synthesizing appropriate primary and secondary sources, and successfully integrating personal knowledge with source material.”IP 1A: “Access and analyze information about topics or issues that are interdisciplinary or transdisciplinary.”

Page 20: Assessment of Library Instruction

National Survey of Student Engagement “First-year students were asked in NSSE about the

frequency with which they ‘worked on a paper or project that required integrating ideas or information from various sources,’ a component of information literacy. UNCW first-year students reported a frequency that was statistically significantly below that reported by our selected peers, significantly below that reported by national master’s universities, and significantly below that reported by all NSSE 2007 institutional participants. This information led to the development of a rubric-based assessment plan for information literacy to be implemented with the comprehensive assessment of Basic Studies beginning Fall 2009.”

Assessment tool selected:◦ The American Association of Colleges and Universities’ (AAC&U)

“Information Literacy Metarubric”

http://uncw.edu/assessment/BasicStudiesAssessment.htmlhttp://uncw.edu/assessment/Documents/General%20Education/2004-2005NSSE-

BasicStudies.pdf

Page 21: Assessment of Library Instruction

Merging EffortsInformation Literate

UNCW Graduates

UNCW (University Studies

Assessment)

Randall Library (Library Instruction Assessment)

Page 22: Assessment of Library Instruction

On the HorizonUniversity Studies (Fall 2011)

Students who complete the University Studies program will:◦Goal 3. Locate, evaluate, and

effectively use information by applying a variety of academic and technological skills (Information Literacy)

http://www.uncw.edu/universitystudies/documents/Univ.StudiesCurriculumReport.pdf

Page 23: Assessment of Library Instruction

Information Literacy Curriculum Component of University Studies

curriculum“Designed to foster the development

of students’ information literacy skills by requiring three information literacy intensive courses: The First Year Experience and two additional information literacy intensive courses, with at least one in the major field of study.”

Page 24: Assessment of Library Instruction

Information Literacy Curriculum Based on the

Information Literacy Competency Standards for Higher Education

The information literate student …1. Determines the nature and extent of the information

needed.2. Accesses needed information effectively and

efficiently.3. Evaluates information and its sources critically and

incorporates selected information into his or her knowledge base and value system.

4. Individually or as a member of a group, uses information effectively to accomplish a specific purpose.

5. Understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally.

Page 25: Assessment of Library Instruction

Information Literacy Curriculum University Studies Information Literacy Student

Learning Outcomes (adopted by Faculty Senate November 2009)

The student will: IL 1. Be able to determine the nature and extent of

information needed to solve a problem. IL 2. Access information effectively and efficiently from a

variety of sources. IL 3. Evaluate information critically and incorporate

appropriate information into his or her knowledge base. IL 4. Individually, or as a member of a group, use

information effectively to accomplish a specific purpose. IL 5. Understand many of the economic, legal, and social

issues surrounding the use of information and access and use information ethically and legally.

Page 26: Assessment of Library Instruction

Information Literacy Curriculum

9 hours of courses

(“Double dipping”

from other component

s)

Content reinforced throughout curriculum

Course in the Major (Research Methods or other)(3 hours)Library instruction sessions focus on IL Comp Standards 2 (Accessing info through discipline specific resources) and 5 (Ethical use: Discipline specific citations and writing)

English Composition or other (e.g. LIB 103) (3 hours) Library instruction sessions focus on IL Comp Standards 1 (Determining info needed), 2 (Accessing info), and 4 (Using info effectively)

Freshman Seminar (3 hours)Library instruction sessions focus on IL Comp Standards 3 (Evaluating information) and 5 (Ethical use: Plagiarism)

Page 27: Assessment of Library Instruction

What’s Different for Us?Every major must have an “information

literacy intensive course”Faculty must supply information literacy

student learning outcomes for that courseInformation literacy on the “radar” in a new

way◦“Just send ‘em to the library” does not apply◦Potential for collaborations that did not exist

beforeInformation Literacy is a goal of the entire

University Studies program ◦The university will assess this

Page 28: Assessment of Library Instruction

Library Instruction Program ShiftFrom bibliographic instruction sessions to

information literacy◦ Moving from scattered “one shot” instruction sessions to

integration of information literacy skills instruction into UNCW curriculum

Defining the “Instruction Program”◦ Information literacy curriculum (University Studies)

Freshman Seminar instruction English composition instruction (ENG 101, 103, and 201) Instruction in each major

◦ Workshops & tours◦ Online tutorials◦ LIB Courses◦ Regional engagement / instruction (high school student

instruction, Pathways workshops, etc.)

Page 29: Assessment of Library Instruction

Current Library Instruction AY 2008-2009 Course-related instruction sessions, workshops, and

tours ◦ 587 sessions◦ 678 contact hours◦ Teaching 10,750 users

PLUS

Thirty four hours of credit courses offered◦ LIB 101 (4 sections, 1credit hour, 19 students in each) ◦ LIB 103 (9 sections, 3 credit hours, 19 students in

each) ◦ LIB 104 (1 section, 3 credit hours, 19 students in

each)https://share.uncw.edu/sites/aa/library/pubsvcs/Shared%20Documents/Annual%20Reports/annrept0809.docx

Page 30: Assessment of Library Instruction

Moving to Assessment: A SystemPart 1: Expected Outcomes (Part of Assessment

Plan)

Part 2: Assessment Methods and Tools (Part of Assessment Plan)

Part 3: Implementation (Part of Assessment Plan)

Part 4: Summary of Findings (Part of Assessment Report)

Part 5: Actions Taken (Part of Assessment Report)

http://www.uncw.edu/cas/assessmentplanandreport.html

Page 31: Assessment of Library Instruction

Steps of Assessment1. Determine what we need to assess by creating Student Learning Outcomes

for: Freshman Seminar library instruction sessions English Composition (ENG 101, 103, and 201) library instruction

sessions Instruction sessions in each major (as applicable) LIB Courses ILSPT has reviewed ACRL Standards

SLOs will “map” to ACRL Standards

2. Determine how we are going to assess Means of Assessment (what method or tool we will use)

“My Research Strategy Worksheet” (scored using a rubric), sample of student products, online quiz, “5 minute paper,” tests, etc.

3. Determine what and when we will assess

4. Review assessment information

5. Make changes based on assessment informationRepeat all steps continuously

Page 32: Assessment of Library Instruction

Assessment CycleImplement Changes

FormulateOutcomes

Develop Means of

Assessment

Gather &Analyze

Data

InterpretResults

Propose Improveme

ntsto Program Library

Instruction Program

Page 33: Assessment of Library Instruction

Assessment Checklist1. What are our research questions? (What are we trying to discover about

student skills, knowledge, abilities, etc.; and what evidence do we have already?)

2. What is the expected level of performance?3. When in the students’ career do we assess this outcome? (entry, end of

sophomore year, senior, etc.)4. In which course(s) or venue?5. What student work/artifacts are collected?6. How is the student work evaluated? (criteria/rubric)7. Who evaluates the student work?8. Who analyzes the results?9. Where do recommendations for action go?10. Who takes action? (And how do we ensure changes are evidence‐based and

data‐driven?)11. How is the process documented?12. Where is the documentation kept?13. What is the timetable/schedule for determining which outcomes are

assessed when?Developed by the General Education Assessment Committee for designing assessment of a learning outcome.

Page 34: Assessment of Library Instruction

Who Is Going to Do This?All Instruction Librarians

◦Comprehensive plan coordinated by Instructional Services Coordinator Supported by Information Literacy Strategic Planning

Team◦Assessment at micro level (instruction sessions)

Done by individual librarians with assistance from Instructional Services Coordinator

Will seek guidance from campus experts (e.g. Dr. Linda Seifert, Director of Assessment, College of Arts and Sciences)

Page 35: Assessment of Library Instruction

First StepsCompletion of assessment structure and plan

outline (Summer 2010)Creation of assessment website (Summer/Fall

2010)LIB 103 assessment planning (Begin Fall 2010)Freshman Seminar assessment planning

◦ SLOs and means of assessment (Begin Fall 2010)English Composition (ENG 101, 103, and 201)

assessment planning◦ Draft of SLOs (Fall 2010)◦ Assessment of SLOs (Spring 2011)

Assessment planning for the majors◦ SLOs and means of assessment (Begin Fall 2010)

Page 36: Assessment of Library Instruction

Other Assessment & EvaluationAssessment/Evaluation of:

◦Learners (SLOs)◦Learning Opportunities◦Learning Materials◦Learning Environment◦Instructors

Page 37: Assessment of Library Instruction

Assessment & Evaluation SWOT/SOAR of library

instruction program (Fall 2010)

Evaluation of library instruction program coordinator based on “Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators” (Fall 2010)

Self analysis of librarians based on “Association of College and Research Libraries Standards for Proficiencies for Instruction Librarians and Coordinators” (Fall 2010)

Continuous use of online student evaluation forms

Continued annual peer assessment of teaching

Continued use of SPOTS for LIB courses

Future implementation of faculty surveys and evaluations

Review benchmarking data (Collected in fall 2008 and spring 2009)

Review of LibQual+ Survey results (Administered in 2008)

UNCW efforts: NSSE, CLA, General Education Assessment, etc.

Page 39: Assessment of Library Instruction

Data ManagementWASSAIL (Open source)

“WASSAIL is a database-driven, web-based application employing PHP, MySQL, and Javascript/AJAX technologies. It was created to manage question and response data from the Augustana Library's library instruction sessions, pre- and post-tests from credit bearing information literacy (IL) courses, and user surveys. It has now expanded beyond its original function and is being used to manage question and  response data from a variety of settings.”

http://www.library.ualberta.ca/augustana/infolit/wassail/

Page 40: Assessment of Library Instruction

Helpful ResourcesACRL Information Literacy website (

http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm)

ACRL Institute for Information Literacy (http://www.ala.org/ala/mgrps/divs/acrl/issues/infolit/index.cfm)

ARL Library Assessment Conference (http://libraryassessment.org)

“A Twenty Year Path: Learning about Assessment; Learning from Assessment” (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER1/97)

Information Literacy in the Disciplines (http://wikis.ala.org/acrl/index.php/Information_literacy_in_the_disciplines)

Page 41: Assessment of Library Instruction

Helpful Resources Internet Resources for Higher Education Outcomes

Assessment (http://www2.acs.ncsu.edu/UPA/assmt/resource.htm)

Megan Oakleaf’s Presentations and Publications (http://meganoakleaf.info/presentations.html)

National Institute for Learning Outcomes Assessment (http://learningoutcomesassessment.org/)

Task Force on Academic Library Outcomes Assessment Report (http://www.ala.org/ala/mgrps/divs/acrl/publications/whitepapers/taskforceacademic.cfm)

Utah State University Library Instruction Assessment (http://library.usu.edu/instruct/assessment/index.php)

“Writing Information Literacy Assessment Plans: A Guide to Best Practice” (http://www.comminfolit.org/index.php/cil/article/view/Vol3-2009PER2/98)

Page 42: Assessment of Library Instruction

Library Instruction Assessment Website

Anne PembertonInstructional Services Coordinator

Chair, Information Literacy Strategic Planning [email protected]

http://library.uncw.edu/staff/anne_pemberton