View
2
Download
0
Category
Preview:
DESCRIPTION
Rubric Assessment of Student Responses to an Information Literacy Tutorial
Citation preview
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
1/43
Rubric Assessment of Student
Responses to an InformationLiteracy Tutorial
Megan OakleafLibrarian for Instruction & Undergraduate
Research
Steve McCannNCSU Libraries Fellow
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
2/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Presentation Objectives
Share our experiences.
In changing the way we assess our program
In adapting ACRL outcomes to our project
In selecting a learning artifact to assess
In piloting our assessment plan
Facilitate your experience with this
type of assessment.
Provide take away ideas.
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
3/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
4/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
5/43
NCSU Libraries
Instruction Program
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
6/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Information Literacy
Information literacy is a set of abilities
requiring individuals to "recognize when
information is needed and have theability to locate, evaluate, and use
effectively the needed information."
Association of College and Research Librarieshttp://www.ala.org/ala/acrl/acrlstandards
/informationliteracycompetency.htm
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
7/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
How have we
measured our success?
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
8/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
The Decision to Change
Goal: To assess student learning of
information literacy skills using
outcomes-based assessment.
Need 2 things
An Artifact to Assess
Outcomes to Measure
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
9/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
What artifact will we assess?
Interactions with students that could
yield assessment artifacts
50-minute one-shot workshops
Library Online Basic Orientation (LOBO)
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
10/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Disadvantages of Using
Workshops for Assessment Perceived lack of time for open-ended
responses, only m.choice and T/F are
options. Taught by numerous librarians who lack
assessment knowledge.
Inconsistent audiences & content. Incomplete spectrum of outcomesaddressed.
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
11/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Advantages of Using
LOBO for Assessment Forms basis for IL instruction at NCSU.
Reaches virtually all incoming freshmen.
Recently redesigned, includes open-
ended questions.
Captures student responses in a
searchable database.
Potential for rich data.
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
12/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
13/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
14/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
15/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
What outcomes will we assess?
Information Literacy Competency
Standards for Higher Education
5 Standards
22 Performance Indicators
87 Outcomes
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
16/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
What outcomes will we assess?
Objectives for Information Literacy
Instruction: A Model Statement for
Academic LibrariansStandards = 5
Perform. Indicators = 14
Outcomes = 35
Bullets = 133
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
17/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
18/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
What outcomes will we assess?
LOBO Objectives & Outcomes
Objectives = 5
Measurable
Outcomes= 45
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
19/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
First Steps
Set up DB to access student answers.
Match outcomes to questions.How will we know the outcomes been met?
Beginning, Developing, Exemplary
Pilot test a section of LOBO.
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
20/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
The Pilot Test
Problem: Students use web sources for
academic purposes without evaluating
their quality.
Are they duped by low-quality sites?
Can we teach them to be morecritical consumers of information?
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
21/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
22/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
23/43M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Evaluating Web Sites
What criteria are you looking for?
What clues can you find?
What specific example can you give
from the web site at hand?
Is the web site a good one for
you to use?
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
24/43
Evaluating Web Sites
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
25/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
26/43
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
27/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Pilot Test Results
What we found out from 50 randomly
selected student accounts
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
28/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
LOBO Sections - % Completed
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%100%
1. The
ResearchProcess
2. Defining
ResearchNeeds
3. Developing
a ResearchStrategy
4. Conducting
the Search
5. Evaluating
Resources
6. Using
Resources
Lobo DataJan-April, 04 1,830 total accounts new
accounts
46% of total questions
answered
Evaluated 50 accounts
Sections Total Answers Ttl Quest. Poss. Ans. % Complete
1. The Research
Process7,031 7 13,230 53%
2. Defining Research
Needs9,056 10 18,900 48%
3. Developing a
Research Strategy9,798 11 20,790 47%
4. Conducting the
Search11,984 12 22,680 53%
5. Evaluating
Resources11,582 16 30,240 38%
6. Using Resources 4,692 6 11,340 41%
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
29/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Objective 5: Evaluating Resources
#3 Evaluate Web Sites questions:1. Locate a website
2. Evaluate websites authority
3. Evaluate recency/currency4. Identify Bias/Point of View
Section 5: Evaluating Resources
Task Answered Null Possible % Complete1. Evaluate Books 4,455 8,824 13,279 34%
2. Evaluate Articles 1,570 2,224 3,794 41%
3. Evaluate Web Sites 5,557 7,722 13,279 42%
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
30/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Q1: Locate a Website
4 points possible: Average score 2.7
Question Text: Type the title and URL (web address) of the web site you will
evaluate here:
Q1: Cumulative Count of Scores
0
5
10
15
20
25
30
35
40
Total 0 3 36 0 11
Count-0 Count-1 Count-2 Count-3 Count-4
Q1: Count of Scores by Outcome
0
5
10
15
20
25
30
35
40
45
Beginning Developing Exemplary Beginning Developing Exemplary
Completion Accuracy
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
31/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Q2: Evaluate a Websites Authority
8 points possible: Average score 5.0
Question Text: Answer the questions above for the web site you're evaluating.
Overall, does what you know about the URL of the web site indicate that it's a good
resource?
Answer the questions above for the web site you're evaluating. Overall, does what
you know about the authorship of the web site indicate that it's a good resource?
Q2: Cumulative Count of Scores
0
2
4
6
8
10
12
14
16
18
Total 0 5 1 2 6 16 9 7 4
Count-
0
Count-
1
Count-
2
Count-
3
Count-
4
Count-
5
Count-
6
Count-
7
Count-
8
Q2: Count of Scores by Outcome
05
101520
25303540
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Us es Criteria Terms Cites Clues Cites Ex amples from Sourc e Judges Whether or Not To
Use
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
32/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Q3: Evaluate a Websites Currency
8 points possible: Average score 5.6
Question Text: Answer the questions above for the web site you're evaluating.
Overall, does what you know about the currency of the web site indicate that it's a
good resource?
Q3: Cumulative Count of Scores
0
2
4
6
8
10
12
14
16
Total 0 6 0 5 4 2 12 6 15
Count-
0
Count-
1
Count-
2
Count-
3
Count-
4
Count-
5
Count-
6
Count-
7
Count-
8
Q3: Count of Scores by Outcome
05
1015
20253035
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Us es Criteria Terms Cites Clues Cites Ex amples f rom Sourc e Judges Whether or Not To
Use
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
33/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Q4: Identify a Websites Bias
8 points possible: Average score 4.1
Question Text: Answer the questions above for the web site you're evaluating.
Overall, does what you know about the bias of the web site indicate that it's a good
resource? Overall, is this web site a good resource to use for your assignment?
Q4: Cumulative Count of Scores
0
2
4
6
8
10
12
Total 1 8 3 4 10 11 8 3 2
Count-
0
Count-
1
Count-
2
Count-
3
Count-
4
Count-
5
Count-
6
Count-
7
Count-
8
Q4: Count of Scores by Outcome
05
101520
25303540
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Beginning
Developing
Exemplary
Uses Criteria Terms Cites Clues Cites Examples from Sourc e Judges Whether or Not To
Use
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
34/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Pilot Test Findings
Students proven successful with
mechanical tasks like checking currency
and identifying URLs.
Students are shown as developing
with judgment tasks such as authority
and bias.
Bias is a potential target area.
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
35/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
How are we using our
assessment results? Changes to LOBO
Add and Reorganize Content
Improve Question Format Enlarge Response Space
Make Rubrics Available?
Train Course Instructors
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
36/43
Changes to LOBO
Answer these questions about the web site youre evaluating in the space below:
Who created the site? What point of view do they represent?What organizations support the site? What biases might they have?Are links included that point to other viewpoints?Are there signs of bias included in the site?Are you biased toward the site?
Overall, does what you know about the bias of the web site indicate that its a good resource?
ReplaceQuestions
with
Content
Sample StudentAnswers
Get Help with Your Answer
Fix
Questions,
Add Links,
Enlarge
Answer
Space
Add Viewlet
to Model
Applicationof Content
Changes Coming Soon!
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
37/43
Changes for Instructors
Share Rubrics,
Continue OngoingTraining,
Add Lesson Plans
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
38/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
How are we using our
assessment results? Changes to the Instruction Program
Including assessment in departmental 3-yr goals
Sharing data with subject-specialist librarians Initiating rubric assessment of advanced
instruction
Reporting results to library administration
and colleagues on other campuses
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
39/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Key Learning
Rubrics require lots of revision! Rubrics are effective in measuring
higher-level thinking skills.
Rubrics provide informationadministrators can use for reportingand instructors can use to improveteaching and learning.
Our colleagues are interested inour progress.
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
40/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Take Aways
1. Can higher-level thinking skills likeinformation literacy or critical thinking beadequately described in a rubric?
2. Are rubrics good tools for assessing studentresponses to tutorials?
3. What problems did you find in your practicewith these rubrics?
4. What part of this process/project could beapplied at your institution?
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
41/43
M. Oakleaf and S. McCann, Undergraduate Assessment Symposium 2004
Muddiest Points?
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
42/43
Contact Information
Megan Oakleaf
Librarian for Instruction &Undergraduate Research
919-513-0302megan_oakleaf@ncsu.edu
Steve McCann
NCSU Libraries Fellow
919-513-7080steve_mccann@ncsu.edu
5/21/2018 Rubric Assessment of Student Responses to an Information Literacy Tutorial
43/43
Rubric Assessment of Student
Responses to an InformationLiteracy Tutorial
Megan OakleafLibrarian for Instruction & Undergraduate
Research
Steve McCannNCSU Libraries Fellow
Recommended