A framework for an

online InfoLit course for

PhD students

DARTS5 June 2016

Nazlin Bhimani,

Research Support & Special Collections Librarian

@NazlinBhimani

UCL Institute of Education Library CC-BY-NC-ND

The Centre for Doctoral

Education at the IOE

The UCL Institute of Education is a world-leading

centre for research in education and related social

sciences with funding of over £20 million each year.

For the third year running the IOE has been ranked

the leading university for education (2016 QS World

University Rankings) and it also topped the league

tables for education research in the last REF.

The Centre for Doctoral Education is the largest

doctoral cohort in these areas – approx.900

research students enrol each year.

The IOE’s PG programmes have been designed to

provide comprehensive and broadly based research

training to meet the requirements of the ESRC and

the AHRC.

The Students

Students registered with the Centre for Doctoral Education undertake

research degrees leading to an MPhil or PhD, MRes (Master of

Research) or DipSSR (Dipoma in Social Science Research), EdD

(Doctor of Education) or DEdPsy (Doctor of Education Psychology).

Over 44% of the students are international and come from over 100

countries. Many have English as a second language.

Many of them study part-time and are distant learners i.e. studying from

abroad, and most of them are remote users.

Many of them are mature students, usually professionals, working in

education at leadership level in their organisations – they could be heads

of schools, university managers, policy advisors in education institutions,

government or non-profit organisations and government officials.

The F2F Course

The ‘Information and Literature Searching’ course was a

compulsory module taken by all MPhil/PhD students and

MRES students.

Since the merger with UCL it is now a ‘recommended core’

module though it is still compulsory for the students on the

Online PhD programme.

It is a 8-hour intensive course (offered 2 hrs/week) six times a

year; it will be an intensive 12-hour module from October 2015

onwards.

The course has been hugely popular with many students

voting it the most useful course taken during their PhD.

It has twice received the highest rating (excellent) for any

course offered by the CDE.

What is eLearning?

“eLearning …can be defined as

'learning facilitated and supported

through the use of information and

communications technology'. It can

cover a spectrum of activities from the

use of technology to support learning as

part of a ‘blended’ approach (a

combination of traditional and e-learning

approaches), to learning that is

delivered entirely online. Whatever the

technology, however, learning is the

vital element.

JISC (2012)

Image source:

https://ec.europa.eu/epale/en/blog/have-

you-joined-e-learning-revolution-1

Changing role of the librarianto information curators and educators

Librarians “must assert

themselves as key players in

the learning process thereby

changing their roles from

information providers to

educators” (Cooper &

Dempsey, 1998)

They have become

providers of technical

information (Hulshof,

1999)

“The librarian can shift the focus

from explaining library resources

to meeting the ongoing

information needs of the

students in the broad

information environment”

(Lippincott, 2002)

They have been transformed

from “information gatekeepers

to “information gateways”

(Haricombe, 1998).

Anderson, T. Editor (2008).

The theory and practice of

online learning (2nd ed).

Edmonton: AU Press.

Online Courses

Model depends on the following criteria:

How ‘exclusive’/tailored the content is for a

specific group of students

Size of the student body

Diversity of the student body

How blended the course is – i.e. Is there an offline

element?

Whether the course is compulsory/credit-bearing

Uniqueness

Each online course is unique like a

snowflake, shaped by:

Individuality of the students

Preferred learning styles

Prior knowledge and backgrounds

Motivations

Language competency

IT competency

The instructor’s pedagogical

stance

The hosting institution’s goals and

agendas

Technological affordances and

limitations

Time

What needs to be acknowledged at the outset is that there

will be an increase in workload for all concerned. It is

generally acknowledged that online courses are labour-

intensive to create. The issue was recently highlighted in

discussions on Massive Open Online Courses (MOOCs):

Stanford University’s Robert Sedgewick spent hundreds of

hours creating content, giving as much as two weeks to

each recording for his online course – the preparation itself

is a ‘full time job’, he stated in an interview for the

Chronicle of Higher Education (Kolowich, 2013, p. A21).

The Online PhD

The Online PhD began in October 2014

We weren’t expecting more than a handful of student in the first year and we

had 8 register in the first year and about the same number in the second

and third year of the programme.

Thus far, the online InfoLit course which is compulsory for these students,

has run three times (once every year) and has been taken by 20 students

1. Cohort A (IOE): October 2014 (5 weeks) 5 students (2 dropped out)

2. Cohort B (IOE): May-June 2015 (7 weeks) 8 students

3. Cohort C (UCL IOE): April-May 2016 (8 weeks) 7 students

Where?

English-Speaking:

3 students from UK

1 from US

1 from Australia

1 from Ireland

Rest of the World

2 from China

1 from Israel

1 from UAE

2 from Japan

1 from Singapore

From Europe:

2 from Greece

1 from Malta

1 from Brussels

1 from Spain

1 from Netherlands

1 from Denmark

1 from Switzerland

At the beginning…

1. Moodle (VLE)

1. Access

2. Content

Research Question

Search, Find and Access

resources

Evaluate resources

Find more resources

Fine-tune Research Question

Beetham, H. McGill, L and Littlejohn, A. H.

(2009) Thriving in the 21st Century:

Learning Literacies for the Digital

British Library and JISC. (2008). Information

Behaviour of the Researcher of the Future.

Coonan, E. (2011). A New Curriculum for

Information.

Gourlay, L. et al. (2013). Digital Literacies as

a Postgraduate Attribute\.

Wong, W. et al. (2010) User Behaviour in

Resource Discovery (UBiRD): Final Report.

Researching

Content: Online User Behaviour Studies

Key findings:

1. These students are heavily reliant on secondary sources;

2. They find access to relevant resources a major constraint;

3. They are confused about open access and copyright which stops

them from networking and collaborating;

4. They do not use the full potential of innovative technology.

5. These students are insufficiently trained to be able to fully embrace

the latest opportunities in the digital information environment.

Researchers of Tomorrow2012

Mapping the course to the

‘Researcher of Tomorrow’

Week 0: Induction

(RoT 3 open access and copyright, 4 insufficiently trained to make full use

of innovative technology)

Week 1 Session 1: The Literature Review and Research Process

(RoT:2 access is a major constraint, 3 open access and copyright, 5,

insufficiently trained or informed on latest opportunities in digital

environment )

Week 2 Session 2: Historical Inquiry and Searching

(RoT: 1 primary sources, 2 access, 3 open access and copyright)

Week 3 Session 3: Impact and Information Evaluation

(RoT 3 open access and copyright, 5, insufficiently trained or informed on

latest opportunities)

Week 4 Session 4: New Technologies and Social Media

(RoT 3 open access and copyright 5, insufficiently trained or informed on

latest opportunities)

Readings

Tasks

Researching

• Researching

• Searching

• Finding

• Accessing

Evaluating

• Bibliometrics and Citation Searching

• Altmetrics

• Researching

Managing Information

• EndNote, Mendeley, Zotero

• Keeping Current with Research: RSS

• Evernote

• Copyright & IPR

Using New Technologies

• Scholarly Communications

• Social Media: Twitter for Researchers

• Digital Researcher to network & share content

• Research Data: Sources & Software

Research Question

Search, Find & Access

Evaluate Literature

Synthesise

Use Ethically & Manage

Information

Disseminate/ Share

IOE LibGuides

Searching, Finding & Accessing

Evaluating information

Using ethically

Communicating/sharing in an ethical manner

Info

rma

tio

n L

ite

rac

yD

igit

al L

ite

rac

y

LibAnswershttp://libanswers.ioe.ac.uk

YouTubePlaylist

http://bit.ly/YouTube_OnlineInfo

Lit

Newsam NewsLibrary’s blog

Take off…

The Online Course

Information and Literature Searching

InductionLevel of engagement

InductionCOHORT A (5)No. of Posts

COHORT B (8)No. of Posts

COHORT C (7)No. of Posts

Task 1 Introductions 12 46 29

Task 2 LibGuides & LibAnswers 13 17 0

The Induction will indicate the level of engagement that is likely to take

place among the cohort during the course.

Cohort B was the most active – and it usually takes one individual to set

the one – and this trend continued throughout the course. Latecomers

(only 2) are asked to introduce themselves first before joining the

discussion forum.

Gaining familiarity with the LibGuides and LibAnswers sends the message

that online support is available via the guides and the online enquiry

service.

Cohort C put their LibGuides and LibAnswers feedback together with their

introductions.

Literature Review & the

Research Process

Library of Resources: The Literature Review and Research Process

Includes videos, readings, blog posts, IOE LibGuides.

Readings:

a. Boote, David N. and Beile, P. (2005) ‘Scholars Before Researchers: On the

Centrality of the Dissertation Literature Review in Research Preparation’.

Educational Researcher 34 (6) : 3–15.

b. Randolph, J. J. (2009). A Guide to Writing the Dissertation Literature Review

Practical Assessment, Research & Evaluation 14 (13): 1-13.

Tasks:

1. Find a thesis and use the rubric discussed in the readings to evaluate the

literature review; post your work on the Discussion Forum

2. Reading the blog post by Pat Thompson on ‘Mapping your literatures’ and

create a concept map (freehand or using freely available software).

1.1. EngagementLiterature review rubric

1.1 EngagementLiterature review rubric

1.2 EngagementConcept maps and documenting search terms

Session 1Total number of posts on the Discussion Forum

Session 1 COHORT A (5) COHORT B (8) COHORT C (7)

Task 1 Lit. Review & Research Process 8 36 10

Task 2 Concept Maps 17 19 9

Summary 1 4 2

Historical Inquiry

& Searching

Case Studies:

Freathy, R., & Parker, S. (2010). The necessity of historical inquiry in

educational research: The case of religious education. British Journal of

Religious Education, 32(3), 229-243.

Kuper, A., Whitehead, C., & Hodges, B. D. (2013). Looking back to move

forward: Using history, discourse and text in medical education research:

AMEE guide no. 73. Medical teacher, 35(1), e849-e860.

Williams, R. (1985). Keywords: a vocabulary of culture and society (rev. ed).

New York: Oxford University Press.

Tasks:

1. How does the historical inquiry relate to your research question and what

primary sources can you identify for your research?

2. Using the resources (videos demos on searching catalogues & databases),

find resources that will help you begin a historical inquiry and discuss these

on the Discussion Forum.

Serendipity

“The serendipity of browsing has yet to be successfully

recreated in electronic form. An online encyclopedia can show

you links to related articles, but what about all the unrelated

ones? The printed codex allows its user to gain an

impressionistic overview of the whole, and to skim through at

high speed until something intriguing catches the eye:

something that no online resource can replicate… For in a world

where we can search for anything, it is getting harder and

harder to happen across what we never knew we wanted to

know”

Lynch, J. (2016). You Could Look It Up: the reference shelf from ancient Babylon to Wikipedia.

London: Bloomsbury pbublishing.

“Serendipity” has both a classical origin in

literature and a more modern manifestation

where it is found in the descriptions of the

problem solving and knowledge acquisition

of humanities and science scholars….Some

[studies] have implied that it may be

stimulated, or that certain people may

“encounter” serendipitous information more

than others. All to some extent accept the

classical definition of serendipity as a

“fortuitous” accident.

Foster, A., & Ford, N. (2003). Serendipity and

information seeking: an empirical study.

Journal of Documentation, 59(3), 321–340.

Serendipity

EngagementSerendipity in Searching

“The article on serendipity was interesting for me personally as I have great

faith in this as a creative force in my research. I agree with the authors who

suggest that "openness, prepared mind, and ability to make connections"

(McCay-Peet & Toms, 2015, p.1471) are all key factors in this mysterious

happenstance and that serendipity can be encouraged by ensuring your

environment is trigger rich,highlights these triggers and enables us to make

connections and follow through on them. The database key word searches

are useful here as we do tend to bump into things we didn't expect, but I must

say I do miss the verbal exchanges, debates and conversations i had during

my masters study (which was F2F) - I found these far richer environments for

bumping into unexpected ideas and following trains of thought I hadn't

considered before. Perhaps I/We should post more on this discussion

board!!!”

McCay-Peet, L., & Toms, E. G. (2015). Investigating serendipity: How it

unfolds and what may influence it. Journal of the Association for Information

Science and Technology, 66(7), 1463–1476.

2.1. EngagementHistorical Inquiry

“I am inclined to agree with the authors on the relevance and

significance of historical inquiry. My research is on a framework

for the development of professional identity and efficacy of the

21st century music teacher in Singapore. While historical

inquiry would not be the dominant research methodology for

such a study, I reckon that the historical inquiry could enrich

and inform my literature review. For example, I could look

into the National Archives in Singapore since the context of my

research is Singapore. A quick search online brought me to

potentially useful primary sources such as oral history

interviews with different music teachers related to music

education, which I might be able to investigate how these

personalities grew their professional identity. Some of the

transcripts were also available online…”

2.2. EngagementSearching

“In my opinion, the up side of using databases is 'systematicity'.

You can systematically type in keywords with AND/OR... one at

a time and you get different results. Once you are done, you can

perhaps try another database. This is much more 'systematic'

than googling with google scholar; however, this is also the down

side I think. With different databases yielding different results,

the whole process may seem intimidating. My trials showed that

different combinations of keywords with different words used in

Boolean search gave different results even using the same

database.”

“What really is a surprise to me in this age of the Internet and of

networked systems is the quite significant differences in

searches using different databases. I was aware that …many

databases draw their data from significantly different sources.”

Bibliometrics and

Information Evaluation

Library of Resources: Videos on Citation Searching on the WoS, SCOPUS,

Google Scholar, Altmetrics etc.

Reading:

a. Falagas, M. E. et al. (2007). Comparison of PubMed, Scopus, Web of

Science, and Google Scholar: strengths and weaknesses. The FASEB

Journal, 22(2), 338–342. http://doi.org/10.1096/fj.07-9492LSF

b. Moed, H. F. et al. (2016). A new methodology for comparing Google Scholar

and Scopus. Journal of Informetrics, 10(2), 533–551.

http://doi.org/10.1016/j.joi.2016.04.017

Tasks:

1. Find a highly-cited article that is relevant to your research and explain why

you chose it; next compare the citations to those on Google Scholar. Why do

you think the counts are different. Discuss this on the Discussion Forum.

2. Now evaluate the article using the REVIEW, CRAAP or PROMPT method.

Explain your evaluation process on the Discussion Forum.

3.1 EngagementBibliometrics and Citation Searching

3.2. EngagementInformation Evaluation

New Technologies

& Social Media

Google Scholar

Researcher IDs

LinkedIn

Academia.edu

Research Gate

Twitter & Facebook

Mendeley & Zotero

Blogs

Open Access, IPR and Copyright

Altmetrics

New technologies:

RSS

Text and Data Mining

Software

Cloud Storage Solutions

Research Data Management

4.1. Engagement Keeping Current with RSS

Cohort A:

Apprehensive of RSS; only one person

managed to do this and the rest opted for

email alerts

Cohort B:

Mixed reactions – some had difficulties

with the concept and two were stumped

by the technology.

Cohort C:

Expert RSS users; using feedly, talking

about Yahoo pipes to create Twitterbots

etc.

4.2. EngagementSocial media

I use Facebook, Whatsapp... but Twitter isn't really that popular in Asia

and... it's my first attempt just now! My first tweet went to Nazlin,

thanking her for her unreserved effort and this useful course. I definitely

need more time to get the hang of using Twitter but after just some

random searches (with #phd #research), I already found some useful

and interesting news/info…

Hi A, I was recently invited to join a shared Zotero list (with a student

who is about to publish his dissertation). It allowed me to benefit from

the sources that he has found/ used which was a bit like Christmas

coming early! With both you and I looking at aspects of leadership

would you be interested in creating a shared list and we can both move

across relevant or interesting pieces from our own Zotero lists?

Landing…

Time

COHORT COHORT A

(IOE)

COHORT B

(IOE)

COHORT C

(UCL IOE)

Students: 5 (2 dropped out) 8 7

Duration: 6/10/14 to 9/11/14

5 weeks

20/4/15 to 7/6/15

7 weeks

11/4/16 to 29/5/16

8 weeks

No. of Sessions 1 week induction + 4

sessions

1 week induction +

4 sessions

1 week induction +

4 sessions

Key Tasks: 12 12 12

Feedback: 5 6 3/7 (ongoing)

Total Posts: 147 451 136

NB’s Posts: 79 or 54% 182 or 40% 59 or 43%

Hrs/Student 4 hrs/student 8.5 hrs/student 3 hrs/student

Feedbackfrom sample of 15

Overall

87% rated the course as ‘Very Good’ – the highest rating

13% rated the course ‘Good’

Level of Difficulty

87% rated it ‘Just Right’

6% rated it ‘Too Difficult’

7% rated it ‘Too Easy’

Recommend it?

100% would recommend it to other students

Feedback #OLInfolit on Twitter

I found this course very, very useful...including in some surprising ways.

On the social media front, I have made a purposeful effort not to be personally engaged in

any way with LinkedIn, Facebook or twitter. I have an array of very specific reasons for

this, but in general it's a time, focus, energy and effectiveness decision on my part not to

use these social media platforms. Twitter's dilemma is that is can be mostly noise, create

lots of "present shock" in pushing the nervous system to be 'endlessly reactive,' is open to

everyone, and can be incredibly intrusive [if you let it be, and, on the other hand, if you try

to ignore it, even briefly, it can become, as some of my colleagues have learned, a prime

source of hate from people who feel you are not instantly responsive]. LinkedIn can be

awesome when looking for jobs or to expand business contacts. As I know from key

colleagues, LinkedIn also showers them with job recommendation inquiries and

networking requests day after day. This can be fine if that's what you want to spend time

and focus on. I use e-mail, the web and Evernote with my team and colleagues. I am

also looking into Mendeley and the Bibliographic Referencing Software suggestions.

WORKLOAD (Cohort B)

Make future students aware

of the level of workload

with regards to expected

hours spent per week on

the completion of each task

in order to complete the

course successfully.

COURSE LENGTH

(Cohort A)

Run the module over 6

weeks. The module is

both interesting and

informative and I would

have appreciated more

time. Material given

needs time to be

digested and I am still

coming back to articles,

videos and explanations.

COURSE LENGTH

(Cohort B

It might be useful to space

the assignments and have

a longer course duration

so that we can have more

time to explore the

resources and benefit more

fully from the course.

(Cohort B)

For the organisers

not to put 3

courses on at the

same time.

Setting up alerts was

time-consuming

MOST

INTERESTING

(Cohort A)

All! Everything is

relevant, useful &

was interesting!

(Cohort B)

All (as directly

relevant to our work)

Exactly what we

needed!

Bibliometrics and

citation searching

(Cohort C)

Mind mapping exercise was new to me and very useful.

Evaluating the quality of information found.

I found interesting to gain new skills in everything from literature searching to information evaluation.

LEAST INTERESTING

(Cohort A)

Nothing

(Cohort B)

RSS & Referencing Software

(Cohort C)

Social Media

(Cohort C)

I found the literature evaluation

section least interesting

because, for me, this is

something that happens on a

more organic basis. Sometimes

REVIEW/CRAAP/PROMPT

does not always capture why a

piece of research may or may

not be relevant.

MOST

DIFFICULT

(Cohort A)

Citation Searching

.

(Cohort B)

Course was not difficult but

rigorous

Everything was at the

same level of difficulty.

I am technically challenged

but this has been worth it! .

HOW CAN WE IMPROVE

THE COURSE?

(Cohort A)

Too pressurised; not

enough time to interact

with others

Make the course longer!

Needs to run over six

weeks.

(Cohort B)

No - it was a great course.

It is already too good and I am

sorry I cannot come up with any

suggestion.

More clarity on the pairing activity

Alerts could be broken down and

made clearer

Except for alerts/altmetrics and the

pairing activity, almost everything

was good practice and explained

clearly and adequately and

responses were positive, helpful

and sensitive

HOW CAN WE IMPROVE THE COURSE?

(Cohort C)

Difficult to pitch this in terms of the level. I think we have

quite some participants who have a lot of experience in

research so this may not all have been "new" for them.

But other students may come straight from a masters or

even bachelors and so may need more thorough

grounding.No, the planning was well executed and clear.

The high level of involvement of Nazlin and her

demandingness was very encouraging and motivated for

me.

No, the planning was well executed and clear.

OTHER

(Cohort A)

A big thank you to the person

behind the conception of the

module and to Nazlin who is

perfect at mediating, answering

questions, giving comments.

Bravo!

Loved the module.

It was a very useful and

challenging online programme

that will definitely help me in my

research journey.

(Cohort C)

I did sometimes have

difficulty finding time for

the class on top of my full-

time job. While many of

the exercises were quite

useful and worth the time,

I felt that few of the videos

contributed to my

knowledge.

My compliments to the

tutor: amazing

expertise and great

communication skills

(Cohort B)

Though I had to struggle

through the course, I am

much better equipped with

new current skills. I have a

long way to go but I am

really thankful.

TUTOR

88% said the tutor was ‘Always Clear’; 12%said she was ‘Frequently

Clear’.

(Cohort A) Impressive

(Cohort B) Excellent

I was always impressed with the tutor's "encyclopaedic knowledge" as

one of us aptly described.

I was greatly encouraged by Nazlin's support and kind words. Clear

and adequate interaction on the course. Nazlin was always very quick

to respond to our questions and was responsive to questions and

comments.

(Cohort C) I was very impressed with the time & attention from Nazlin!

Nazlin's individual comments to our comments were very helpful and

helped create a sense of connection over the internet. She also

encouraged us to respond to one another, which I think would have

also moved us in that direction though it did not always come to

fruition.

Reflection

Feedback has generally been positive and the concerns raised by students

have been actioned:

The duration of the course has increased from 5 to 8 weeks

The course is scheduled at a time when there are no other courses running

I also wait for the conversations to begin rather than jumping in!

The success of the online course has influenced my F2F teaching:

I now include the same readings and tasks in my F2F teaching

I am a better listener and am more aware of the importance of picking up

on issues that may not be clear to the students

I attempt to get the students in my F2F courses to engage more than I had

previously done.

However, the mixed abilities issue is a difficult issue to address in both the

F2F and Online course. I aim for the higher level and ensure those with less

experience catch up with individual support.

Can it be scaled up?

CC-BY-NC-ND

Yenn Lee

Conclusion

• Online course

designs need

thinking through

• An online course is

not a panacea for

lack of resources

• Simple scaling up

does not necessarily

mean readiness for

global consumptionOne size does not fit all!

Q & A