CIT in information literacy, ECIL 2016, Sabina Cisek

Critical Incident Technique in Information Literacy Research

in the XXI Century

Sabina Cisek

Institute of Information and Library Science, Jagiellonian University in Kraków, Poland

European Conference on Information Literacy 2016, Prague, 10-13 October 2016

Contents

Critical Incident Technique (CIT) –what is it?

My research goal, questions and method

Findings

Conclusions

2

Critical Incident Technique (CIT) –what is it?

3

Creator of the method

John C. Flanagan

Flanagan, John C. (1954). The critical incident technique. Psychological Bulletin, vol. 51, no. 4, p. 327-358.

4

Scholarly disciplines in which CIT is used

Business,

Consumer studies,

Education,

LIS, in particular – information behavior research

Management,

Marketing,

Medicine,

Military studies,

Nursing,

Political science,

Psychology,

Sociology

5

Definition of CIT (1)

Critical Incident Technique (CIT) is

a set of procedures

used to systematically identify and analyze

human behaviors

that have critical importance and contribute to

success or failure

of individuals or organizations

in specific situations (contexts).

6

Definition of CIT (2)

CIT is used to identify effective (or ineffective) behaviours relating to a particular activity

7

What is a critical incident ? (1)

A critical incident

is a single, „separable” event (fact, factor),

that significantly influences an activity, phenomenon, task,

and has an impact on someone’s achievements, attitudes, success or failure.

Critical incidents need not to be dramatic or unusual.

8

What is a critical incident ? (2)

„Critical incidents are not ‘things’ which exist independently of an observer and are awaiting discovery like gold nuggets or desert islands but like all data, critical incidents are created. Incidents happen, but critical incidents are produced by the way we look at a situation: a critical incident is an interpretation of the significance of an event. To take something as a critical incident is a value judgment we make, and the basis of that judgment is the significance we attach to the meaning of the incident.” [Tripp, 1993, p. 29]

9

Types of critical incidents

„A critical incident is considered positive (effective) if it helps to solve a problem and negative(ineffective) if it fails to solve a problem, creates new problems or facilitates the need for further actions.” [Cordes, 2012, p. 358]

10

Data gathering techniques within CIT(1)

„The CIT encourages participants to tell their story.” [Urquhart et al., 2003]

The details and context of a single event are collected – in order to avoid generalizations (by the study participants) from similar incidents.

11

Data gathering techniques within CIT(2)

Empirical data for CIT come from:

semi-structured interviews,

observation,

questionnaires,

written accounts (essays, reports),

other sources.

12

Features of CIT

Critical Incident Technique is a qualitative approach,

particularly useful for exploratorystudies.

CIT is a flexible method, may be modified and adapted to meet the specific situation at hand.

13

My research goal, questions and method

14

My general goal

To check if, how and what for the Critical Incident Technique was used in information literacy (IL) research in years 2001-2015

15

My research questions

1. How often CIT was used in

empirical research on IL in years

2001-2015?

2. What IL aspects and problems

were studied using CIT in years

2001-15?

3. How that research was

conducted?

16

My method (1)

Systematic review (critical literature review)

Qualitative content analysis

17

My method (2)

Six databases were searched with query „critical incident”AND „information literacy” (or similar), with limiters (publication dates 2001-2015)

18

My method (3)

The recall appeared to be very limited → only few records in each database

Emerald Management Extra (5 results)

Francis (3 results)

LISTA (5 results)

SAGE Journals (12 results)

Scopus (6 results)

Web of Science – SSCI (4 results)

19

My method (4)

After elimination od duplicates etc. only 10 articles occurred to be relevant for the present study –

and these were read in depth to answer the research questions (using a priori analytical categories)

20

Analytical categories related to question 3

How the CIT-based research on IL was conducted?

How critical incidents have been defined/understood in the analyzed papers?

What data gathering techniques (empirical data sources) were employed?

What types of sampling (convenience, probability, purposive, self-selecting, snowballing, theoretical) were used, and what were the sample sizes (the number of respondents)?

How the analyses of empirical data were conducted?

What conceptual or theoretical frameworks were used?

Were the reported research enterprises purely qualitative or mixed methodology?

21

Findings

22

Research question 1

How often CIT was used in the empirical research on IL in years 2001-15?

CIT was not common in the IL research in years 2001-2015 (only 10 relevant works found)

23

Research question 2

What IL aspects and problems were studied using CIT in years 2001-15?

Critical reflection and critical thinking during library instruction programs [Gilstrap, Dupree, 2008a; 2008b]

Developing information seeking profiles of nursing students to support development of IL programmes [Stokes, Urquhart, 2015]

Health information literacy of public library patrons and librarians [Yi, You 2015; Yi, 2015]

International students using online information sources to learn (for study purposes) [Hughes, 2012; 2013]

Library impact on student learning [Rodriguez, 2012]

Relationship between critical thinking ability (in its both cognitive and affective dimensions) and library anxiety [Kwon, 2008]

Technological (ICT), information seeking and information gathering competencies of undergraduate students [Cordes, 2012]

24

Research question 3 (a)

How the CIT-based research on IL was conductedin years 2001-15?

How critical incidents have been defined/understood in the analyzed papers?

Assignments based on online information resources, recently completed by students [Hughes, 2013]

Critical incidents related to a research project in history [Rodriguez, 2012]

„Most recent or most memorable experience of using the library and its resources to write a research paper” [Kwon, 2008, p. 120]

Most rewarding and most surprising experiences in a class; Moments of understanding and moments of confusion about the instructional content [Gilstrap, Dupree, 2008a]

Negative critical incidents [Cordes, 2012]

Specific health information activities, including health information needs, situations causing health information seeking, and obstacles/difficulties encountered [Yi, 2015]

25

Research question 3 (a)

How the CIT-based research on IL was conducted in years 2001-15?

How critical incidents have been defined/understood in the analyzed papers?

An example:

„1) At what moment in the class today did you feel like you most understood the instructional content?

2) At what moment in the class today did you feel most confused about the instructional content?

3) What was your most rewarding experience in today’s class?

4) What was your most surprising experience in today’s class?” [Gilstrap, Dupree, 2008b, p. 424]

26

Research question 3 (b)

How the CIT-based research on IL was conducted in years 2001-15?

What data gathering techniques (empirical data sources) occured in the analyzed set of 10 papers?

An activity log page for a class assignment (creating a set of wiki pages) [Cordes, 2012]

Critical incident type interviews [Stokes, Urquhart, 2015]

Essays (written by students to report their critical incidents from the past) [Kwon, 2008]

Observation, observed task [Hughes, 2013]

Questionnaires – on paper [Gilstrap, Dupree, 2008a; 2008b] and web-based [Rodriguez, 012]

Semi-structured interviews [Hughes 2012; 2013; Yi, 2015]

27

Research question 3 (c)

How the CIT-based research on IL was conducted in years 2001-15?

What types of sampling were used?

Convenience sample, purposive sample (usually students) [e.g., Cordes, 2012]

Purposive sample [Yi, 2015]

Random selection [Stokes, Urquhart, 2015]

Self-selecting sample [Hughes 2012; 2013]

What were the sample sizes (the number of respondents)?

11, 20, 22, 25, 127, 137, 321, 348

28

Research question 3 (d)

How the CIT-based research on IL was conducted in years 2001-15?

How the analyses of empirical data were done?

Constant comparative method [Cordes, 2012]

Content analysis [Cordes, 2012; Kwon, 2008; Yi, 2015]

Inductive analysis [Hughes, 2013]

Qualitative interpretative categorisation QIC [Stokes, Urquhart, 2015]

Statistical analysis [e.g., Gilstrap, Dupree, 2008a; 2008b]

29

Research question 3 (e)

How the CIT-based research on IL was conducted in years 2001-15?

What conceptual or theoretical frameworks were used?

ACRL Information Literacy Competency Standards for Higher Education [Cordes, 2012; Gilstrap, Dupree, 2008a; 2008b]

Brookfield’s Critical Incident Questionnaire CIQ [Gilstrap, Dupree, 2008a; 2008b]

California Critical Thinking Disposition Inventory CCTDI [Kwon, 2008]

Expanded Critical Incident Approach ECIA [Hughes 2012; 2013]

Phenomenography [Yi, You, 2015]

Reflective learning and teaching [Gilstrap, Dupree, 2008a; 2008b]

Self-perception theory of Daryl Bem [Yi, 2015]

30

Research question 3 (f)

How the CIT-based research on IL was conducted in years 2001-15?

Were the reported research enterprises purely qualitative or mixed methodology?

Both, „pure” qualitative [e.g., Hughes, 2012; 2013] as well as mixed methodology [e.g., Kwon, 2008; Yi, 2015]

31

Conclusions

32

The advantage of CIT

Critical incident technique is based on real-life experiences, and sees the incidents (e.g., „instances” of databases searching or information seeking asignments) exactly as they are perceived by their actual partcicipants.

So, the true meaning and importance level attached to them by subjects may be discovered.

33

Analyzed works, found in the six databases

1. Cordes, S.: Student Technology Use in the Information-Seeking and Information-Gathering Process: A Critical Incident Approach for Benchmarking Performance. E-Learning and Digital Media 9 (4), 356–367 (2012)

2. Gilstrap, D.L., Dupree, J.: A Regression Model of Predictor Variables on Critical Reflection in the Classroom: Integration of the Critical Incident Questionnaire and the Framework for Reflective Thinking. Journal of Academic Librarianship 34 (6), 469–481 (2008)

3. Gilstrap, D.L., Dupree, J.: Assessing Learning, Critical Reflection, and Quality Educational Outcomes: The Critical Incident Questionnaire. College and Research Libraries 69 (5), 407–426 (2008)

4. Hughes, H.: An Expanded Critical Incident Approach for Exploring Information Use and Learning. Library and Information Research 36 (112), 72–95 (2012)

5. Hughes, H.: International Students Using Online Information Resources To Learn: Complex Experience and Learning Needs. Journal of Further and Higher Education 37 (1), 126–146 (2013)

6. Kwon, N.: A Mixed-Methods Investigation of the Relationship between Critical Thinking and Library Anxiety among Undergraduate Students in Their Information Search Process. College and Research Libraries, 69 (2), 117–131 (2008)

7. Rodriguez, D.: Answering Questions About Library Impact on Student Learning. In the Library with the Lead Pipe, 1–14 (2012), http://www.inthelibrarywiththeleadpipe.org/2012/answering-questions-about-library-impact-on-student-learning/

8. Stokes, P., Urquhart, C.: Profiling Information Behaviour of Nursing Students: Part 2: Derivation of Profiles. Journal of Documentation 71 (1), 52–79 (2015)

9. Yi, Y.J., You, S.: Understanding the Librarian/User Gap in Perception of Health Information Services: A Phenomenographic Approach. Journal of Librarianship and Information Science 47 (4), 356–367 (2015)

10.Yi, Y.J.: Health Literacy and Health Information Behavior of Florida Public Library Users: A Mixed Methods Study. Journal of Librarianship and Information Science 47 (1), 17–29 (2015)

34

Selected bibliography on the Critical Incident Technique

Butterflield, Lee D. et al. (2005). Fifty years of the critical incident technique: 1954–2004 and beyond. Qualitative Research, Vol. 5, No. 4, p. 475-497.

Fivars, Grace (2001). Critical Incident Technique Bibliography – 2001. Introduction. http://www.apa.org/pubs/databases/psycinfo/cit-intro.pdf

Fivars, Grace; Fitzpatrick, Robert (2001). Critical Incident Technique Bibliography – 2001. http://www.apa.org/pubs/databases/psycinfo/cit-full.pdf

Flanagan, John C. (1954). The critical incident technique. Psychological Bulletin, Vol. 51, No. 4, p. 327-358.

Hughes, Hilary (2007). Critical Incident Technique. In: Lipu, S., Williamson, K., Lloyd, A. eds. Exploring Methods in Information Literacy Research. Wagga Wagga, N.S.W. : Centre for Information Studies, Charles Sturt University, pp. 49-66.

Tripp, David (1993). Critical Incidents in Teaching: Developing Professional Judgment. London: Routledge.

Urquhart, Christine et al. (2003). Critical incident technique and explicitation interviewing in studies of information behavior. Library and Information Science Research, Vol. 25, Issue 1, p. 63–88.

35