Jacqui DowdIntroduction to LibQUAL+University of Westminster5th February 2010

LibQUAL+ v LibQUAL Lite at the University of Glasgow

History of LibQUAL+ at the University of Glasgow

Member of the First SCONUL Consortium in 2003,

and in 2004, 2005, 2006

And again in Spring 2008

Winter 2008 LibQUAL Lite Pilot

And Spring 2009

And 2010

Why Participate LibQUAL+ Lite Beta Testing?

Hopefully to

• Increase the response rates -Although we have always had a representative sample, the response rates have been consistently below 10%

By

• Reducing the burden on the respondents -LibQUAL+ requires 97 responses – unreasonable expectation!LibQUAL+ Lite requires only (?) 51 responses

Increase in Response Rates?

LibQUAL+ Lite sample = 6,808 Undergraduates

Survey Response Rates

5.0%

6.0%

7.0%

8.0%

9.0%

10.0%

Response Rate 9.5% 7.9% 5.2% 5.7% 6.1% 8.2% 5.6%

2003 2004 2005 2006 20082008 Beta Lite

70%2009

Increase Valid Survey Yield?

“Typically about half of the people who view the survey tend to submit a complete version of the survey.” Martha Kyrillidou, Item Sampling in Service Quality Assessment Surveys To Improve Response Rates and Reduce Respondent Burden: The “LibQUAL+ Lite” Randomized Control Trial (RCT) Dissertation, University of Illinois, 2009

Not at Glasgow!

Glasgow Beta yield: Full 40%, Lite 55%

2003 2004 2005 2006 20082008 Beta

Lite 70% 2009Surveys Viewed 1273 5475 4012 3998 5059 1113 3175Completed Surveys 502 2212 1423 1535 1789 572 1173Valid Surveys 493 2178 1384 1509 1683 560 1117Valid yeild 39% 40% 34% 38% 33% 50% 35%

Reduction in the Respondent Burden?

LibQUAL+ Lite is a survey methodology in which (a) All users answer a few, selected survey questions, but (b) the remaining survey questions are answered ONLY by a randomly-selected subsample of the users. Thus, (a) data are collected on ALL questions, but (b) each user answers FEWER QUESTIONS, thus shortening the required response time.

Martha Kyrillidou, ibidem

Reduction in the Respondent Burden?

Data from the Beta test indicated –

•The average completion time of the Lite survey was only 6 minutes 58 seconds as opposed to 10 minutes 59 seconds for the long form

• The median completion time of the Lite survey was 5 minutes 2 seconds as opposed to 8 minutes 27 seconds for the long form

Saving respondents:Saving respondents:

• 4 minutes 1 second on the average completion time4 minutes 1 second on the average completion time

•2 minutes 45 seconds on the median completion time2 minutes 45 seconds on the median completion time

Reduction in the Respondent Burden?

Glasgow Beta Test• Yes – in so much as fewer responses were required

• No – not significantly in terms of the average completion time

• Yes – significantly in terms of the median completion time

Average completion time of all Full iterations is 12 minutes 40 seconds compared to 11 minutes 54 seconds for the Beta test (minus 46 seconds)

Median Completion time of all Full iterations is 9 minutes 16 seconds compared to 5 minutes 38 seconds for the Beta Test (minus 3 minutes 38 seconds)

N.B. On average, 5% fewer respondents added comments in the Beta test.

Survey Completion Times

Glasgow Average Completion Time

04:48

06:00

07:12

08:24

09:36

10:48

12:00

13:12

14:24

2003 2004 2005 2006 2008 2008 Beta Lite70%

2009

Median Survey Time Average Survey Time

Beta Survey Completion Times

Glasgow Beta Test – 70% LiteGlasgow Beta Full v Lite 2008

04:45

05:57

07:09

08:21

09:33

10:45

11:57

13:09

UGL 2008 Beta Full 30% UGL 2008 Beta Lite 70%

Median Survey Time Average Survey Time

Full v Lite: 22 Core Scores

• Lite Perceived Service Level scores < than 2008 & 2009

• Lite Desired & Minimum Service Level scores > 2008 & 2009

22 Core Means

6

6.2

6.4

6.6

6.8

7

7.2

7.4

7.6

7.8

8

UGL 2008 UGL Lite UGL 2009

Perceived Service Level Desired Service Level Minimum Service Level

Affect of Service Scores

• All Lite scores > 2008 & 2009Affect of Service Means

6

6.2

6.4

6.6

6.8

7

7.2

7.4

7.6

7.8

8

UGL 2008 UGL Lite UGL 2009

Perceived Service Level Desired Service Level Minimum Service Level

Affect of Service Scores

Item Perceived scores

Affect of Service Perceived Service Level Scores

5.50

6.00

6.50

7.00

7.50

8.00

AS-1 AS-2 AS-3 AS-4 AS-5 AS-6 AS-7 AS-8 AS-9

UGL 2008 UGL Lite UGL 2009

Information Control scores

• Lite Perceived & Minimum scores < 2008 & 2009

• Lite Desired scores > 2008 & 2009Information Control Means

6.00

6.50

7.00

7.50

8.00

8.50

UGL 2008 UGL Lite UGL 2009

Perceived Service Level Desired Service Level Minimum Service Level

Information Control scores

Item Perceived scores

Information control Perceived Service Level Scores

5.50

6.00

6.50

7.00

7.50

8.00

IC-1 IC-2 IC-3 IC-4 IC-5 IC-6 IC-7 IC-8

UGL 2008 UGL Lite UGL 2009

Library As Place Scores

• All Lite scores < 2008 & 2009

Library as Place Means

6

6.5

7

7.5

8

8.5

UGL 2008 UGL Lite UGL 2009

Perceived Service Level Desired Service Level Minimum Service Level

Library As Place Scores

Item Perceived scores

Library As Place Perceived Service Level Scores

5.50

6.00

6.50

7.00

7.50

8.00

LP-1 LP-2 LP-3 LP-4 LP-5

UGL 2008 UGL Lite UGL 2009

General Satisfaction

Lite score < 2008 & 2009General Satisfaction Mean Scores

6.40

6.50

6.60

6.70

6.80

6.90

7.00

7.10

7.20

7.30

7.40

2008 2008 Lite 2009

Information Literacy Outcomes

Lite score > 2008 & < 2009 Information Literacy Outcomes - Mean Scores

6.40

6.45

6.50

6.55

6.60

6.65

2008 2008 Lite 2009

Affect on Benchmarking

Lite scores have a negative effect when benchmarking with other Russell Group Libraries!

Perceived Scores Glasgow & Russell Group Libraries

6.40 6.50 6.60 6.70 6.80 6.90 7.00 7.10 7.20

22 Core

AS

IC

LP

UGL 2009 UGL Lite SCL 2009 RG1 2009 RG2 2008 RG3 2009 RG4 2009 RG5 2009

The Future Lite or Full?

We want

• To increase response rates by reducing the burden on respondents

• To continue to benchmark longitudinally & with peers

However, this depend on what protocol we and our peers use in future & what LibQUAL+ offer.

The Future Lite or Full?

In the Beta testing, Glasgow was identified as one of four large research libraries participating.

“Though score conversion is not needed, there are some circumstances under which score conversion may be more useful for large research libraries that rely heavily on the LibQUAL+ protocol through annual or biennial implementations.”Martha Kyrillidou, ibidem

Lite of Full

Will SCONUL participants agree which protocol to use?

If yes and they agree Lite:

– will they also agree the degree of Lite?

– will LibQUAL+ provide score conversion for the previous year to enable continuous benchmarking?