Dagmar Kern, Peter Mutschke, Philipp MayrGESIS – Leibniz Institute for the Social Sciences, Cologne, Germany

Contact: dagmar.kern@gesis.org

Mission:� Supporting a user centered design process for developing

Interactive Information Retrieval (IIR) systems

Main problem of user studies:� Enormous recurring effort to recruit suitable participants for

user studies

Approach:� IIRpanel - an online access panel for IIR user studies

� A pool of participants for web surveys and lab studies

Benefits:� Interested participants sign up only once with the panel

� Researchers send participation calls to all or to a sample of panelists

� Stable population of users - repeated examinations easily possible

Open questions:� How to address representativeness?

� How to balance participants’ workload?

� How to control participants’ learning effects?

� How to make study data available for other researchers?

Strategic goal:� Establishing an international consortium of IIR researchers

interested in a joined management of the panel

Do you know this situation? Prototype and user study are ready to go …

… but you don‘t know how to locate a sufficient number of suitable participants to test your system?

Yeah, I have a

great idea for a

new IIR system.

Let‘s start

implementing a first

prototype.Hi, how is it going

with your project?

Great! Now it‘s

time for the first

user study.

That shouldn’t

be a problem.Arrrg, we need

participants!!!!

Establishing an Online Access Panel for Interactive

Information Retrieval Research2 weeks later

Urrg …

Rejection

no 37

6 weeks laterGood luck finding

participants!

(6) select

and contact

panelists

(7) provide

study meta

data (e.g.

response rate)

(5) notification

of acceptance

Panelist DB IIRpanelwww.gesis.org/iirpanel

Panelists Consortium

IIR Researcher

Oh, you don’t have

time, ok. Sigh …