Live Interest Meter - Learning from Quantified Feedback in Mass Lectures

FZ

I FO

RS

CH

UN

GS

ZE

NT

RU

M

INF

OR

MA

TIK

Live Interest Meter - Learning from Quantified

Feedback in Mass Lectures

V. Rivera-Pelayo, J. Munk, V. Zacharias, and S. Braun

FZI Research Center for Information Technology, Karlsruhe, Germany

LAK Conference 2013 – Leuven, Belgium

10th April 2013

Agenda

Introduction

Use Case and Motivation

Live Interest Meter App

Tests and User Study

Conclusions

Ongoing Work

12.04.2013 © FZI Forschungszentrum Informatik 2



Holding a good

presentation is hard!



How to get better at it?

• With practice

• Get feedback

• Learn from the feedback! Holding a good

presentation is hard!


LA for workplace learning

Capturing of feedback for reflective learning

Inspired by Quantified Self approaches

Mass lectures, virtual courses and conferences



LA for workplace learning

Capturing of feedback for reflective learning

Inspired by Quantified Self approaches

Mass lectures, virtual courses and conferences


Live Interest meter - LIM App

Gathering, aggregation and visualization of feedback

Protagonist: the meter


General

performance

Speech speed

Comprehension

LIM App: Evolution graph

Available for Android and for Browsers (JavaScript)

Browser to facilitate the configuration of the presenter


LIM App: Polls and Questions

Polling and anonymous rated questions


LIM App: JavaScript Version


Experimental tests


(1) Project meeting

10 participants used LIM

Discussion driven

Experimental tests


(1) Project meeting


Discussion driven

(2) Lecture at university

15 participants

Interactive session

Technical results very satisfactory

Acceptance results rather limited

Experimental tests


Project meeting


Discussion driven

Lecture at university

15 participants

Interactive session

Technical results very satisfactory

Acceptance results rather limited

Suitable for big audiences Presenter‘s role well defined

User Study


Main goals

Refine use case

Guide further development

In which scenarios and how can the quantification of feedback performed

by the LIM App support reflective learning?

Which features are more appreciated by users, both presenters and

audience members?

User Study


20 qualitative interviews

Willing to substitute traditional questionnaires for captured feedback

Anonymity appreciated (for feedback and questions)

Polls but Chat

Reflection needs time – preferred after the session

Online survey

1 month (May-June 2012)

120 participants

87 valid responses

Presenter 44,82% Audience

55,18%

User Study – Reaction to feedback


What do you consider realistic regarding

how you can react to the feedback (as presenter)?

User Study – Data collection


Data collected live and continuously is

better and more significant than only periodically collected data.

64,37 %

User Study – Information needed


Which information would the presenters

like to know about and which information

does the audience want to evaluate?

User Study – Characteristics of the LIM App


How participants evaluated the LIM App features

Conclusion

Readiness of presenters to learn retrospectively

Motivate students to use the application

Main concern is distraction

Applying Learning Analytics to informal workplace learning and

data gathered during the learning process


78,16 % found the idea behind

the LIM App very positive

57,47 % would like to test the

LIM App

Towards a second prototype

Usability improvements

Support for reflection after the presentation

From presenter to collaborative approach e.g. topics

Mock-ups to adapt the results to real features


THANK YOU!

Any questions?


[email protected]

@veronicarp

vriverapelayo

Theoretical Model & Related Work


Theoretical background


[1] Applying Quantified Self Approaches to Support Reflective Learning. Verónica Rivera-Pelayo, Valentin Zacharias, Lars Müller,

Simone Braun. Learning Analytics and Knowledge 2012 (LAK 2012), Vancouver, Canada

[2] A Framework for Applying Quantified Self Approaches to Support Reflective Learning. Verónica Rivera-Pelayo, Valentin

Zacharias, Lars Müller, Simone Braun. IADIS International Conference on Mobile Learning (Mlearning 2012), Berlin, Germany

Related Work

Clickers and feedback tools

Polling

Student outcomes and comprehension

Student attendance and interest on the course

Reflective Learning in TEL

Student’s perspective

Data from LMS, blogs or software used

LIM App

Improve lecturer/presenter’s professional performance

Students give feedback and contribute to it

Enrich lecture

Feedback contextualized by polls, questions and chat

Gathering of data during the lecture

4/12/2013 © FZI Forschungszentrum Informatik 25

ARS and Clickers

[1] M. Akbari, G. Böhm, and U. Schroeder. Enabling communication and feedback in mass

lectures. In ICALT, pages 254{258, 2010.

[2] M. Bonn, S. Dieter, and H. Schmeck. Kooperationstools der Notebook Universität Karlsruhe

(TH). In Mobiles Lernen und Forschen, pages 63-71. Klaus David, Lutz Wegener (Hrsg.),

November 2003.

[3] J. E. Caldwell. Clickers in the large classroom: Current research and Best-Practice tips. CBE

Life SciEduc, 6(1):9-20, Mar. 2007.

[4] D. Duncan and E. Mazur. Clickers in the Classroom: How to Enhance Science Teaching Using

Classroom Response Systems. Pearson Education, 2005.

[5] J. Hadersberger, A. Pohl, and F. Bry. Discerning actuality in backstage - comprehensible

contextual aging. In EC-TEL, volume 7563 of Lecture Notes in Computer Science, pages 126-

139. Springer, 2012.

[6] D. Kundisch, P. Herrmann, M. Whittaker, M. Beutner, G. Fels, J. Magenheim, W. Reinhardt, M.

Sievers, and A. Zoyke. Designing a Web-Based Application to Support Peer Instruction for Very

Large Groups. In ICIS '12, Research in Progress, Orlando, USA, December 2012.

[7] G. Rubner. mbclick - an electronic voting system that returns individual feedback. In WMUTE,

pages 221-222. IEEE, 2012.

[8] A. Wessels, S. Fries, H. Horz, N. Scheele, and W. Effelsberg. Interactive lectures: Effective

teaching and learning in lectures using wireless networks. Comput. Hum. Behav., 23(5):2524-

2537, Sept. 2007.


Reflective learning in TEL

[1] R. Ferguson, S. B. Shum, and R. D. Crick. EnquiryBlogger: using widgets to support

awareness and reflection in a PLE Setting. In ARPLE11, PLE Conference 2011, Southampton,

UK, 11-13 July, 2011.

[2] S. Govaerts, K. Verbert, E. Duval, and A. Pardo. The student activity meter for awareness and

self-reflection. In CHI '12 Extended Abstracts on Human Factors in Computing Systems, pages

869-884, New York, NY, USA, 2012. ACM.

[3] J. L. Santos, S. Govaerts, K. Verbert, and E. Duval. Goal-oriented visualizations of activity

tracking: a case study with engineering students. In 2nd International Conference on Learning

Analytics and Knowledge, LAK '12, pages 143-152, New York, NY, USA, 2012. ACM.
