Adaptive Support for Educational Question Answering @ EC-TEL 2013 DC

Adaptive Support

for Educational Question Answering

17th September 2013

EC-TEL 2013 DC

Ivan Srba

Supervisor: Mária Bieliková

Faculty of Informatics and Information Technologies

Slovak University of Technology in Bratislava

2

• How can Internet users satisfy their information needs?

– Web search engines (e.g. Google, Bing)

– Community-based knowledge sharing applications (e.g. forums,

social networking sites, community question answering)

Community Question Answering (CQA)

3

• How can Internet users satisfy their information needs?

– Web search engines (e.g. Google, Bing)

– Community-based knowledge sharing applications (e.g. forums,

social networking sites, community question answering)

• Community Question Answering (CQA) is a web service

where people can:

– seek information by asking a question and

– share knowledge by providing answer on the particular question

Community Question Answering (CQA)

4

1. Question creation

2. Question routing

3. Question answering

4. Question search

Question Answering Process in a Nutshell

5

Knowledge sharing Collaborative learning

Different Perspectives on CQA

Knowledge Management

Knowledge Embedded in Communities

Communities of Practice

Online Communities of Practice

Technology-Enhanced Learning

Computer-Supported Collaborative Learning

Peer Learning

6

• Analyses of more than

70 research studies

– Adaptive approaches

which concern with open

problems in CQA

– Categorization of

approaches

State of the Art: CQA Approaches

CQA Approaches

Domain Entities Analysis

Question Routing

Question Search

Question Answering

Question Creation

Question and User Clasification

Question Retrieval

User Expertise Estimation

Topic Hierarchy Maintenance

Automatic Topic Assignment

Answer Ranking

Answer Summarization

Voting Calibration

User Activity Estimation

Content Quality Estimation

Asker Satisfaction Estimation

Bridging the Gap between Web Search and CQA

User Motivation Estimation

7

• Question answering in intra-organizational context

– Educational, business or research organizations

– We focus on educational organizations (universities)

• Question routing

– Recommendation of the most appropriate answerers

Research Scope of the Dissertation Project

CQA Approaches

Domain Entities Analysis

Question Routing

Question Search

Question Answering

Question CreationUser Expertise Estimation

User Activity Estimation

User Motivation Estimation

8

• Q1: Identification of Specific Organizational Conditions

– Organizational and educational environment is different in

comparison with standard CQA

• The number of users is significantly lower, overloading users

• The great amount of accessible information about users

• The presence of a supervisor

• Students’ knowledge changes significantly in time

• Shift of partial responsibility for learning from a teacher to

students themselves

What are the specific conditions in intra-organizational educational

environments and how these specifics influence collaboration

during question answering process?

Organizational Question Answering

9

Organizational Question Routing

• Q2: Proposal of Organizational Question Routing Methods

– Existing question routing methods focus primarily on knowledge

sharing perspective

• Achieve the best answer in the shortest possible time

• Involve mainly the experts in providing answers

– In organizational context, collaborative learning perspective is

important as well as knowledge sharing perspective

• Achieve the best answer as the result of learning process

• Harness a potential of the whole community

How can we propose new approaches for question routing to match the

specifics of intra-organizational educational environments?

10

• Q3: Designing Organizational CQA Systems

– Educational environment requires additional learning support

• Dashboards with visualizations of students’ profiles

• Multimedia integration into questions and answers (e.g. UML

diagrams)

Which additional/modified functionalities should be provided by intra-

organizational educational CQA system?

Organizational CQA System

11


– Study on influence of students’ personal characteristics on

collaborative learning

• Dataset obtained during a long-term experiment (12 weeks)

• 129 students, 254 dynamic groups, 3,763 collaborative activities

• Students solved short-term questions prepared by a teacher

• Two connected learning systems ALEF and PopCorm

– Implications for CQA:

• Better students can become a driver in collaboration mediated by

students themselves

• Students are able to argument and reach consensus

• …

Preliminary Results

12

Adaptive Learning Framework (ALEF)

13

Popular Collaborative Platform (PopCorm)

14


– Study on influence of students’ personal characteristics on

collaborative learning

• Dataset obtained during a long-term experiment (12 weeks)

• 129 students, 254 dynamic groups, 3,763 collaborative activities

• Students solved short-term questions prepared by a teacher

• Two connected learning systems ALEF and PopCorm

– Implications for CQA:

• Better students can become a driver in collaboration mediated by

students themselves

• Students are able to argument and reach consensus

• …

Preliminary Results

15

• Q2: Proposal of Organizational Question Routing Methods

– User expertise

• Competition-based expertise network: pairwise comparison of users

• An answerer who provided the best answer has higher expertise in

comparison with all other non-best answerers

– User activity

• Overall users’ activity, availability and authority

• Implications from the study on influence of users’ characteristics

– User motivation

• Knowledge bartering: considering the symmetry in knowledge each

student provides and receives

Preliminary Results

16

• The first preliminary phase

– State of the art

– Initial identification of organizational specifics

• The current second phase

– Designing and implementation of educational CQA system

– Long-term experiment with limited number of students

– Verification of specifics (Q1) and enhanced CQA system (Q2)

• The next third phase

– Proposal of the new method to question routing

– Synthetic tests followed by long-term experiment (Q3)

– Conclude results and derive implications

Research Methodology

17

• Case study of employing CQA in educational environment

– Valuable input for further research

• Proposal of the new approach to question routing

– Considers the organizational specifics

– Evaluation in long-term experiments at our faculty

• Open problems

– What is the ideal status in educational question answering?

• Fast answers, symmetry in activities, …

– Which specifics should be considered by the proposed method?

• Organizational and educational specifics

– How should they influence question routing?

• User expertise, activity and motivation

Contributions and Conclusion