Towards Visual Analytics to Improve Scientific Reading & Writing

TOWARDS

VISUAL ANALYTICS

TO IMPROVE

SCIENTIFIC

READING & WRITING

Duygu Simsek

PhD Research Student

Knowledge Media Institute

The Open University, UK

The Open Science Laboratory

Learning Analytics Summer Institute (LASI), UK, Informatics Forum, Edinburgh July 5, 2013

Supervisors: Prof. Simon Buckingham Shum, Dr. Rebecca Ferguson, Dr. Anna De Liddo

Xerox Research Centre Europe: Ágnes Sándor

people.kmi.open.ac.uk/simsek

[email protected]

simsekduygu_

“Piled Higher and Deeper” by Jorge Cham www.phdcomics.com


Imagine if a machine that could detect…

Human analyst

-Highlight key contributions of the project outcomes

http://bit.ly/HumanVsXIP


Human analyst

http://bit.ly/HumanVsXIP

XIP


Imagine if a machine that could detect…

Reading a Paper Writing a Paper

CRITICAL MIND


Engaging with the literature

Being able to critique and

contrast

Identifying gaps in

our knowledge

Identifying when

significant claims made

Reading a Paper Writing a Paper


Engaging with the literature

oWhat’s the research problem about X?

oWhat are the open questions about X?

oWhat has been found about X?

oWhat are the research themes for X?

oAre there any major advances concerning X?


Scholarly writing has particular hallmarks!

oMeta-discourse Signals

o The problem, the claim, the

argument, the evidence and the

implications

o Explicitly organises the

discourse, expresses a

viewpoint, engages the readers,

and signals the writer's stance


Meta-discourse

oAutomatic processing of scientific documents

oRecognition of the rhetorically significant sentences

o8 categories of Rhetorical Moves

o Background Knowledge

o Summarising

o Tendency

o Novelty

o Significance

o Surprise

o Open Question

o Contrasting Ideas


Xerox Incremental Parser (XIP)


Xerox Incremental Parser (XIP)

Xerox Incremental Parser (XIP) BACKGROUND KNOWLEDGE:

Recent studies indicate …

the previously proposed …

… is universally accepted

http://bit.ly/DCLA13






SUMMARISING:

The goal of this study ...

Here, we show ...

Our results ... indicate







TENDENCY:

... emerging as a promising

approach

Our understanding ... has

grown exponentially ...

Growing recognition of the

importance ...

SUMMARISING:


Here, we show ...








NOVELTY:

New insights provide direct evidence… …suggest a new approach… Results define a novel role ...

TENDENCY:


approach




importance ...

SUMMARISING:


Here, we show ...








NOVELTY:


TENDENCY:


approach




importance ...

SIGNIFICANCE:

studies ... have provided important

advances

... is crucial for ... understanding

valuable information ... from

SUMMARISING:


Here, we show ...








NOVELTY:


TENDENCY:


approach




importance ...

SIGNIFICANCE:


advances



SURPRISE:

We have recently observed ...

surprisingly

We have identified ... unusual

The recent discovery ... suggests

intriguing roles

SUMMARISING:


Here, we show ...








NOVELTY:


OPEN QUESTION:

Little is known …

… role … has been elusive

Current data is insufficient…

TENDENCY:


approach




importance ...

SIGNIFICANCE:


advances



SURPRISE:


surprisingly



intriguing roles

SUMMARISING:


Here, we show ...








NOVELTY:


OPEN QUESTION:

Little is known …

… role … has been elusive

Current data is insufficient…

TENDENCY:


approach




importance ...

CONTRASTING IDEAS:

In contrast with previous

hypotheses ...

... inconsistent with past

findings ...

SIGNIFICANCE:


advances



SURPRISE:


surprisingly



intriguing roles

SUMMARISING:


Here, we show ...




o Fine for researchers or

machines but it is not

learner/educator friendly


XIP Output

Visual Analytics of XIP Output o Dataset

Proceedings of the Learning Analytics and Knowledge (LAK)

conferences since 2011and a journal special issue, and the

Educational Data Mining (EDM) conferences since 2008 and

journal.

o XIP Analysis

All of the papers (66 LAK and 239 EDM papers 305 in

total) were analysed using XIP, extracting 7847 sentences and

40163 concepts.

o Paper Prototype


o Similar patterns in LAK &

EDM

o Summary & Contrast

categories higher &

rising

Visual Analytics of XIP Output









Learning Analytics Potential?

oA promising language technology

oNow visual analytics built on top of it

oWhat happens when we bring it into the realm

of learning?


S288 Practical Science

o ~300 undergraduate students

o 5 strands:

o Physics and Astronomy

o Chemistry and Analysis

o Environmental Sciences

o Earth and Environment

o Biology and Health

o Collaborative scientific writing assignment




XIP Highlighted Student Essays

Is the use of XIP’s rhetorical

categories predictive of final

grades?

Student Grades



What is the overlap between

XIP’s output and how tutors

judge quality?

Tutor Highlighted Student Essays


XIP Highlighted Student Essays

Do learners find it helpful to receive

formative feedback of this sort on their

own writing, or of the literature?


Learning Analytics Potential? C

O

L

L

A

B

O

R

A

T

I

V

E

W

R

I

T

I

N

G

E

D

I

T

O

R

Resources & Contact Details

o Simsek, D., Buckingham Shum, S., Sándor, Á., De Liddo, A. and Ferguson, R.

(2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of

Scientific Metadiscourse. 1st International Workshop on Discourse-Centric

Learning Analytics. (3rd International Conference on Learning Analytics &

Knowledge, 8 April 2013, Leuven, Belgium). http://bit.ly/DCLA13

o Taibi, D., Sándor, Á., Simsek, D., Buckingham Shum, S., De Liddo, A. and

Ferguson, R. (2013) Visualizing the LAK/EDM Literature Using Combined

Concept and Rhetorical Sentence Extraction, 1st Learning Analytics and

Knowledge Data Challenge at Learning Analytics and Knowledge (LAK ’13),

Leuven, Belgium http://bit.ly/DataChal

o De Liddo, Anna; Sándor, Ágnes and Buckingham Shum, Simon (2012).

Contested Collective Intelligence: rationale, technologies, and a human-machine

annotation study. Computer Supported Cooperative Work (CSCW), 21(4-5) pp.

417–448. http://bit.ly/HumanVsXIP

people.kmi.open.ac.uk/simsek

[email protected]

simsekduygu_
