Flash Forums and ForumReader:

Crafting an Interface for a New Kind of Online Discussion

Kushal Dave, kushal@google.com*Martin Wattenberg, mwatten@us.ibm.com

Michael J. Muller, michael_muller@us.ibm.com

IBM Research / Collaborative User Experience

Cambridge, MA USA

*Work done at IBM

Outline

• Flash Forums– What are they?– Why do they matter?

• ForumReader– Design decisions– Demo

• User feedback– Jam trial– Lab study

• Conclusions

Reading 892 of these is hard!

Flash forum examples

• Slashdot– “News for nerds” portal– Several articles discussed daily– Hundreds of posts per topic within a day

• IBM Jams– Company-wide discussions– Several broad forums– Thousands of posts over 3 days

• Blog comments, news discussions, et al.• As much text as a small novel

Flash Forums

In contrast to Usenet and other ongoing forums…

• Diffuse authorship• Large size• Focused topic• Short duration

• Cf. flash mobs: “a large group of people who gather in a usually predetermined location, perform some brief action, and then quickly disperse” [wordspy.com]

• Often, shallower threads

Flash forums are less conversational

0.00

10.00

20.00

30.00

40.00

50.00

60.00

70.00

80.00

90.00

0.00 20.00 40.00 60.00 80.00 100.00

Percent of messages that are replies

Pe

rce

nt

of

au

tho

rs p

os

tin

g m

ore

th

an

on

ce

Usenet

Slashdot

JamFlash forums

Flash forum threads are shallower

0

10

20

30

40

50

60

70

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19Thread depth

Per

cen

t o

f m

ess

ages

Usenet (Linux)

Usenet (Prog)

VJam (Role)

VJam (Impact)

Slashdot (MySQL)

Slashdot (Windows)

Flash forums

Why diffuse authorship matters

• Authors are relevant…– Some known users on Slashdot– Some key personalities in IBM– Metadata about users (reputation,

seniority) provide cues• …but ideas become focus

– Users indicate ideas trump authors (unlike Usenet)

– Official and distributed moderation (try to) create meritocracy

Why time limits matter

“A Jam’s authenticity is derived from the fact that it’s a real-time and finite event, and that there are real, often serendipitous ‘knowledge accidents’ among participants that emerge because of the time constraint imposed.”

[Dorsett, Fontaine, O’Driscoll]

Why size and topic matter

• Many simultaneous posts, constrained topic, and shallow threads lead to:– Thread drift– Particular themes split across threads– Diminished utility of threads as filters

Discussion interfaces

• Much work on Usenet– Conversation Map– Netscan

• Our problem is different– Authors are inadequate filters– Threads are inadequate filters– Everything happens at once– Basically, the discussion is one big mess

ForumReader

• Easily move around the discussion• Many ways to enter discussion• Sense of orientation• Integrate visualization and text analytics

• Builds on existing work in thumbnail interfaces– SeeSoft/SeeSys, Reader’s Helper, Context

Lens

DEMO!

Jam feedback

• 8,973 posts, 22,000 participants• Survey of 1,248 participants:

– 16% used ForumReader successfully– Important (3.5 / 5) and satisfying (3.2 / 5)– Value in orientation– Ability to find themes came up repeatedly “Amazing. To be able to locate

commonalities, etc., and analyze the worth of this VALUABLE effort IS GREAT!”

But we still didn’t know…

• How do users navigate discussions? What cues do they use?

• Does the interface really help them understand the scope of discussions and find information? Which features are most valuable?

Lab study design

2x2 varying visualization, text analysis

Data Collected• Describe expectation• Explore discussion• Identify key arguments• Generate mindmap or outline• Indicate relative amounts of discussion of

topics• Argue for or against• Evaluation

4 4

4 4

VisualizationY N

Tex

t A

nal

ytic

sN

Y

Preference

0

1

2

3

4

5

6

Color by author

Color by re

cency

Layered text v

iew

Tree View

Color by ty

pe

Click to

search

List of e

xtracted to

pics

Color by sim

ilarity

Graphic map

Color selector

Color by m

oderation

Type to search

Scrollbar

Simple te

xt view

Su

bje

cts

' ra

tin

g o

f fe

atu

re

Users consider map, search andmoderation highlighting as valuableas scrollbar and text view.

Performance

• Visualization and text analytics might improve performance individually, but detract together. Bullet Items

0

1

2

3

4

5

6

7

8

9

10

noViz Viz

Visual i zation

Mindmap Items

0

2

4

6

8

10

noViz Viz

Visualization

Reasons

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

noViz Viz

Visual i zation

Subjective Score

0

0.5

1

1.5

2

2.5

noViz Viz

Visual i zation

Purple:Text analytics

Blue:No text

Left columns:No viz.

Right columns:Visualization

N.S.

P<.04

p<.03

P<.05

F(1,12)=1.95

F(1,12)=5.20

F(1,12)=6.57

F(1,12)=4.26

Navigation patterns

Rows are conditions: map (top), NLP (second), control (third), map + NLP (bottom)Dots show navigation events: scrollbar/arrow keys (blue), map (red), tree (cyan)Lines show click search (black), typed search (cyan), highlighting (all others)

Users used the map (red dots) extensively, often nonlinearly.

Reading patterns

Users spent more time viewing starts of threads, highly-moderated posts, and starts of discussions.

Summary

• Flash forums present novel structure and dynamics (diffuse, big, focused, short)

• Users value cues like readwear, moderation, threads, authors (especially themselves!)

• Navigation is idiosyncratic• Our visualization and text analytics appear

to help users see more of the discussion• But too much complexity may be distracting

Future Work

• Much more to learn about flash forums– Textual analysis– User goals– Moderation systems

• Opportunities for better interfaces– Multi-dimensional filtering– Anti-filtering: emphasize novelty, variety

Thanks!