Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
1
Seeing Shakespeare (and Sequences):
Making Pictures to understand things you don’t want to read
Michael GleicherDept of Computer SciencesUniversity of Wisconsin ‐Madison
Acknowledgements• All of this work is done in collaboration with a great group of students and collaborators
• This talk is work done with students:(who didn’t want their pictures shown)
Feng Liu – multimedia, video(work supported by NSF, Adobe)Now at Portland State Univ
Greg Cipriano– molecules, vis(work supported by DOE,NIH)Now at Solidworks
Aaron Bryden– molecule motion(work supported by NIH)Now at DOE Ames Lab
Danielle Albers – genomics(work supported by NSF, DOE)
Adrian Mayorga– vis(work supported by NIH)
Michael Correll – humanities, virology(work supported by NSF, NIH)
Kevin Ponto– VR(work supported by NIH)
Sean Andrist, Tomislav Pejsa– agents(work supported by NSF)
Zack Krejci – vis(volunteer)
Seeing Shakespeare (and Sequences):
Making Pictures to understand things you don’t want to read
Michael GleicherDept of Computer SciencesUniversity of Wisconsin ‐Madison
Seeing Shakespeare (and Sequences):
Making Pictures to understand things you don’t want to read
Michael GleicherDept of Computer SciencesUniversity of Wisconsin ‐Madison
Seeing Shakespeare (and Sequences):
Making Pictures to understand things you don’t want to read
Michael GleicherDept of Computer SciencesUniversity of Wisconsin ‐Madison
Data Visualization
why Comp Sci Vis research might apply to you(especially if you are a humanist) Where is this guy coming from?
Some context
Some Stuff I Do(besides Visualization)
Analysis of Proteins Motion Synthesisfor Characters
Video Authoring
Image and VideoRetargeting
Non‐Verbal Cues forCommunicative Agents
Virtual Reality for Home Healthcare
What do these have in common?
It’s all stuff I’ve done in the past few years
It involves large amounts of data
It involves creating effective presentations
It requires some understanding of the datain order to simplify it
What do these have in common?
It’s all stuff I’ve done in the past few years
It involves large amounts of data
It involves creating effective presentations
It requires some understanding of the datain order to simplify it
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
2
What do these have in common?
It’s all stuff I’ve done in the past few years
It involves large amounts of data
It involves creating effective presentations
It requires some understanding of the datain order to simplify it
Does this all tie together?
How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and comprehending?
How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and comprehending?
How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and comprehending?
Talk Roadmap
Molecular Surface Abstraction
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
Talk Roadmap
Molecular Surface Abstraction
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
Biological Applications
BiologyTo
Humanities
Humanities Applications
Talk Roadmap
Molecular Surface Abstraction
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
ArtAnd
Perception
Art
Perception/Design
Application
The Future:Computational
Thinking
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
3
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
Molecular Surface Abstraction
Prelude: Art or Perception? A Protein Surface
Work with Greg Cipriano and George Phillips
An aside…How do scientists look at proteins?
Stick and Ball Model (internals)
An aside…How do scientists look at proteins?
Stick and Ball Model (internals)
Ribbon Diagram (internals)
An aside…How do scientists look at proteins?
Stick and Ball Model (internals)
Molecular Surface (externals)
A Protein Surface
Molecular Surface Abstraction
Work with Greg Cipriano and George Phillips. TVCG 2007, NAR 2010.
What’s Happening?
Simplification
Stylized Display
Surface Indications
Art
Abstraction
Good Lighting
Line Drawings
Non‐Photorealism
Visual Cognitive Science
Cue reduction
Provide Depth Cues
Enhance Contours
Tolerance of Shading
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
4
Why does B fight cancer?
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
Molecular Surface Abstraction
Inspired by Art (and a math trick) …
From Cutting (taken from McCloud), Representing motion in a static image, 2002
Molecular Motions
Bryden, Phillips, Gleicher. TVCG Jan ‘12
Molecular Motions
Coarse‐grained models
Normal‐mode Analysis (NMA)
Motion Illustration Motion Illustration
Artistic Inspirations:Comic Books / Diagrams
Abstract
Model
Illustrate
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
5
The Problem:Vector per point
AbstractGroup to fit models
Model
Illustrate
AbstractGroup to fit models
ModelAffine model per group
Illustrate x = A x
x = A x
AbstractGroup to fit models
ModelAffine model per group
IllustrateAffine ExponentialsGlyph Design
x = A x
x = A x
p(t) = eAt
p(t) = eAt
x = A x
x = A x
p(t) = eAt
p(t) = eAt
Explanations for Exploration Literature without Reading
Molecular MotionsMolecular Surface Abstraction
Learning from perception…
Sequence Comparison
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
6
How do we interpret massive amounts of sequence data?
A tough problem…
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
TACTAGCTAGTAGCTAGCATCGACTACGACTGAC
GCTAGCTGTTGCTAGCCGACTCATCGACTAAGCT
TCGACTAGCTAGATCGACTTATCGACTCACACTA
CTGGCTAGTTACACTATCTACCGACTGATCGACT
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
7
Mauve sequence alignment and visualization. Perna Lab.
How to see anything in this sea of data?
How to see anything in this sea of data?
1. Be Realistic2. New Designs based on Perception
Scalable Overviews of Whole Genome Sequence Alignments
Sequence Surveyor
Albers, Dewey, Gleicher. IEEE TVCG (InfoVis 2011)
Number of Genomes Length of Genomes Types of Inquiry
Scalable
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
8
Mauve (Perna lab) Sequence Surveyor
Visual Search
Visual Clutter Summarization
Pre-Attentive Phenomena
Perceptual Principles
Visual Search
Visual Clutter Summarization
Pre-Attentive Phenomena
Visual Search
Visual Clutter Summarization
Pre-Attentive Phenomena Visual Search
Visual Clutter Summarization
Pre-Attentive Phenomena Visual Search
Visual Clutter Summarization
Pre-Attentive Phenomena
What are you searching for?
What textures form? What statistics do you get?
What pops out?
What are you looking for? MappingColor Mapping Color Schemes Position Mapping
Index Membership Freq Grouped Freq Pos in Reference
Index
Group
ed Freq
Pos in Re
ference
Combinations of different color and position mappings reveal interesting things in the data
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
9
Blocking
Group (relatively) continuous sets of neighboring genes into a single unit
roftilS yaeQ phnA tadG
Aggregate Encodings
Average
Aggregate Encodings
Average Robust Average
Color Weaving Event Striping
Anecdotes: 100 Bacteria
Conservation relationships between different families of genomes
Color by position in reference (arrow), order by relative ordering
Anecdotes: Buchnera
Color by position in reference (arrow), order by set of genomes containing each gene
Anecdotes: Buchnera
Averaging:
No significant trend
Color Weaving: Overall distribution
Anecdotes: Fungi
Bioinformatics applications allow users to test algorithms using visual checks
Color by overall frequency, order by relative ordering
Anecdotes: Fungi
Bioinformatics applications allow users to test algorithms using visual checks
Color by position in a reference, order by relative ordering
Another ApplicationSince I can’t pronounce the Fungi …
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
10
A Different Kind of Evolution . . .
Google Books Word Count DataDe
cade
s ‐>
Word Rank ‐> Color by position in reference
1600: and the of to a in that it is not ...
1610: the of and to in a that his for is ...
1620: the and of to in that a is he his ...
1630: the of and in to a was that his as ...
1640: the of and to in a that was is it ...
1650: the of and to a in is that as it ...
1660: the and of to in a that is it he ...
. . .
1970: the of and to in a is that for as ...
1980: the of and to in a is that for as ...
1990: the of and to in a is that for as ...
2000: the of and to in a that is for was ...
A Different Kind of Evolution . . .
Google Books Word Count Data
Decade
s ‐>
Word Rank ‐> Color by position in reference
A Different Kind of Evolution . . .
Google Books Word Count Data
Decade
s ‐>
Word Rank ‐> Color by occurrence count
Number of Decades (aggregate)
Event Striping Selection: 15 Decades
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
11
Does this really work?Color Weaving for Aggregate Displays
Correll, Albers, Franconeri, Gleicher. Comparing Averages in Time Series Data. CHI 2012.
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
12
A Conversation aboutVisualization
Michael GleicherDept of Computer SciencesUniversity of Wisconsin ‐Madison
The spectra of Visualization
“Information”Visualization
“Scientific”Visualization
The spectra of Visualization
PresentationVisualization
ExploratoryVisualization
Science(of Visualization)
Practice(of Visualization)
Art(of Visualization)
Science(of Visualization)
Practice(of Visualization)
Art(of Visualization)
Caveat:Domain Science vs. Visualization Science
&
Tool Users
Toolsmiths
Theoreticians Designers
What do you need to know?
Domain science
Art / DesignPerception (Visual Cognition)Implementation (graphics, stats, databases, …)
Examples of other things that worked
The Class 2010Voluntary OverloadScheduled at Last MinuteGrad Special Topics (838)
14 “paying customers”4 dissertators/staff / facultyLess than half from CS
Can I learn this?
2012Regularly Scheduled ClassAdvertised via friendsUndergrad / Grad “meets with”
28 (+1) paying customers20 departments in original roster11 on final
Can I teach this?
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
13
http://graphics.cs.wisc.edu/Courses/Visualization12/
Case Study
What is VisualizationWhy VisualizeHow to Evaluate
Perception 101EncodingsColor
Multi‐VariateScalabilityInteraction
Case Study
Graphs / Networks / TreesAnimation / Presentation3D
2010Voluntary OverloadScheduled at Last MinuteGrad Special Topics (838)
14 “paying customers”4 dissertators/staff / facultyLess than half from CS
Can I learn this?
2012Scheduled special topicsAdvertised via friendsUndergrad / Grad “meets with”
28 (+1) paying customers20 departments in original11 departments on final
Can I teach this?
201XCreate a new courseHow to spread the word?What level?
Who to teach?What to teach them?Diversity of Needs
Should I teach this?Who to teach to?
Basic Math <‐> Statistics Grad Students
Artistically Inexperienced <‐> (ex‐) Pro Designer
Never Programmed <‐> Expert Implementer
No pet problems <‐> Has a “home” domain
Sophomore <‐> Dissertator
Never thought about it <‐> Visually Literate
Writes like an Engineer/Scientist <‐> Writes like a Huamities/Social Scientist
Who is interested?
A Research Program?
Lots of fun domains!
Visual ComparisonsA Common Thread?
There are general principles that apply across domains, data types, …
Visual Comparison:
And if we can figure it out, it’ll be easier to crank out the comparison tools/techniques quickly
Mike’s theory of visual comparison 0.2*
* This is a work in progress.Comments welcomed!
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
14
The 3 3s
3 Axes of Scalability / Hardness3 Strategies for Scalability3 Basic Designs
Comparison is easy. Until it gets hard.
3 Axes of Hardness
Number of things to compareSize/complexity of things to compareComplexity of the relationships
People can only compare a few things at a time*3 Strategies for scalingSelect SubsetSummarize StatisticallyScan Serially
My cognitive scientist friends say the magic number 7+/‐2 is an over‐simplification, but …
All designs appear to fall into 3 categories
3 Basic Designs*
* Each has its pros and cons
Develop Methods Specifically to Support Visual Comparison
Visual Comparison Punchline:
One example…
Mayorga and Gleicher. Splatterplots: Overcoming Overdraw. Submitted for publication.
Scatterplot? Splatterplot!
Mayorga and Gleicher. Splatterplots: Overcoming Overdraw. Submitted for publication.
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
15
Goo
gle Bo
oks
(broad
set)
Novels
Sequence Comparison
Explanations for Exploration
Molecular MotionsMolecular Surface Abstraction
Humanities Scholarship
Literature without Reading
Study Literature without Reading?
See patterns across language
Consider Larger Collections of Books
See small scale patterns in familiar texts
Be uncultured and still hang out with the cool kids
1. Do measurements of texts2. Make inferences from the statistics3. Build humanist‐style arguments
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
16
Until we take the time to learn about how the other side thinks, we can’t really work together.
Once we learn how each other thinks, our ways of thinking can infuse each other’s.
This is not just building tools for our friends.It’s a lotmore fun and interesting
The statistics are not the argument
Exemplars and OutliersGo back to the sources
Arguments based on context and knowledge
Multiple viewpoints and lenses
“Humanist” Collaborators:Robin Valenza*, Mike Witmore, Jonathan HopeCathy DeRose, Jason Whitt, …
Comp Sci Collaborators:Michael Correll, Danielle Albers, Zack Krejci, …
* Robin had a prior life in which she was one of us
1. Do measurements of texts2. Make inferences from the statistics3. Build humanist‐style arguments
1: Count Text tagging
Text Vector
4, 0, 3.6, 4.7, 0, 3.4, …
Count of wordCategory 1
Count of wordCategory 1
Count of wordCategory 1
CouCate
Texts Vectors
4, 0, 3.6, 4.7, 0, 3.4, …3, 2.4, 0, 4.2, 4.7, 5, …1.5, 2.3, 0, 1.2, 6.2, ……
Pieces of Texts Vectors
4, 0, 3.6, 4.7, 0, 3.4, …3, 2.4, 0, 4.2, 4.7, 5, …1.5, 2.3, 0, 1.2, 6.2, ……
Just counting
Words (phrases) have a type (tag)
DocuscopeSimple matching against a dictionaryHand‐built dictionaries
100‐115 Categories, 12‐17 Clusters (groups)
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
17
N
Not in dictionary
Not Tagged
15 “Clusters” (115 LATs)
How to look at 100+ dimensions?
Visualize them directlyDimensionality Reduction
Let the machine do it for you
1. Make Picture2. Read tea leaves
How to support “Humanist” arguments?
The statistics are not the argument
Exemplars and OutliersGo back to the sources
Arguments based on context and knowledge
Multiple viewpoints and lenses
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
18
Domain Specific Tools
Correll, Witmore, Gleicher. Exploring Collections of Tagged Text for Literary Scholarship.Computer Graphics Forum (Proceedings Eurovis), 2011.
Back to the details that cause patterns
Literature without Reading
Sequence Comparison
Molecular MotionsMolecular Surface Abstraction
And for my next trick…
Explanations for Exploration
Exploring High‐Dimensional Spaces
Gleicher. Unpublished Work in Progress
What we have:Measurements (counts) of things
What we want:Explanations of how these lead to properties of the objects that we care about
Shakespeare’s PlaysA Source of Examples
Genre?
AutoBio
Upd
ates
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
19
Request
Time Shift
Positivity
Negativity
The Comedicness axis?
Puts comedies on the rightNon‐comedies on the left
Provides a “measurement” of comedicness
Just a different view of the data
Positivity
Negativity
Dividing Line(best I can do)
Comedicness0 1 2 3‐1
Comedicness
Tragicne
ss
Comedicness
Tragicne
ss
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
20
Using expert knowledge to guide exploration
Is this cheating?
Machine LearningCorrectness
GeneralizesLarge Margin (gap)Efficient, Robust, Concise(proxies for generality)
Classifiers
Explanation BuildingCorrect – or interesting
Leads to good argumentsSimple (concise)Aligns with other knowledgeProvides multiple viewpointsProvides exemplarsOutliers to examine
Explainers!
-2.702 Immediacy - 2.519 Negativity + 1.894 Positivity - 1.846 ReportingStates + 1.160 FirstPer + 0.990 DirectAddress + 0.536 Question -0.395 CommonAuthorities -0.164 ProjectAhead + 0.067 ReportingEvents
(all 36 correct, large margin, 10 variables)
-0.304 Negativity + 0.147 FirstPer -0.095 CommonAuthorities
(5 “wrong”, 3 variables)
-10 Negativity + 5 FirstPer - 3 CommonAuthorities
(5 “wrong”, 3 variables)
-2 Negativity + 1 FirstPer - 1 CommonAuthorities
(5 “wrong”, 3 variables)
Explanations from Expert Explorations
Generate Explanations from given relationsTrade off accuracy vs. parsimony
Add additional knowledge to shape viewsGenerate alternate explanations
No additional ConstraintsSimpler ‐
Hamlet and TwelfthNight ConstrainedSimpler ‐
z =SVMpursuit(d,cp,5,svmparamsched=[.1,.2,.3,.4],quantize=5)
-5 Negativity + 2 FirstPer - 2 ComAuthor(3 variables)
-5 Inclusive + 3 Positivity - 2 AbstractConcepts + 1 PersProp(4 variables)
5 SelfDisclosure - 4 Motions - 2 StandardsPos + 1 DirectAddress(4 variables)
5 PredFuture - 2 SpaceRelation -2 Resistance + 1 Question + 1 PersProp - 1 SenseObject
(6 variables)
All with 32 correct
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
21
What do you want to ask?How do you want to look at data?
Expert Explorations lead to Explanations
Can you identify the genre of an act?Just based only on word usage…
We’re telling it what to explain
Stuff it “discovers” helps confirm
What act the play is?Can the word usage tell you…
I
II
III
IV
V
Presentation and Interactionis Challenging!
Isn’t this a Visualization talk?
Seeing Shakespeare (and Sequences) ‐ Talk by Gleicher 4/26/2012
22
Uniqueness of a play?One more Shakespeare case study
A Midsummer Night’s Dream
35 Other Plays
Person Property
Sense Object
Still need …
an integrated system for experimenting
ways to specify knowledgeways to assess explanationssystematic ways to generate alternatives
to see if this works in more casesto expand to richer classes of explanations
to do a lot of work. . . . .
Until we take the time to learn about how the other side thinks, we can’t really work together.
Once we learn how each other thinks, our ways of thinking can infuse each other’s.
This is not just building tools for our friends.It’s a lotmore fun and interesting
Literature without Reading
Sequence Comparison
Molecular MotionsMolecular Surface Abstraction
Where have we been?
Explanations for Exploration
Talk Roadmap
Molecular Surface Abstraction
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
Biological Applications
BiologyTo
Humanities
Humanities Applications
Talk Roadmap
Molecular Surface Abstraction
Explanations for Exploration Literature without Reading
Molecular Motions
Sequence Comparison
ArtAnd
Perception
Art
Perception/Design
Application
The Future:Computational
Thinking
How can we use our understanding of human perception and artistic traditions to improve our tools for communicating and comprehending?
Thanks!To you for listeningTo Robin and Carrie for inviting me
To my students and collaborators
To the folks who pay the bills(NSF, NIH, Mellon, …)
Michael [email protected] of Computer SciencesUniversity of Wisconsin ‐Madison