View
219
Download
5
Tags:
Embed Size (px)
Citation preview
1
Tipping Points, Butterflies, and Black Swans: A Vision for Spatio-temporal Data Mining Analysis
Dr. James M. Kang and Daniel L. EdwardsInnoVision Basic and Applied Research OfficeNational Geospatial-Intelligence Agency
August 24, 2011
Approved for Public Release 11-412
2
VisionVision
The development of data mining and spatio-temporal analytical techniques to discover tipping-points, butterflies,
and black swans.
The development of data mining and spatio-temporal analytical techniques to discover tipping-points, butterflies,
and black swans.
Approved for Public Release 11-412
3
What are Tipping Points?What are Tipping Points?“the moment of critical mass, the threshold, the
boiling point” – M. Gladwell“the moment of critical mass, the threshold, the
boiling point” – M. Gladwell
Climate Tipping Point, Upsala Glacier, Patagonia, Argentina
(Source: http://www.changeclimate.org/)
Approved for Public Release 11-412
4
What is the Butterfly Effect?What is the Butterfly Effect?Behavior of dynamic systems
• Highly sensitive to initial conditions – J. Gleck• Involve topologically mixing – B. Hasselblatt
Behavior of dynamic systems • Highly sensitive to initial conditions – J. Gleck• Involve topologically mixing – B. Hasselblatt
Source: http://www.guardian.co.uk/world/interactive/2011/mar/22/middle-east-protest-interactive-timeline
Approved for Public Release 11-412
5
What are Black Swans?What are Black Swans?• Unpredictable patterns that do not appear to
be Gaussian with an exponential diminishing tail, but a flatter curve with tails that are fatter
• Have the following characteristics:• The event is a surprise (to the observer).• The event has a major impact.• After its first recording, the event is rationalized
by hindsight, as if it could have been expected.
– N. Taleb
• Unpredictable patterns that do not appear to be Gaussian with an exponential diminishing tail, but a flatter curve with tails that are fatter
• Have the following characteristics:• The event is a surprise (to the observer).• The event has a major impact.• After its first recording, the event is rationalized
by hindsight, as if it could have been expected.
– N. Taleb
Approved for Public Release 11-412
6
ChallengesChallenges
Approved for Public Release 11-412
7
Tipping Point ChallengesTipping Point Challenges• Assumptions about
a dataset may change before and after a tipping point event
• Tobler’s Law vs. Teleconnections
• Assumptions about a dataset may change before and after a tipping point event
• Tobler’s Law vs. Teleconnections
Approved for Public Release 11-412
8
Butterfly Effect ChallengesButterfly Effect Challenges
1. Depth – sufficient data to mine vs. scope of problem?
2. Breadth - breadth of data sufficient to sample problem?
3. Missing – key data/meta data missing?
3. Stability - of mined patterns?
1. Depth – sufficient data to mine vs. scope of problem?
2. Breadth - breadth of data sufficient to sample problem?
3. Missing – key data/meta data missing?
3. Stability - of mined patterns?
Bounding problem with sufficiency
Approved for Public Release 11-412
9
Black Swan ChallengesBlack Swan Challenges• As a Black Swan unfolds,
• Mined patterns over populations and time may not become “interesting”
• May not be prevalent or anomalous
• After a Black Swan is recognized (hindsight),• Bounding sufficiency may be
too complex to overcome • May not generalize to other
known black swans
• As a Black Swan unfolds, • Mined patterns over populations
and time may not become “interesting”
• May not be prevalent or anomalous
• After a Black Swan is recognized (hindsight),• Bounding sufficiency may be
too complex to overcome • May not generalize to other
known black swans
Approved for Public Release 11-412
10
First and Next StepsFirst and Next Steps• Tipping Points
• Existing literature in abrupt changes, transitions, etc.
• Transient vs. Persistent
• Butterflies and Black Swans• Can these be generalized?• Are these even possible?• How can we begin quantifying
these events?
• Example Datasets• Guardian’s event dataset of
middle-east• CIA World Factbook dataset
• Tipping Points• Existing literature in
abrupt changes, transitions, etc.
• Transient vs. Persistent
• Butterflies and Black Swans• Can these be generalized?• Are these even possible?• How can we begin quantifying
these events?
• Example Datasets• Guardian’s event dataset of
middle-east• CIA World Factbook dataset
Source: https://www.cia.gov/library/publications/the-world-factbook/
Approved for Public Release 11-412
11
www.nga.mil
Approved for Public Release 11-412