View
0
Download
0
Category
Preview:
Citation preview
Enterprise Intelligencevs. Enterprise Amnesia
Jeff JonasChief Scientist, IBM Entity AnalyticsIBM Distinguished Engineer
© 2009 IBM Corporation© 2009 IBM Corporation
The data is the question.
More data is better and faster.
Bad data is good.
Intelligence: What If?
© 2009 IBM Corporation© 2009 IBM Corporation
1983: Founded Systems Research & Development (SRD)
1992: Assisted Vegas casinos in detecting the unwanted – resulting in a technology known as NORA
1996: Developed a consolidated consumer database drawing from 4,200 disparate systems, modeling 80M consumers and their 1.6B transactions
2001/2003: Funded by In-Q-Tel, the CIA’s venture capital arm
2003: Hired a CEO
2005: Acquired by IBM, now Chief Scientist, IBM Entity Analytics
Today: Focus is in the area of ‘sensemaking on streams’ with special attention towards privacy and civil liberties protections
Background
© 2009 IBM Corporation© 2009 IBM Corporation
”The data must find the data … and the relevance
must find the user!”
© 2009 IBM Corporation© 2009 IBM Corporation
State of the Union:Enterprise Amnesia
© 2009 IBM Corporation© 2009 IBM Corporation
EmployeeDatabase
InvestigationsDatabase
Human Resources
CorporateSecurity
Enterprise Amnesia
Sales and Service
CustomerDatabase
© 2009 IBM Corporation© 2009 IBM Corporation
EmployeeDatabase
InvestigationsDatabase
The newly hired employee had previously been arrested for stealing from you … same store!
Human Resources
CorporateSecurity
Enterprise Amnesia
Sales and Service
CustomerDatabase
© 2009 IBM Corporation© 2009 IBM Corporation
Amnesia is Embarrassing
Amnesia is Expensive
© 2009 IBM Corporation© 2009 IBM Corporation
Time
Com
puti
ng P
ower
Gro
wth
All Digital Data
SensemakingAlgorithms
Amnesia Index
Faster Computers Making Us Dumber
© 2009 IBM Corporation© 2009 IBM Corporation
is the leading cause of
enterprise amnesia.
Perception isolation
© 2009 IBM Corporation© 2009 IBM Corporation
Enterprise Intelligence
requires
“Persistent Context.”
The Brain!
© 2009 IBM Corporation© 2009 IBM Corporation
scrila34@msn.com
Information Without Context
© 2009 IBM Corporation© 2009 IBM Corporation
scrila34@msn.com
Information Without Context
© 2009 IBM Corporation© 2009 IBM Corporation
scrila34@msn.com
Information Without ContextData
© 2009 IBM Corporation© 2009 IBM Corporation
Information In Context
© 2009 IBM Corporation© 2009 IBM Corporation
scrila34@msn.com
Information In Context
© 2009 IBM Corporation© 2009 IBM Corporation
NewsletterSubscriber
Top 200Customer
Term“no rehire”
Pending Investigation
scrila34@msn.com
Information In Context
© 2009 IBM Corporation© 2009 IBM Corporation
Introducing Persistent Context
© 2009 IBM Corporation© 2009 IBM Corporation
“Remembering in a database (persistent) how things relate to each other (context)”
Introducing Persistent Context
© 2009 IBM Corporation© 2009 IBM Corporation
Context Accumulation: Think Pixels to Pictures
An assertion (connects, near its neighbors, or un-associated) is made with the arrival of each new piece
Assertions favor the false negative
New connecting pieces … create new entities that are re-evaluated against the remaining entities
Some pieces produce epiphanies
New pieces sometimes reverse earlier assertions
Working space quickly begins to exceed the final size
There can come a tipping point – a collapsing of the working space
© 2009 IBM Corporation© 2009 IBM Corporation
Observations
Uni
que
Iden
titi
es
Actual Identities
Six (6) Liars Detected Here
More, Better, Smarter & Faster
© 2009 IBM Corporation© 2009 IBM Corporation
Ingesting 800Menterprise
observations in 4 days!
Very Fast
P595– 32 CPU (64 SMT)– 512GB RAM– 32x 1GB adaptors
4x DS4800 controllers– 14x EXP810 trays with 36GB 15k RPM drives
1x DS4700 controllers
6x BladeCenters– 14x HS21 blades (4 cores each)
1x Force 10 switch
© 2009 IBM Corporation© 2009 IBM Corporation
ProspectDatabase
Human Resources
EmployeeDatabase
CorporateSecurity
InvestigationsDatabase
Sales and Service
CustomerDatabase
The “Data is the Query” Beats “Boil the Ocean”
© 2009 IBM Corporation© 2009 IBM Corporation
ProspectDatabase
Human Resources
EmployeeDatabase
CorporateSecurity
InvestigationsDatabase
Sales and Service
CustomerDatabase
Batch Analytics
The “Data is the Query” Beats “Boil the Ocean”
© 2009 IBM Corporation© 2009 IBM Corporation
Persistent Context
ContextAnalysis
Relevance Detection
FeatureExtract & Enhance
Publish
Notice Respond
Sensemaking on Streams … How To
© 2009 IBM Corporation© 2009 IBM Corporation
Persistent Context
ContextAnalysis
RelevanDetecti
FeatureExtract & Enhance
Notice
ObservationsStructured
UnstructuredAudio/VideoGeospatialBiometrics
Etc.
QuestionsSearch, Discovery,Context Requests
Etc.
Sensemaking on Streams … How To
© 2009 IBM Corporation© 2009 IBM Corporation
sistent Context
extysis
Relevance Detection
Publish
Respond
CONSUMERSOperational SystemsBusiness IntelligenceData MiningPattern DiscoveryPredictive ModelingCase ManagementVisualizationEtc.
Answers to questions
Sensemaking on Streams … How To
© 2009 IBM Corporation© 2009 IBM Corporation
1st Principle
If you do not process new enterprise observations first like a question … then you will
not know if it matters …unless someone asks.
© 2009 IBM Corporation© 2009 IBM Corporation
What Makes a Smart System … Smart?
Dynamic and resilient “domain in-specific”engineering
No such thing as a single version of truth
Bad data good
New observations reverse earlier assertions
© 2009 IBM Corporation© 2009 IBM Corporation
Drivers?
Competition.
© 2009 IBM Corporation© 2009 IBM Corporation
Human Capital
ToolsData
Domains for Competitive Advantage
© 2009 IBM Corporation© 2009 IBM Corporation
Human Capital
ToolsData
FastestSensemaking
Domains for Competitive Advantage
© 2009 IBM Corporation© 2009 IBM Corporation
Human Capital
ToolsData
First
FastestSensemaking
Domains for Competitive Advantage
© 2009 IBM Corporation© 2009 IBM Corporation
“Every millisecond gained in our program trading applications is worth $100 million a year.”
Goldman Sachs, 2007 * Source Automated Trader Magazine 2007
© 2009 IBM Corporation© 2009 IBM Corporation
The Way Forward
© 2009 IBM Corporation© 2009 IBM Corporation
Time
Com
puti
ng P
ower
Gro
wth
All Digital Data
SensemakingAlgorithms
Amnesia Index
Faster Computers Making Us Dumber
© 2009 IBM Corporation© 2009 IBM Corporation
Time
Com
puti
ng P
ower
Gro
wth
SensemakingAlgorithms
All Digital Data
The Way Forward
© 2009 IBM Corporation© 2009 IBM Corporation
Time
Com
puti
ng P
ower
Gro
wth
New/Useful Information
SensemakingAlgorithms
All Digital Data
Data Reduction
The Way Forward
© 2009 IBM Corporation© 2009 IBM Corporation
Time
Com
puti
ng P
ower
Gro
wth
New/Useful Information
SensemakingAlgorithms
All Digital Data
Data Reduction
ContextEngines
The Way Forward
© 2009 IBM Corporation© 2009 IBM Corporation
Enterprise Service Bus
ContextEngine
TransactionalActivity
Prospects
CustomerAcquisition
Call CenterCalls
Employees and Applicants
Vendors
Marketing Campaigns
Investigations and Arrests
Context Engine: A General Service
© 2009 IBM Corporation© 2009 IBM Corporation
1. Opportunity/risk analysis at point of transaction
2. Know your customer beyond householding – leverage social circles
3. Improve data quality - at collection
4. Higher dimensional data for improved modeling, statistics and management
5. Perpetual analytics for real-time changes to risk/opportunity
Enterprise Intelligence - For Example:
© 2009 IBM Corporation© 2009 IBM Corporation
y
x
good
bad
Risk ……… Moves
© 2009 IBM Corporation© 2009 IBM Corporation
y
x
good
bad
Risk ……… Moves
© 2009 IBM Corporation© 2009 IBM Corporation
y
x
good
bad
Risk ……… Moves
© 2009 IBM Corporation© 2009 IBM Corporation
y
x
good
bad
Mortgage Backed Securities?
© 2009 IBM Corporation© 2009 IBM Corporation
Smart Planet
New Intelligence
Information Agenda
Trusted Information
© 2009 IBM Corporation© 2009 IBM Corporation
FinancialServices
Retail
HealthCare
Energy
CounterTerrorism
CyberSecurity
SocialServices
PharmaResearch
Context Engines: Whole-Spectrum Benefits
© 2009 IBM Corporation© 2009 IBM Corporation
Blog References
Puzzling: How Observations Are Accumulated Into Context
Smart Systems Flip-Flop
When Risk Assessment is the Risk
Context: A Must-Have and Thoughts on Getting Some …
Algorithms At Dead-End: Cannot Squeeze Knowledge Out Of A Pixel
You Won't Have to Ask - The Data Will Find Data and Relevance Will Find the User
Streaming Analytics vs. Perpetual Analytics (Advantages of Windowless Thinking)
Federated Discovery vs. Persistent Context – Enterprise Intelligence Requires the Later
It Turns Out Both Bad Data and a Teaspoon of Dirt May Be Good For You
How to Use a Glue Gun to Catch a Liar
© 2009 IBM Corporation© 2009 IBM Corporation
Enterprise Intelligencevs. Enterprise Amnesia
Jeff JonasChief Scientist, IBM Entity AnalyticsIBM Distinguished Engineer
Recommended