Enterprise intelligence apr2012 load - romania - 30 min

Enterprise Intelligence

Jeff Jonas, IBM Distinguished EngineerChief Scientist, IBM Entity Analytics

Email: jeffjonas@us.ibm.comBlog: www.jeffjonas.typepad.com

Twitter: http://www.twitter.com/jeffjonas

My Background

Early 80‟s: Founded Systems Research & Development (SRD), a custom software consultancy

Personally designed and deployed +/- 100 systems, a number of which contained multi-billions of transactions describing 100‟s of millions of entities

1989 – 2003: Built numerous systems for Las Vegas casinos including a technology known as Non-Obvious Relationship Awareness (NORA)

2001: Funded by In-Q-Tel, the venture capital arm of the CIA

2005: IBM acquires SRD

Today: Primarily focused on „sensemaking on streams‟ with special attention towards privacy and civil liberties protections

Sensemaking Algorithms

Available Observation

Context

Trend: Organizations Are Getting Dumber

EnterpriseAmnesia

Every two days now we create as much information as we did from the dawn of civilization up until 2003.”

~ Eric Schmidt, CEO Google

Amnesia, definition

A defect in memory, especially resulting from brain damage.

Enterprise Amnesia, definition

A defect in memory, resulting in wasted resources, lower revenues, unnecessary fraud losses, etc.

ContextWHY?

Trend: Organizations Are Getting DumberC

Algorithms at Dead End.

You Can‟t Squeeze Knowledge

Out of a Pixel.

scrila34@msn.com

No Context

Context, definition

Better understanding something by taking into account the things around it.

Information in Context … and Accumulating

Top 200Customer

Job Applicant

IdentityThief

CriminalInvestigation

scrila34@msn.com

The Puzzle Metaphor

Imagine an ever-growing pile of puzzle pieces of varying sizes, shapes and colors

What it represents is unknown – there is no picture on hand

Is it one puzzle, 15 puzzles, or 1,500 different puzzles?

Some pieces are duplicates, missing, incomplete, low quality, or have been misinterpreted

Some pieces may even be professionally fabricated lies

Until you take the pieces to the table and attempt assembly, you don‟t know what you are dealing with

Puzzling

Cottage Garden

Down Home Music

Artist

Licensed by Cypress Fine

Art Licensing

Neuschwanstein Beauty

Robert Cushman Hayes

Artwork provided by

Hadley House Licensing,

Minneapolis

270 pieces

90%200 pieces

150 pieces

6 pieces2%

30 pieces10% (duplicates)

First Discovery

More Data Finds Data

Duplicates in Front Of Your Eyes

First Duplicate Found Here

Incremental Context – Incremental Discovery

6:40pm START

22min “Hey, this one is a duplicate!”

35min “I think some pieces are missing.”

37min “Looks like a bunch of hillbillies ona porch.”

44min “Hillbillies, playing guitars, sittingon a porch, near a barber sign …and a banjo!”

150 pieces

Incremental Context – Incremental Discovery

47min “We should take the sky and grassoff the table.”

2hr “Let‟s switch sides, and see if wecan make sense of this fromdifferent perspectives.”

2hr10m “Wait, there are three … no, fourpuzzles.”

2hr17m “We need a bigger table.”

2hr18m “I think you threw in a few randompieces.”

How Context Accumulates

With each new observation … one of three assertions are made: 1) Un-associated; 2) placed near like neighbors; or 3) connected

Must favor the false negative

New observations sometimes reverse earlier assertions

Some observations produce novel discovery

As the working space expands, computational effort increases

Given sufficient observations, there can come a tipping point

Thereafter, confidence improves while computational effort decreases!

Big Data [in context]. New Physics.

More data: better the predictions– Lower false positives

– Lower false negatives

More data: bad data good– Suddenly glad your data is not perfect

More data: less compute

Big Data

Pile of ____ In Context

One Form of Context: “Expert Counting”

Is it 5 people each with 1 account … or is it 1 person with 5 accounts?

Is it 20 cases of H1N1 in 20 cities … or one case reported 20 times?

If one cannot count … one cannot estimate vector or velocity (direction and speed).

Without vector and velocity … prediction is nearly impossible.

Entity ResolutionDemonstration

VOTERGeorge F Balston

YOB: 1951 D/L: 4801

13070 SW Karen Blvd Apt 7

Beaverton, OR 97005

Last voted: 2008

DECEASED PERSONGeorge Balston

YOB: 1951 SSN: 5598

DOD: 1995

Entity Resolution Demonstration

When it comes to best practices in voter matching, if only a name and year of birth match, this is insufficient proof of a match. Many

different people in the U.S. share a name and year of birth.

Human review is required.

Unfortunately, there are thousands and thousands of cases just like this and state election offices don‟t have the staff (or budget) to

manually review such volumes.

YOB: 1951 D/L: 4801

Beaverton, OR 97005

Last voted: 2008

YOB: 1951 SSN: 5598

DOD: 1995

Now Consider This Tertiary DMV Record

DMVGeorge F Balston

YOB: 1951 SSN: 5598 D/L: 4801

3043 SW Clementine Blvd Apt 210

Beaverton, OR 97005

The DMV record contains enough features to match both the voter (name, year of birth and driver‟s license) and/or the deceased persons record (name, year of birth and SSN). For the sake of argument, let‟s

say it matches the voter best.

YOB: 1951 D/L: 4801

Beaverton, OR 97005

Last voted: 2008

DMVGeorge F Balston

YOB: 1951 SSN: 5598 D/L: 4801

Beaverton, OR 97005

YOB: 1951 SSN: 5598

DOD: 1995

Features Accumulate

The voter/DMV record now shares a name, year of birth and SSN with the deceased person record. In voter matching best practices, this evidence would be sufficient to make a determination that this voter

is in fact deceased. This case no longer needs human review.

YOB: 1951 D/L: 4801

Beaverton, OR 97005

Last voted: 2008

DMVGeorge F Balston

YOB: 1951 SSN: 5598 D/L: 4801

Beaverton, OR 97005

YOB: 1951 SSN: 5598

DOD: 1995

Useful Insight Revealed!

As features accumulate it becomes possible to resolve previous un-resolvable identity

records.

As events and transactions accumulate –

detection of relevance improves.

Here we can see George who died in 1995 voted in

IBM InfoSphere Identity Insight V8

MoneyGram International

Enterprise IntelligenceOne Plausible Journey

ObservationSpace

Sense and Respond

What you know

New Observations

ObservationSpace

Decide

?Relevance

Finds the Sensor(<200ms)

Data Finds Data

Sense and Respond

Explore and Reflect

ObservationSpace

Decide

DirectedAttention

Relevance Find You

DeepReflection

CuratedData

PatternDiscovery

RelevanceFinds the Sensor

(<200ms)

Data Finds Data

Sense and Respond

ObservationSpace

Decide

DirectedAttention

NEWINTERESTS

DeepReflection

CuratedData

PatternDiscovery

(<200ms)

Data Finds Data

Explore and ReflectSense and Respond

ObservationSpace

Decide

DeepReflection

CuratedData

PatternDiscovery

(<200ms)

Data Finds Data

InfoSphere StreamsILog

NetezzaSPSS

Watson

DirectedAttention

Cognos

InfoSphere Streams

NEWINTERESTS

SPSSSensemaking

ObservationSpace

Decide

DirectedAttention

NEWINTERESTS

DeepReflection

CuratedData

PatternDiscovery

(<200ms)

Data Finds Data

Report and Manage

Decide

DirectedAttention

NEWINTERESTS

PatternDiscovery

(<200ms)

Data Finds Data

Info Management Systems

Content ManagementCase ManagementData Warehousing

Report and Manage

Big Data Trends

The Greater the Context, the Greater the Value

Pile of Data

Records Managed(Big) (Ludicrous Big)

Data in Context

Willing

tThe better the

predictions … the faster they will be

wanted.

“Why did we have to wait until the

end of the day for the smart answer?”

Time Is Of The Essence

Relevance (Iffy) (Totally)

Real-Time

Closing Thoughts

The most competitive organizations

are going to make sense of what they are observing

fast enough to do something about it

while they are observing it.

Context

Wish This On The Competitor

EnterpriseAmnesia

The Way Forward: Enterprise Intelligence

Context

Enterprise intelligence apr2012 load - romania - 30 min

Technology

20120329 Apr2012 Guidelines LowBackPain

Linda Gallery Updates [Jan-Apr2012]

Top 5 Mistakes eBook Apr2012 Final

Schaefers and godfrey mcda 2 apr2012

hidden treasure-apr2012

Architect&Specificator: Mar/Apr2012

Conference bid West University of Timisoara, Romania · 2014. 4. 3. · NH Timisoara 4, 83 rooms 15 min walking from university or center Continental, 4, 164 rooms 15 min walking

SQL Server 2012 Whats New Breakthrough Insight Whitepaper Apr2012

New times. New life. New center.€¦ · National Bank of Romania (11 min drive) Main banks branches in close proximity BANKS Bucharest Tribunal (4 min drive) Trade Registry (10 min

BWPM Apr2012: Intro Slides

PRELIMINARY SYMPOSIUM PROGRAM...Dantes Nicolae Bratu, Romania Marius Cioca, Romania Sorin Mihai Radu, Romania Nicolae Voinoiu, Romania Stanislaw Prusek, Poland Constantin Bujor, Romania

Cockings - Revision Apr2012 (1)

Altaeros External Summaryv2 Apr2012

E2020 Field+Unit+Plan+Program+Summary+APR2012

DnV RP C103[Apr2012] Column Stabilized Units

WORLD GASTROENTEROLOGY NEWS€¦ · e-WGN Editorial Board • Anita Afzali, USA • Min-Hu Chen, China • Dan Dumitrascu, Romania • Sara Elfadil, Sudan • Amy Foxx-Orenstein,

Prehospital scene safety apr2012

4th - meditech.utcluj.ro · BADEA, Radu Romania RAFIROIU, Dan V. Romania MUJI, Marius Romania MAN, Victoria Romania CRETU, Mihaela Romania ROMAN, N. Marius Romania MUNTEANU, Mihai

FCEmodel descriptions Apr2012 - Florida Coastal …fcelter.fiu.edu/.../FCEmodel_descriptions_Apr2012.pdf!"#$%&’()$’(+,-./-&0$12.,-)$34536$ 5$ BACKROUND Historically, models were

Mms Mbs Apr2012