26
David Talby @davidtalby SVP Engineering, Atigeo HUNTING CRIMINALS WITH HYBRID ANALYTICS, SEMI-SUPERVISED LEARNING, AND AGENT FEEDBACK Claudiu Branzan @melcutz Principal Data Science Lead, Atigeo

Hunting criminals with hybrid analytics -- October 2015

Embed Size (px)

Citation preview

Page 1: Hunting criminals with hybrid analytics -- October 2015

David Talby

@davidtalby

SVP Engineering, Atigeo

HUNTING CRIMINALS WITH HYBRID ANALYTICS, SEMI-SUPERVISED LEARNING, AND AGENT FEEDBACK

Claudiu Branzan

@melcutz

Principal Data Science Lead, Atigeo

Page 2: Hunting criminals with hybrid analytics -- October 2015

2

Page 3: Hunting criminals with hybrid analytics -- October 2015

3

IT’S NOT (JUST) ABOUT MONEY

33

Page 4: Hunting criminals with hybrid analytics -- October 2015

4

WHAT WE’RE UP AGAINST

44

50+Schemes(and counting)

99.9999%‘Good’ messages

6+Monthsper case

Needle in a haystack

Hybrid analytics

No training data

Semi-supervised learning

Adversarial learning

Online feedback

Page 5: Hunting criminals with hybrid analytics -- October 2015

5

WHY HYBRID ANALYTICS?

55

Ignore more rules

Unusual timing of

eventsUnusualpersonal network

Teamwork & scale

Think & talk differently

Page 6: Hunting criminals with hybrid analytics -- October 2015

6

(BITS OF) THE TOOLBOX

66

Rule Inference

Time Series

AnalysisLink Analysis

Ensemble Learning

Natural Language

Page 7: Hunting criminals with hybrid analytics -- October 2015

7

CAN WE SEE SOME CODE PLEASE?

7 7

Freely available IPython notebooks

Open source libraries & open data

Jump-start via AWS Marketplace

Page 8: Hunting criminals with hybrid analytics -- October 2015

8

STREAM PROCESSING

88

Kafka

Email Stream

Account transactions Stream

Email NLP Features

People graph

Transactions time series

Page 9: Hunting criminals with hybrid analytics -- October 2015

9

SAMPLE EMAIL PATTERNS

Page 10: Hunting criminals with hybrid analytics -- October 2015

1 0

SAMPLE NATURAL LANGUAGE ANNOTATORS

Understand vocabulary

– Jargon

– Code words

– Multi-lingual

Understand grammar

– Who are we talking about?

– Past, present or future?

– Compound sentences

Understand context

– Email: Re:, Fwd:, attachments

– SMS & IM have their own grammar

Page 11: Hunting criminals with hybrid analytics -- October 2015

1 1

SAMPLE GRAPH FEATURESStandard algorithms like KMeans don’t work on “haystacks”

Page 12: Hunting criminals with hybrid analytics -- October 2015

1 2

SAMPLE GRAPH FEATURESBregman Bubble Clustering

Page 13: Hunting criminals with hybrid analytics -- October 2015

1 3

USER ANALYSIS ITERATION

Email NLP Features

User graph

Transactionstime series

Graph Features

Time SeriesFeatures

NLP Features

Agent Feedback

Trai

n /

Tes

t C

lass

ifie

r

Page 14: Hunting criminals with hybrid analytics -- October 2015

1 4

Really

• Makes the world a better place • Needle in a very large haystack– Actually needs a petabyte-scale platform

• Multi-modal: no single trick works– Hybrid analytics

• No labeled data– Semi-supervised learning

– Cold start problem

• Sparse & high-dimensional

– Graph based features & change over time

• Adversarial– Feedback & online learning

Technically

SUMMARY: WHY HUNTING CRIMINALS IS COOL

1414

Page 15: Hunting criminals with hybrid analytics -- October 2015

1 5 15

@Atigeo

@melcutz

@davidtalby

Page 16: Hunting criminals with hybrid analytics -- October 2015

© 2015 Atigeo, Corporation. All rights reserved. Atigeo and the xPatterns logo are trademarks of Atigeo. The information herein is for informational purposes only and represents the current view of Atigeo as of the date of this presentation. Because Atigeomust respond to changing market conditions, it should not be interpreted to be a commitment on the part of Atigeo, and Atigeo cannot guarantee the accuracy of any information provided after the date of this presentation. ATIGEO MAKES NO WARRANTIES,EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Page 17: Hunting criminals with hybrid analytics -- October 2015

APPENDIXIn case the live demo gets cold feet on stage

1 7

Page 18: Hunting criminals with hybrid analytics -- October 2015

1 8

Page 19: Hunting criminals with hybrid analytics -- October 2015

1 9

Page 20: Hunting criminals with hybrid analytics -- October 2015

2 0

Page 21: Hunting criminals with hybrid analytics -- October 2015

2 1

Page 22: Hunting criminals with hybrid analytics -- October 2015

2 2

Page 23: Hunting criminals with hybrid analytics -- October 2015

2 3

Page 24: Hunting criminals with hybrid analytics -- October 2015

2 4

Page 25: Hunting criminals with hybrid analytics -- October 2015

2 5

Page 26: Hunting criminals with hybrid analytics -- October 2015

2 6