Evolving Classification of the Behaviour of Computer Users...97,61% Number of commands subsequences:...

Evolving Classification ofthe Behaviour ofComputer Users

Jose Antonio IglesiasCAOS Research Group.

Carlos III University of Madrid (Spain)jiglesia@inf.uc3m.es

Comunication System Department. Lancaster Univeresity (UK)p.angelov@lancaster.ac.uk

Plamen Angelov

Jose Antonio Iglesias and Plamen Angelov

Evolving Classification of Computer Users Behavior 1

Evolving Classification of Computer Users Behavior

OutlineOutline

•From RoboCup …

•… to UNIX User Classification

• Creating a Profile Model

• Evolving classification

• Results

• Conclusions

FromFrom RoboCupRoboCup……

Opponent behavior Modelling / Classification(Environment: soccer simulation domain)

OpponentModeling

PatternRecognition

Off-Line Analysis

No-PatternLogFile

PatternLogFile

Base Estrategy

Pattern

RecognizedPatterns

On-Line ComparingMethod

PatternDetection

On-Line Detection

Environment Information Advices to

Players

RoboCup Soccer Server

PatternRecognized

FromFrom RoboCupRoboCup……

BehaviorBehavior ClassificationClassification

Behavior as sequence of actions

Pass1to2, Dribble2, Pass2to10, Goal10, Pass1to2, Dribble2, Pass2to10, Goal10,

Pass5to6, Pass6to3, Foul5, Pass5to6, Pass6to3, Foul5, MissedShot3MissedShot3……

…… toto UNIX UNIX UsersUsers CommandsCommands

BehaviorBehavior ClassificationClassification

Behavior as sequence of commands

*SS**SS*, , postnewspostnews, , enscriptenscript, , fgfg, , enscriptenscript, , cdcd, , *ES*, *SS*,*ES*, *SS*, rloginrlogin……

*SS*,*SS*, date, ls, emacs, rm, ls, date ... date, ls, emacs, rm, ls, date ...

OutlineOutline

•From RoboCup …

• Results

• Conclusions

SequenceSequence ofof commandscommands �� ProfileProfile ModelModel

ImportantImportant : : sequentialitysequentiality

TrieTrie ((reretrietrievalval)) data data structurestructure::

SpecialSpecial searchsearch treetree usedused forfor storingstoringelementselements andand itsits prefixesprefixes..

EveryEvery nodenode: : – represents a command– stores useful information (times appeared,…)

Behavior as sequence of commands

AnAn exampleexample trietrie

Sequence to insert initially in the trie: {ls � date � ls � dat � cat}

Sequence

AnAn exampleexample trietrie

Sequence to insert initially in the trie: {ls � date � ls � date � cat}

Sequence

SubSub--sequencesequence lengthlength: 3: 3

{ls � date � ls � date � cat}

{ls � date � ls}

Sub-sequences to insert in the trie:

and {date � ls � date } and {ls � date � cat}

Sub-sequences to insert in the trie:

{ls � date � ls} and {date � ls � date} and {ls � date � cat}

AnAn exampleexample trietrieSequenceSequence ofof commandscommands �� ProfileProfile ModelModel

ls [1]

date [1]

ls [1]

Sub-sequences to insert in the trie: {ls � date � ls} and {date � ls � date} and {ls � date � cat}

ls [1]

date [1]

ls [1]

date [1]

ls [1]

ls [2]

date [1]

ls [1]

date [1]

ls [1]

ls [2]

date [1]

ls [1]

date [2]

ls [2]

date [1]

ls [4]

date [3]

cat [1]

date [4]

ls [2]

date [1]ls [1]

cat [1]

Evaluate the relation/dependence between an element and its prefix

Two approaches:– Frequency-based method.� Statistical dependence method.

Our approach: Statistical Value used: Chi-square value.This value is stored in every node of the trie

EvaluatingEvaluating DependencesDependencesSequenceSequence ofof commandscommands �� ProfileProfile ModelModel

Support or

Relative Frequency

(of a subseq)=

Number of times the subsequence has been inserted into the tre.

Total number of subsequencesof equal size inserted

ls [4] [0.44]

date [3] [0.5]

cat [1] [0.33]

date [4] [0.44]

ls [2] [0.33]

date [1] [0.33]ls [1] [0.33]

cat [1] [0.11]

cat [1] [0.16]

Behavior � Distribution of relevant events.

ls ls-date ls-date-ls ls-date-cat date date-ls date-cat date-ls-date cat

OutlineOutline

•From RoboCup …

• Results

• Conclusions

Profile a user is a difficult task for different aspects:

1. Human behavior is usually erratic

2. Humans behave differently because of a change in their goals.

ClassificationClassification……

How to accurately profile a user while his/herbehavior changes constantly ???

ClassificationClassification……

Using Evolving Fuzzy Systems !!

Evolving fuzzy systems (EFS) can be defined as self-developing, self-learning fuzzy rule-based or neuro-fuzzy

systems that have both their parameters but also theirstructure self-adapting on-line.

EvolvingEvolving ClassificationClassification……

They are usually associated with streaming data andon-line (often real-time) modes of operation.

EvolvingEvolving ClassificationClassification ofof UNIX UNIX UsersUsers

ArchitectureArchitecture ofof thethe EvolvingEvolving ClassifierClassifierFRB Classifier:

Ri = IF (x1 is around x1i*) AND (x2 is around x2

i*) AND … (xn is around xn

THEN (Labeli)

X = [x1, x2, … xn] denotes the vector of features .

L = Li*; i*= arg min(DistCosine)i=1

0,9----0,5------0,3

0.80.3----0.2--0,1----0,2

0,30,80,2----0,10,6

0,10,20,750,30,5

asfdmvmailrmemacsvicatls-datedatels

ArchitectureArchitecture ofof thethe EvolvingEvolving ClassifierClassifier

ArchitectureArchitecture ofof thethe EvolvingEvolving ClassifierClassifierFor each User…

1. Recursive Calculation of Potential of Data Vector (x) of a User.

2. Update Potentials of the F.P. with the new data (x)

3. If (Pot (x)) > Max Pot(F.Pi))

Insert New F.P.

1. Classify x in a Cluster (using Cosine Distance)Add x as a new “point” in the Cluster.

1. ¿Necesary Remove any F.P.?

Calculate µ for each F.P.

If µi > e-1 � Remove F.P.

OutlineOutline

•From RoboCup …

• Results

• Conclusions

168 Users, 4 different user groups:

1. Novice – programmers 55 Users

2. Experienced-programmers 36 Users

3. Computer-Scientists 52 Users

4. Non-Programmers 25 Users

168 Users

ResultsResults -- Experimental Experimental DesignDesign

Commands per user: from 600 to 3000

ResultsResults -- Experimental Experimental DesignDesign

After processing the users commands:

Different commands per group:1. Novice – programmers (25614)2. Experienced-programmers (43049) 3. Computer-Scientists (66490)4. Non-Programmers (10572)

Different commands in the 4 groups: 135317

ResultsResults--UsingUsing 10Fold Cross 10Fold Cross ValidationValidationNumber of F.P. per group:

FP Nov = 2FP Exp = 2FP Sci = 1FP Non = 1

Number of F.P. using Cross Validation:

Novices: 3 / 2 / 7 / 2 / 2 / 2 / 3 / 4 / 2 / 2 /Experiences: 2 / 3 / 3 / 2 / 2 / 2 / 2 / 2 / 2 / 2 /Sciences: 2 / 2 / 1 / 2 / 1 / 1 / 1 / 1 / 3 / 3 /Non: 2 / 2 / 1 / 2 / 2 / 2 / 2 / 2 / 2 / 2 /

ResultsResults--UsingUsing 10Fold Cross 10Fold Cross ValidationValidation

25000non

05200sci

00360exp

00055nov

nonsciexpnov

----------

10NN5NN3NN1NNC4.5

Evolving System…

ClassificationRate:

Number of commands subsequences: 135317

24010non

05011sci

00351exp

00055nov

nonsciexpnov

----------

10NN5NN3NN1NNC4.5

Evolving System…

Similar number of F.P.

ClassificationRate:

97,61%

Number of commands subsequences: 45930Support > 0.005

24010non

05011sci

10350exp

10054nov

nonsciexpnov

----------

10NN5NN3NN1NNC4.5

Evolving System…

ClassificationRate:

97,02%

22030non

236122sci

10314exp

10450nov

nonsciexpnov

40,47%36,90%50,00%54,16%73,80%

10NN5NN3NN1NNC4.5

Evolving System…

ClassificationRate:

82,74%

16054non

63556sci

50364exp

20350nov

nonsciexpnov

52,97%53,57%53,57%44,64%73,80%

10NN5NN3NN1NNC4.5

Evolving System…

ClassificationRate:

81,54%

71215non

28141sci

201221exp

311239nov

nonsciexpnov

30,95%30,95%30,35%40,47%52,97%

10NN5NN3NN1NNC4.5

Evolving System…

ClassificationRate:

39,28%

22030non

236122sci

10314exp

10450nov

nonsciexpnov

40,47%36,90%50,00%54,16%73,80%

10NN5NN3NN1NNC4.5

Evolving System…

ClassificationRate:

82,74%

ConclusionsConclusions

The proposed Evolving System Classifier…

• Works with large data stream of data.

• Good results with large data stream of data.

• Fast

• Stores data very reduced

• There is no thresholds

• Does not requiere pre-training

• Starts “from scratch”

• ….

Thanks!

Evolving Classification of the Behaviour of Computer Users...97,61% Number of commands subsequences:...

Documents

Evolving Needs, Evolving Spaces - Haworthmedia.haworth.com/asset/53745/haworth-2west-casestudy... · Evolving Needs, Evolving Spaces One Haworth Center, 2West Renovation. One Haworth

Evolving a System of Classification for Scalp Defects and ...Successful reconstruction of the scalp requires careful preoperative planning, adequate debridement, precise intraoperative

The pleasures of contrapurposiveness: Kant, the sublime, and ...sro.sussex.ac.uk/45930/1/Deligiorgi_-_Kant_on_the_Sublime...de Kant Berlin: de Gruyter 1997), pp.338-55; Robert R. Clewis

Evolving Classification of Learners' Profiles using Machine Learning …trap.ncirl.ie/3851/1/parthpatel.pdf · 2019-10-11 · Machine Learning Techniques for Better Retention Rates

Evolving land cover classification algorithms for multi ...public.lanl.gov/jt/Papers/brumby_SPIE4480-14.pdf · algorithms that distinguish a range of land cover categories, and an

NEW TRENDS IN GASTRIC POLYPS: EVOLVING CLASSIFICATION …

Evolving markets require evolving market approaches

Evolving web, evolving search

Time-Evolving Relational Classiﬁcation and Ensemble …Time-Evolving Relational Classification and Ensemble Methods. In Proceedings of the 16th Pacific-Asia Conference on Knowledge

Collective Spammer Detection in Evolving Multi-Relational ...Collective Classification with Reports Results of Classification Using Reports Collective Spammer Detection in Evolving

Evolving Interoperable Network Architectures for NATO ... · Evolving Interoperable Network Architectures for NATO Coalition ... (by using cryptography, steganography, ... Evolving

Folksonomies: Diverse, Democratic and Evolving Classification

The Evolving African SecurityThe Evolving African Security ...aardvarkaoc.co.za/wp-content/Conf_Aug_2009/Henri Boshoff 1.pdf · The Evolving African SecurityThe Evolving African Security

Discovery in Databases II - LMU Munich · Classification over dynamic data II •Non‐stationary data (evolving distribution) • Concept drift Knowledge Discovery in Databases II:

The Evolving Roles of the Gemini, Blanco and … Mid IR photometry Taxonomic classification/ minerology (composition) Goodman GMOS, SCORPIO NASA IRTF Single object, high throughout,

"Evolving Algorithmic Requirements for Recognition and Classification in Augmented Reality," a Presentation from CogniVue

Evolving Technology, Evolving Audiences - Jonathan Marshall

Introduction...Parameters Learning Evolving the System process info goals Layer 3 Layer 2 Layer 1 Data Predictions/ Classification/ Control Density Estimation Increasing Flexibility,

Evolving Ratings for Evolving Systems...Evolving Ratings for Evolving Systems Mapping Out the Future of Residential HVAC Equipment Metrics Alice Rosenberg September 20, 2018 New Orleans,

Food Business Models An Evolving Classification Scheme of ... · 10/2/2014 An Evolving Classification Scheme of Local Food Business Models - eXtension)