Apache Unomi In Depth - ApacheCon EU 2015 Session

APACHE UNOMI IN-DEPTH I 2nd October 20151

APACHE UNOMI IN-DEPTHSERGE HUBER, CTO & Co-founder, [email protected]


HISTORYNOT SO LONG AGO, IN A GALAXY NOT SO FAR AWAY…










INTRODUCING UNOMI

YOU KNOW ME ?

WHAT IS UNOMI?


TARGETED FUNCTIONALITIESMARKETING TO MARKETERS

UNOMI

User tracking

Event tracking

Goal tracking, scoring Segmentation FormInput

tracking

Downloadtracking

Impersonification(personas)

A/B testing

Profile (visitor,contact,leads)

management and exportReporting

Privacy management


ARCHITECTURE DESIGN GOALSFROM LESSONS LEARNED THE HARD WAY

1

2

3

Avoid putting additional load on the WCM nodes

Highly scalable (avoiding node cross-talk)

Make it easy for users to define and use marketing segments

4

5

6

Integration with existing analytics

Integration with multiple systems including mobile apps, not just WCM

Highly pluggable architecture, making it easy to extend and build different solutions based on a common core


SCALING ARCHITECTURETHROWING MORE SERVERS AT THE PROBLEM

CONTEXT SERVER

CMS

IssuetrackingCRMSocial


INPUT/OUTPUT

RULES

ACTIONS

ProtocolsHTTP REST + JSON

SEGMENTSEVENTS

CONTEXT

CONDITIONS

PROFILES

MUST REFRAIN… NO JOKES ON I/O

CONTEXT SERVER


REQUEST FLOW

HEY LOOK I HAVE ARROWS

Load HTML page

Load HTML + JS to connect Context Server

Load context.js

Context for current request

(Optional) Load content based on context

(Optional) Send event such as login / hover

BROWSER CMS CONTEXT SERVER

JS callbacks


CONTEXT EXAMPLE

digitalData = {"loaded": true,"user": [{

"profiles": [{"profileInfo": {

"profileId": "d6454520-f2b6-40b7-829c-d17214f209d1”,"firstName": "Serge","lastName": "Huber”,"userName": "Serge Huber","email": "[email protected]","gender": "male”,"itemClass": "org.oasis_open.context.server.api.User","segments": ["alwaysTrue", "maleGender"]

}}]

}]};

FOR THE JSON ADDICTS LIKE ME


UNOMI ARCHITECTURELOTS OF BOXES

APACHE KARAF

PERSISTENCE IMPLEMENTATION (ELASTICSEARCH, …)

PERSISTENCE SPI (SERVICE PROVIDER INTERFACE)

APACHE CXF

REST API

OSGI SERVICES


UNOMI SERVICESSTUFF THAT ACTUALLY DOES STUFF

PROFILESManages profiles and personas

(loading, updating, saving, deleting, merging, …)

PRIVACYManages privacy (anonymous

browsing, event filtering, profile deletion)

QUERIESPerforms queries against the

stored data (events, profiles, …)

SEGMENTSManages segments (definitions, loading, saving, scoring)

DEFINITIONSManages property types, condition types, actions types, tags and other metadata

CLUSTERINGManages the cluster of context servers


VISITOR PROFILESYOU KNOW ME, I GET IT !

A profile is created as soon as a visitor arrives on a site

Tracked by navigator cookies

Populated through events, such as page views, CMS login, Social login, mobile action, beacon detected or form submissions

Sessions are tracked for each profile, history of navigation is accessible too.


SEGMENTSI’M A PERSON NOT A SEGMENT

Built through

simple UI

Conditions match user

profile properties

Potentially complex

assembly of conditions

Example Profiles younger than 25 and have an income of more than USD 100’000 (young rich segment)


EVENTSI DID WHAT WHEN ?

Client or server-side events can be sent to the context server, in theory in very high volumeThe context server will process the events using plugins, either synchronously or asynchronously (for example using an Apache Mahout BigData analysis cluster)

Example: login event sends additional user data, page view event used to build user or general analytics, mouse over event can be used to build content heat maps

Sample request:GET http://wemicontext.name:8181/eventcollector/view?url=PAGE_URL&referrer=REFERRER_URL (referrer is different for AJAX requests so we use the document.referrer to build this request)


IMPERSONIFICATIONS (PERSONAS)FOR WHEN YOU WISH YOU WERE SOMEONE ELSE

⎯ Persona definition make it possible to “emulate” a certain type of profile, e.g : US visitor, non-US visitor, search engine bot, …

⎯ Predefined personas as well as persona editing UI available

⎯ Jahia provides a side-panel UI to view profile and session data for both the current profile or switch between available personas and see immediate personalization changes


RULESONE SERVER TO RULE THEM ALL

⎯ Executed mostly on event triggers

⎯ Use same conditions are segments to match events

⎯ May perform a list of actions (update profile, session, send email)


RULES, CONTINUEDANOTHER RULE : YOU SHOULD LISTEN TO THE SPEAKER INSTEAD OF READING THESE

⎯ Conditions evaluate to a boolean that will determine if the actions of the rule will be executed or not

⎯ Conditions may be complex, using operators such as and/or for building sub-conditions (condition tree)

⎯ Conditions and actions are either pre-defined or may be extended using Unomi plugins

Structure:when

conditionsthen

actions

Example:when

login event is receivedthen

merge profiles based on login id


PROFILE MERGINGBECAUSE MERGING PEOPLE DIDN’T WORK WELL AS A TITLE

A single (physical) visitor may come to a web site or web service using different devices or browsers


PROFILE MERGINGBECAUSE MERGING PEOPLE DIDN’T WORK WELL AS A TITLE

Each visit will (initially) generate a separate profile ID

Upon visitor identication (usually after login), the current profile will be merge with any existing profiles with the same identifier

Tracking cookies will be modified to associated with the identifier of the merged profile

Recognized visitors will therefore be tracked across devices, as the same profile instead of different ones


PLUGINSNOT THAT TYPE OF PLUG, COME ON

Custom conditions to be used in segments or rules

Custom actions to be used in rules

Custom session or profile property definitions

Resource bundles

Predefined rules

Predefined segments

Predefined scoring

PLUGINS MAY INCLUDEPlugins are OSGi bundles that may extend the built-in functionality of the Context Server


PLUGIN EXAMPLEYEAH SOME MORE JSON !

MAIL ACTION PLUGIN

⎯ Defines a new custom action : send an email

⎯ Provides a Java implementation of the action

⎯ Provides a JSON descriptor to declare the action and it’s parameters to Unomi

⎯ Parameters will be used to build UI to edit the action instance

{"id": "sendMailAction","actionExecutor": "sendMail","tags": [ "event" ],"parameters": [{"id": "from","type": "string","multivalued": false

},{"id": "to","type": "string","multivalued": false

},{"id": "cc","type": "string","multivalued": false

},


PLUGIN EXAMPLEYEAH SOME MORE JSON !

{"id": "bcc","type": "string","multivalued": false

},{"id": "subject","type": "string","multivalued": false

},{"id": "template","type": "string","multivalued": false,

"defaultValue": "Hello $profile.properties.(\"j:firstName\")$ $profile.properties.(\"j:lastName\")$,\nHoware you doing today?"

}

]}


PERSISTENCESAVING THOSE PRECIOUS MOMENTS

⎯ Service Provider Interface

⎯ Default implementation : ElasticSearch

⎯ ElasticSearch provides out of the box :

⎯ NoSQL JSON document storage

⎯ Built-in scaling and clustering

⎯ Separate nodes for data storage & query execution

⎯ Powerful query sub-system

⎯ Apache License


REST APIEXPOSING OURSELVES TO THE OUTSIDE, CLEANLY

FUNCTIONALITIES

⎯ Implemented using Apache CXF

⎯ REST + JSON protocol (no SOAP or other bindings to keep things simple)

⎯ Exposes most of Unomi’s built-in services

⎯ Protected by Karaf container security

⎯ Will be standardized by OASIS Context Server specification

⎯ Can be used to either built administration UI or interface with third party systems


PRIVACY SERVICEYOU KNOW ME A BIT TOO MUCH ?

FUNCTIONALITIES

⎯ Server and event discovery⎯ Current visitor profile access⎯ Retrieving and setting list of events to may or may not be collected for current profile⎯ Management of anonymous browsing, including anonymous collection activation or anonymizing of previously collected data⎯ Profile deletion

3RD PARTY UIS may be built to give end-users control over their data.


ALGORITHMSDISCOVERING (BAD) HABBITS AND BEHAVIORS

This is not yet implemented in Unomi…

The Context Server can also integrate “algorithms”, meaning that data can be processed/mined to extract additional knowledge


ALGORITHMSOUR IDEAS

RECOMMENDATION ENGINEpropose products based on other

SIMILARITY ENGINE

APACHE MAHOUT INTEGRATION

BIG DATA INTEGRATION


ALGORITHMS, CONTINUEDEVEN BETTER TARGETING

Open source reference implementation will also help guarantee interoperability

Algorithms have the potential to make it much easier for integrators and

developers to provide functionality to users of

Unomi

If/When OASIS Context Server specification includes them, the potential for re-usability between implementation could be very interesting


CONNECTORSNO MORE INAPPROPRIATE JOKES PLEASE

CUSTOM CONDITIONSmay be used to

perform dynamicdata exchanges upon incoming

events

can performcalls to external

services

EXTERNAL SERVICES

may use Unomi’spublic URLs or

administration URLsto access REST

services (for querying, reporting,

editing, etc)

EXTERNAL SYSTEMS

may push data intoUnomi either

through events

DIRECT ACCESS

to ElasticSearchis also possible

to directlyinterface with

the persistencesystem

CUSTOM ACTIONS


OASIS CONTEXT SERVER STANDARDAN OVERVIEW WITH LOTS OF COMPETITORS

Co-chairs

Serge Huber (myself) Jahia

Thomas SigdestadEnonic

Established technical committee in April 2015


OASIS CONTEXT SERVER STANDARDMEMBER COMPANIES


OASIS CONTEXT SERVER STANDARDGOOOOOOAAAAALLLLLSSSS !

PROVIDE A STANDARD

§ A context server REST API

§ An associated domain model

§ An open source reference

implementation (Apache Unomi)

APPROVED

Make sure that the

standard is based on real

needs of the industry

DESIGNED FOR GREATNESS

Not only CMS-specific, designed

to integrate with as many tools as

possible (CRM, ERP, mobile

applications, …)


OASIS CONTEXT SERVER STANDARDWORK ORGANIZATION AND STUFF DONE

https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=cxs

Whole TC meets once per month

2 workgroups meet once per week

CURRENT STATUSMEETINGS

⎯ Use case listing & discussions : completion

⎯ Domain model definition : undergoing

⎯ REST API : to be done

⎯ Formal specification : to be done


INCUBATION PROPOSALWHO TO BLAME J

https://wiki.apache.org/incubator/UnomiProposal

ChampionJean-Baptiste Onofré

Talend

MentorBertrand Delacretaz

Adobe

MentorRoman Shaposchnik

Pivotal

MentorChris Mattmann

NASA JPL


CURRENT STATUS & NEXT STEPSTHE FORCE IS STRONG IN THIS ONE

1

2

3

PROPOSAL

to Apache incubator

VOTE

Open right now

ESTABLISH

Incubation project

4

5

6

MOVE CODEBASEFrom Github to Apache

Git repository

GROW COMMUNITY

We need you!

WORK

On first project release


COMMERCIAL APPLICATIONSCOMPLETELY GRATUITOUS PLUG

Build your own ! It’s the beauty of the Apache

License !

First (known) application

Jahia Marketing Factory


TAKING SOME RISKS LIVE

DEMOJAHIA MARKETING FACTORY + UNOMI

AN EXAMPLE APPLICATION


YOUR TURN

HAVE YOU BEEN READING THESE ?

Q & A


LEARN MORE

THANK YOU SO MUCH FOR YOUR ATTENTION !

Incubation proposalhttps://wiki.apache.org/incubator/UnomiProposal

Unomi Github repositoryhttps://github.com/Jahia/unomi

Jahia Marketing Factoryhttps://www.jahia.com/products/marketing-factory

Technology

Apache Unomi In Depth - ApacheCon EU 2015 Session