1

SERENDIPITI SEnsoR ENricheD

Information Prediction and InTegratIon

2

the effect by which one accidentally discovers something fortunate, especially while looking for something entirely unrelated. (wikipedia.org)

Serendipity n.Scoperta (Italian)Serendipiteit (Dutch)Vrozené štěstí (Czech)

/ˌsɛ.rɛn.ˈdɪp.ə.ti/

3

Today’s Agenda

VisionObjectivesTechnology

Partners

4

Vision What happens when you aggregate partial

observations?

• Maps and ship captains– No single human has been to

every point on a map– Cartographers resolved partial

observations from ship captains– Many needed, potentially

conflicting– Slowly, there emerged a map of

the world• Can we do something similar

to learn something new about our cities?

5

Objectives• Aim? Stimulate and integrate research groups

from currently fragmented research areas, creating synergies at the cross-over points.

• Approach? By fostering long-term relationships between research groups, based around people, and laying the foundations for a Virtual Centre of Excellence (VCE).

• In what areas? Real-time, large-scale data analysis and inference for fusing semantic information and predicting events.

• How? By harvesting, mining, correlating and clustering extremely large, highly dynamic, very noisy, contradictory and incomplete information from multiple sources including: tweets, logs, RSS, web-sources, mobile texts, web-cams, CCTV and other publicly-available multimedia archives.

6

What’s New in SERENDIPITI?• Real-time• Prediction (events)• Multimodal• Noisy, errorsome data• Novel applications

7

How do we do this?• Sensing

– Aggregate diverse sources of information from the real and online worlds

• Analysis– Extract stable spatio-temporal patterns of

human activity – Track these over time

• Use Case Scenarios– City Planning– Journalist (e.g. see Appendix)– Police

8

Physical

Online

How do we do this?

SensorBase

9

How do we do this?

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online

10

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online Machine learning and data mining

How do we do this?

11

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online Machine learning and data mining

How do we do this?

SERENDIPITIPlatform

12

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online Machine learning and data mining

How do we do this?

SERENDIPITIPlatform

Planning (City)

Journalist

Police

Use Case Scenarios

13

Partners & RolesPatricia Ho-Hune

Alan Smeaton, Noel O’Connor, Barry Smyth

Giovanni Tummarello, John Breslin, Paul Buitelaar

Keith van Rijsbergen, Joemon Jose, Mark Girolami

Ebroul Izquierdo

Maarten de Rijke, Arnold Smeulders

Vojtech Svatek

ERCIMDCU-

CLARITYDERIGLAQMULUvAUEP

0

1

2

3

4

5

6

14

Partners & RolesEU Project Management

Analysis of multimodal sensor data, real-time integration of physical & online sources, organisation & management of online sourcesSemantic text analysis/mining, large-scale semantic search/indexing, linked data, social semantics, online communitiesMultimedia information retrieval, formal models, mining information from large data sets, event detection, information fusion, machine learningCorrelating/mining media and textual data, sensor base, automatic (CCTV-based) event analysis

Focused crawling, wrapper induction, mining social media, information extraction, data integration, cross-media mining and information fusion, machine learning

Mining rich associations from large databases, information extraction from the Web, semantic and social Web technology, effectiveness of ICT

ERCIMDCU-

CLARITYDERIGLAQMULUvAUEP

0

1

2

3

4

5

6

15

1 2 3 4 5 6DCU QMULDERI GLA UvA UEP

PARTNERS

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online Machine learning and data mining

SERENDIPITIPlatform

Planning (City)

Journalist

Police

Use Case Scenarios

16

1 2 3 4 5 6DCU QMULDERI GLA UvA UEP

PARTNERS

Track evolution of events over space

and time

SensorBase

- Crowd movements, CCTV- Traffic - pedestrian / vehicular- Bluetooth sensing & proximity

- Blogs, wikis, web feeds- Tweets, RSS- Event guides

Physical

Online Machine learning and data mining

SERENDIPITIPlatform

Planning (City)

Journalist

Police

Use Case Scenarios134

546

3 5

2 36

12

56

123456

17

Work Packages• WP1 – Management• WP2 – Integration of Organisation & People

• WP6 – Applications & Infrastructure Sharing• WP7 – Outreach (Spreading Excellence)

Joint Program of Activities• WP3 – Real-time Large-scale Data Analysis• WP4 – Semantic Information Fusion• WP5 – Inference & Event Prediction

18

Implementation

Planning (City)

Journalist

Police

User Group Data Provision Board

Industrial Advisory Board

Wikimedia FoundationBoards.ieTWS+VEPA/MI/Met

Ken WoodMilek BoverE. AartsCarto RattiOne other !

19

Tangible Outputs• Provision of mini projects to tackle

unforeseen research topics• Industrial placements of research staff• Academic exchanges• Contribution to standardisation efforts• Public software repositories and

tools/data access through the SERENDIPITI platform

• Outreach to other targeted projects

20

Impact• SERENDIPITI : “SEnsoR ENricheD Information Prediction and InTegratIon”

• People - Research - Outreach - Platform

• Large-scale, semantic urban computing