Utilising Cloud Resources to Gain Knowledge from Social Media

Utilising Cloud Resources to Gain Knowledge from Social Media

Presentation to

Social Media Research Workshop

Wednesday, Feb 5 - Feb 6, 2014

IIIT Bangalore, Electronic City, Bangalore

Professor Gerard Parr

Chair in Telecommunications Engineering

Computer Science Research Institute

University of Ulster

Northern Ireland, UK

[email protected] 1

mailto:[email protected]

My Background

• Professor of Telecommunications Engineering at University of Ulster in

Northern Ireland, UK which is the largest University on the island of Ireland

• PhD in Self Stablizing Protocols- Aspects under Prof Jon Postel (USC-ISI)

DARPA USA

• Invited Member of EPSRC ICT Strategic Advisory Team- UK Gov’t

• Member of Technical Advisory Board at EBTIC , Abu Dhabi, UAE

• Visiting Professor to Trinity College Dublin

• Contributor to NSF in USA on US-UK-India Links

• PhD External Examiner for IIT Delhi , IIT Mumbai and IIT Madras

• Invited Member of EPSRC Centre for Doctoral Training in Communications

Engineering, University of Bristol

• Research Interests include Cloud Computing for Health, Virtualisation,

Smart Cities, Internet Technologies, High-speed Network Management,

Wireless Sensor Networks and ICT for Bridging the Urban Rural Divide

• Lead UK-PI on India-UK Advanced Technology Centre (IU-ATC)

• Co-PI for ESRC Administrative Data Research Centre (ADRC)

• Founding Member of IEEE Intercloud Testbed Initiative Oct 8th 2013 2

So where does BIG data

come from?

This data comes from everywhere including:-

• Sensors used to gather climate information

• Posts to social media sites

• Digital pictures and videos

• Experiments for drug discovery

• Tracking mobile phones

• Streaming NETFLIX Movies

• Manufacturers monitoring equipment

• Financial Services

• Medical data and patient records

• e.gov having services and public data available on the Internet

• Social media ‘noise’

Ref: Cisco et al.

The Growth of Global Internet

Traffic

Worldwide mobile data traffic will increase 18-fold over the next five years.

Mobile Internet will be the main driver - connected devices which will exceed the

number of people on earth- (2016 world population estimate of 7.3 billion: source:

United Nations).

During 2011−2016 Cisco anticipates that global mobile data

traffic will outgrow global fixed data traffic by three times.

In context this is equivalent to:-

33 billion DVDs.

4.3 quadrillion MP3 files (music/audio).

813 quadrillion text messages.

Ref: Cisco® Visual Networking Index Global Mobile Data Traffic Forecast for 2011 to 2016,

Google Project Glass:

Internet enabled!

http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html

Terabyte: A Terabyte is approximately one trillion bytes, or 1,000 Gigabytes. To put it in some

perspective, a Terabyte could hold about 3.6 million 300 Kilobyte images or maybe about 300

hours of good quality video. A Terabyte could hold 1,000 copies of the Encyclopaedia

Britannica. Ten Terabytes could hold the printed collection of the Library of Congress.

Petabyte: A Petabyte is approximately 1,000 Terabytes or one million Gigabytes.

1 Petabyte could hold approximately 20 million 4-door filing cabinets full of text.

It could hold 500 billion pages of standard printed text.

Exabyte: An Exabyte is approximately 1,000 Petabytes.

An Exabyte is approximately one quintillion bytes

or one billion Gigabytes.

There is not much to compare an Exabyte to.

It has been said that 5 Exabytes would be

equal to all of the words ever spoken by mankind.

Zettabyte:

A Zettabyte is approximately 1,000 Exabytes.

A few helpful terms…..

6

Ref: Asigra

The Busy Routes for Digital Traffic

BIG DATA is not just HADOOP (Courtesy of Martin Pavlík- IBM)

Manage & store huge

volume of any data

Hadoop File System

MapReduce

Manage streaming data Stream Computing

Analyze unstructured data Text Analytics Engine

Data Warehousing Structure and control data

Integrate and govern all

data sources

Integration, Data Quality, Security,

Lifecycle Management, MDM

Understand and navigate

federated big data sources Federated Discovery and Navigation

9

Supported by Research Councils UK(EPSRC) and Department of Science and Technology(DST), Government of

India

Professor Gerard Parr UK- PI of the IU-ATC

Chair in Telecommunications Engineering University of Ulster, Coleraine, Northern Ireland, UK

[email protected] http://www.iu-atc.com

10


http://www.iu-atc.com/




What is the IU-ATC?

The IU-ATC (India-UK Advanced Technology Centre) of Excellence in Next Generation Networks, Systems and Services is a joint multi-million pound research initiative funded by the British and Indian Governments which will put in place a focused agenda to support collaborative PhD, Post Doctorate projects and joint fundamental research programmes and technology transfer between the UK and India. Phase One of the Centre officially started in June 2009 with total funding of £9.2 million and funding for Phase two was announced at the UK-India Science and Innovation Council meeting in a joint statement during April 2012 with total funding of just over £10 million: It is an internationally leading resource for UK and Indian Research Institutions and ICT Companies to collaborate in support of the future Global Information Economies. .

11

• The IU-ATC is structured on three Integrated Research Groups comprising of thirteen Cross-layer Research Work packages .

• These inter-related work packages were

identified after detailed technical discussions at various consortium workshops to ensure our programme of joint research between UK and Indian scientists is internationally leading and competitive.

• We also wish to contribute to the further

development and deployment of Next Generation Converged Networks in support of affordable applications.

• These Work packages build on the strength

of our consortium members and the successful outcomes of IU-ATC Phase One.

• They provide the greatest opportunities for the scoping of technology demonstrators that will underpin the development of policies and initiatives for both the rural and urban Digital Information Economy in India and the UK. 12

13

14

Group 1: Services and Applications

This group aims to develop a number of service demonstrators, with focus on novel applications to meet the requirements in rural India and remote areas of the UK. The services must be scalable and have the necessary user interfaces to cater for different user needs.

Services and Applications

WP 1: e-Agriculture WP 2: e-Health ICT - Towards Managing Healthcare Delivery

WP 3: Collaborative

Learning

WP4: Internet of Things (IoT) - e-Health

WP 5: Provisioning and Delivery of

Emergency Services 15

Exemplar project- Cloud Based Crop

Disease Detection I. Pest Disease Image Upload

application (PDIU)

Mobile application allows

farmers to take photos of

affected crops and upload

them, making them

available to the expert

II. Advisories related to pest and

disease

Using existing call centre set-up, personalized advisories

regarding pest and diseases were given to farmers

III. Automatic colour correction

(University of Bristol/University of Ulster) Demonstrated simple disease classification/recognition in plant images

captured in controlled environments. The research produced a number

of approaches for colour correction/normalisation and leaf segmentation

to aid recognition.

16

Crop-Imaging on the India-UK Test-bed

17


– Provisioning and Delivery of Emergency Services

WP5 focuses on developing a scalable and robust Early Warning System for natural and manmade disasters, leveraging existing communication infrastructure and taking into account the vast differences in end users’ device capabilities.

The Early Warning System uses a diverse set of delivery mechanisms including SMS, CBS and IP messaging over the available access networks to disseminate warnings, with context-aware adaptation of content and user interface.

18

ADAPTIVE MULTIMODAL INTERFACES


We follow a user centred inclusive design approach that involves end-users in all stages of development ensuring that the final product is usable and enjoyable for users with a wide range of abilities. We have developed an Inclusive User Model that simulates users’ interaction patterns and we deploy this to personalize electronic interfaces for both health, physical and situational impairments

We have conducted an extensive user survey in both UK and India to understand the objective range of abilities and the subjective attitudes towards technology of elderly users.

19


ADAPTIVE MULTIMODAL INTERFACES

We interviewed key First Responder and governmental stake-holders for the design of a resilient wireless disaster management system, that is contributing towards an inclusive disaster early warning and rescue management system to be trialled in UK and India.

20

Deriving Real-Time Value.. A new proposal under development

Global First Responder System for Man-made or Natural Disasters

Integrating, Fusing and Extracting

Useful Intelligence from Social Media

Real-time Feeds

21

Group 2: Core Network Systems

Core Fixed and Wireless Infrastructure with Intelligent Monitoring and Dynamic Resource Allocation

22


WP1: Resource Allocation for Wireless Access to the Cloud

With recent developments in cloud services, Group 2 WP1 will aim to develop a framework that will dynamically expose and scale cloud resources used in delivering mobile services. Our work will propose and develop a “Cloud Resource Broker” that interfaces the mobile environment to existing cloud management platforms for dynamic resource allocation via programmable interfaces. This will enable the deployment of cloud resources at the edge of the mobile network. Why is the support for mobile environments important? IoT services are introducing more intelligent mobile devices with critical requirements – i.e. healthcare and security services. Efficient use of mobile resources such as spectrum, battery power and location services. Optimisation of applications by using techniques such as edge caching and opportunistic networks. The ability to offload cloud resources to the edge of the network.

23


WP2: Cloud Aware Transport Protocols and Adaptation

Users consume a range of applications and data over WLAN using a variety of devices. The demand for bandwidth varies according to the user’s needs. New research focuses on assigning the appropriate 802.11 protocol to a connecting device, according to its requirements. This dynamic and intelligent protocol selection should increase energy efficiency while assuring adequate QoS for users. This research is built upon a test-bed consisting of a range of fixed and wireless technologies with provision for inter-platform operability.

Cloud Edge Node

Wireless Devices

802.11g

802.11g

802.11n

Internet

To Central Data Centre

App0

App1

App2

24

WP3: WiCAP – Wireless Cloud Application Proxy


Mobile networks currently play a key role in the evolution of the Internet due to exponential increase in demand for Internet-enabled mobile devices and applications. This has led to various demands to re-think basic designs of the current Internet architecture, investigating new and innovative ways in which key functionalities such as end-to-end connectivity, mobility, security, cloud services and future requirements can be added to its foundational core design.

A data centre scaling out resources to the Wireless Application Cloud Proxy

In this work package, we investigate, propose and design a functional element, known as the mobile cloud proxy, that enables the seamless integration and extension of core cloud services on the public Internet into mobile networks. The mobile cloud proxy function addresses current limitations in the deployment of cloud services in mobile networks tackling limitations such as dynamic resource allocation, transport protocols, application caching and security. This is achieved by leveraging advances in software-defined radios (SDRs) and networks (SDNs) to dynamically interface key functions within the mobile and Internet domains.

25

Group 3: Heterogeneous Wireless Access Networks

Heterogeneous Wireless Access

Networks

WP 1: PHY layer technologies for

4G+

WP 2: Radio Resource Management (RRM) in

Future HETNETS

WP4: Enhancement of the Wireless Testbed

WP 3: Cognitive Radio and Future

Spectrum

Energy Efficient Heterogeneous and Cognitive Access Networks This work area aims to investigate and develop effective techniques to provide low-cost and ubiquitous wireless access networks, particularly, to rural communities as the enabling technologies for green and sustainable economies in both UK and India. The technical approach in this project focuses on exploitation of self-organised heterogeneous and cognitive techniques in designing both physical layer and upper layers of wireless communications systems.

26

New UK International Connectivity and

Research Infrastructure

Supporting BIG Data and Cloud Services

Globally

“PROJECT KELVIN”


Scientific Advisor to UK and Irish Governments

Department of Enterprise Trade and Investment (DETI) in Northern

Ireland and the Minister for Communications, Energy and Natural

Resources (MCENR) in Ireland

Natural Catastrophes? Contingency Plan?

Operating in 125 POPs across 50 Markets in North America, Europe and

Asia

26,000 km of owned terrestrial and submarine fibre assets

with leases on an additional 80,000 km worldwide

Sales come from high growth segments, specifically from three

primary vertical solutions

Wholesale Capacity

Financial Network Services

Media Network Services

Hibernia’s network is:

– Utilised by all of the top 20 largest carriers in the world

– Connected to all of the top 10 largest financial exchanges

– Transporting live video programming regularly viewed

by an estimated 20M people per day

Landing Station Locations

29

The Cable Landing Station at

Coleraine…. “our TARDIS”

Time and Relative Dimension in Space (and Data)!

£millions of equipment carrying

£billions of data

Top-level Hibernia Network

Map

• 1.92Tbit/s Northern Ireland to US & Canada Coast- currently under

upgrade

• 7.68Tbit/s Northern Ireland to European Coast

• 13 x Carrier Neutral Points of Presence (POPs) on Island of Ireland

• 52ms from Coleraine to Halifax (Nova Scotia)- lowest latency!

USPs for KELVIN Link in Northern Ireland

To put in context…

Using the 100Gigabit/sec capabilities the KELVIN

Links could carry :

979,584 simultaneous HD videos

600,000,000 simultaneous SKYPE calls

….and the link is getting faster!!!

Ref: Hibernia Networks

27th March 2013

International Links (Project Kelvin)

34

International Connectivity

KELVIN provides the fastest link across the Atlantic Ocean:

Excellent for High frequency Real-time Trading, large-scale multi-player

Cloud Gaming, real-time TV streaming or analysing oil/gas exploration data.

Key Customer Segments

Ma

nag

ed

Se

rvic

es

Govern

ment

Te

lco

s

E-C

om

merc

e

Me

dia

&

Bro

ad

ca

st

Fin

an

cia

l

Se

rvic

es

En

terp

ris

e a

nd

Wh

ole

sa

le

Department of Communications, Energy and Natural Resources

Roinn Cumarsáide, Fuinnimh agus Acmhainní Nádúrtha

36

• Linkage service • Safe setting • Safe-setting support • Data acquisition

• Public engagement • Training, capacity building • Research

The Northern Ireland ESRC Administrative Data Research Centre

37

Providing the tools and techniques to give the wider public (citizens, agencies and companies) access to the data they paid government to Collate in the Cloud!!!

38

IT as a Utility funded under Research Council UK's Digital Economy theme

The Digital Economy vision is of the transformational impact of digital technologies on all aspects of life.

39

• IT as a Utility Network+ – 1 of 4 networks supported by RCUK Digital

Economy Theme

• Aim: better understanding the benefits and opportunities afforded by the digital economy

• Objective: foster collaboration between academia, business and policy-making bodies

40

Partners & collaborators

• PI and Co-Is:

Jeremy Frey ( Uni Southampton)

Gerard Parr (Ulster)

Mark Sandler (QMUL)

Richard Mortier ( Uni Nottingham)

Mike Surridge (Uni Southampton)

• Advisory group: – ustwo

– IBM

– BT

– Thales

– Microsoft

– BBC Reseaerch

– Cabinet Office

– DSTL

– Zenotech

– Cambridge & Newcastle Universities

Steve Brewer – Network Coordinator

41

The Digital Economy and ITaaU

The Cloud

food security

traditional

utilities

cloud computing

sensors &

actuators

telecommunications

data-driven

science electronic lab

notebooks

“Network+

smart spaces/ smart cities”

libraries of

the future

“tangible

Interfaces”

“apps are the

new taps…”

3D printing

42

Workshop themes include:- • libraries of the future

• emerging economies,

• user interaction design

• trust and security

• Food Security

• Smart Environments

• Diversity

• Massive Data Analytics

43

First two successful pilot projects have now been running for a

while:

• Trusted Tiny Things – led from University of Aberdeen

• Using Wireless Networks to Support First Responders

and Resilience in Upland Areas – led from QMUL &

Cambridge

44

Pilot Projects about to start • Typology of Loss in Vaccine Supply (TOLIVS)

– Deploy pilot cloud service/mobile sensing in the field for vaccine tracking. Particularly typology of drug loss in the cold supply chain

• CloudMaker – A utility to support social creativity between children

• BluPoint – Provision of digital content as a utility in low-resourced off-grid

communities

• Sun & Sky – A sun and sky environmental monitoring system for crowd sourcing

• Communities in the Cloud – Technology to support high-density/high/rise communities

45

Secondment Projects being arranged

• Exploring social knowledge integration with digital mapping technologies to benefit communities

• Integration of IoT and M2M technologies for the weightless network. Investigation of challenging propagation environments. Workshop planned.

• Social pedestrian modelling of Clapham Junction and London Bridge Station.

• Story as Utility: how can HDI unleash ITaaU research in the wild?

46

Where does this lead us? • Data lies at the heart of the digital economy

• This has many implications for the future

• Big data

– huge quantities of homogenous, heterogeneous and disparate – new mathematics needed

• Security and trust issues will pervade

• Design matters – democratization of data and its by products: information and knowledge

47

(some of the team!)

The RCUK DE “IT as a Utility”

Project would like to explore

how we might help

with the follow-up

to this Bangalore workshop.

Please let me know

what we can do..!


UK Lead PI for India-UK CoE

University of Ulster, UK

[email protected]

Thanks for your attention
