Big Data - Where from Where to

Hanmin JungHead of the Dept. of Computer Intelligence Research

Big Data:Where from? Where to?

� Very Recent Activities on Big Data

� (National Science and Technology Commission) Member of Big Data Technical Impact Assessment Committee

� (Korea Communications Commission) Sub-committee Chair of Big Data Forum

� (Ministry of Knowledge Economy) Technical Secretary of Big Data Program Planning Committee

� (Ministry of Educational Science and Technology) Member of Big Data Information Strategic Program Expert Committee

� (National IT Industry Promotion Agency) Lecturer of Big Data Expertise Reinforcement Program

Let Me Introduce Myself :-)

Questions

Where are Big Data from?

Who gathers and consumes the data?

Is the data used for?

Smart Work

http://files.thinkpool.com/files/bbs/2010/07/21/%EC%8A%A4%EB%A7%88%ED%8A%B8%EC%9B%8C%ED%81%AC1.jpg

Cloud Computing

� Service Platform Accelerated by Mobile Devices

http://simpleroot.com/wp-content/uploads/2012/10/Remote-Cloud-Computing.jpg

Cloud Computing – 建建建建て前前前前& 本音本音本音本音

� Introducing iCloud

Cloud Computing

� Google Data Center

http://www.youtube.com/watch?v=avP5d16wEp0

Data Sources

Web -> Social -> Thing

“The next Google or Facebook may well bean Internet of Things company.”by R. MacManus (ReadWriteWeb)

Social Data

http://bynoy.files.wordpress.com/2011/08/united-noy-weblife-60-seconds.jpg

Machine Data

T. Baer, “What is Big Data? The Reality for Analytics”, OVUM, 2011.

Call data recordsCall data records

Sensory dataSensory data

Web log filesWeb log files

Financial Instrument TradeFinancial Instrument Trade

Internet of Things

K. Escherich, “Internet of Things”, 2011.

Big Data in the World

http://www.ektron.com/billcavablog/Big-Data-Big-Content-Big-Challenges/

Infographics for Big Data

http://thumbnails.visually.netdna-cdn.com/big-data_50291c3b16257.jpg

Google.com Traffic

http://siteanalytics.compete.com/naver.com/

Naver.com Traffic

http://siteanalytics.compete.com/naver.com/

Foreseeable Future

� Google Project Glass

Hype Cycle

Hype Cycle – 2010

Emerging Technologies Hype Cycle 2010

Hype Cycle – 2011

Hype Cycle – 2012

Google Insights

http://www.google.com/insights/search/

Bottleneck in Data Ecosystem

http://quizzicaleyebrow.files.wordpress.com/2011/03/pict0044.jpg

Big Data Ecosystem

http://imexresearch.com/Newsletter_HTML/bd2.png

Big Data Ecosystem

� New Approaches Required for

� Persistence

� Indexing

� Caching and query optimization

� Processing

� Structure

� Query language

� Compression

24T. Baer, “What is Big Data? The Reality for Analytics”, OVUM, 2011.

Insights for Search

http://www.google.com/insights/search/

Mobile Phone

� Worldwide Market Share

� Worldwide mobile device sales to end users in 2008 ~ 2012

Gartner, IDC Worldwide Mobile Phone Tracker

4.0, 14.14.3, 17.19.9, 47.8Apple

7.5, 23.011.0, 31.68.1, 28.45.4, 21.1LG

3.3, 15.8Huawei

Company4Q2012

(%, M. Units)3Q2011

(%, M. Units)3Q2010

(%, M. Units)3Q2009

(%, M. Units)3Q2008

(%, M. Units)

Nokia 17.9, 86.3 27.1, 106.6 31.6,110.4 37.8, 108.5 38.6, 117.9

Samsung 23.0, 111.2 22.3, 87.8 20.5, 71.4 21.0, 60.2 17.0, 52.0

ZTE 3.6, 17.6 4.9, 19.1 3.5, 12.1

Sony Ericsson 4.9, 14.1 8.4, 25.7

Motorola 4.7, 13.6 8.3, 25.4

Others 42.3, 203.8 36.1, 142 32.2, 112.5 20.6, 59.1 20.1, 61.5

Total 482.5 393.7 348.9 287.1 305.4

CDC Influenza Summary

http://www.cdc.gov/flu/weekly/usmap.htm

Google Flu Trends

J. Ginsberg, “Detecting influenza epidemics using search engine query data”

Voice Search Evaluation

http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en//pubs/archive/40491.pdf

Causes of Death

http://image.guardian.co.uk/sys-files/Guardian/documents/2011/10/28/Factfile_deaths_2_2011.pdf

IBM Watson

http://powet.tv/powetblog/wp-content/uploads/2011/02/watson_the_computer_beats_ken_jennings_and_brad_rutter_at_jeopardy_full.jpg

Search

Clustering

Extracting

DecisionSupport

Forecasting

ScenarioPlanning

Advising

Modified from D. Bousfield & P. Fooladi, “STM Information: 2009 Final Market Size and Share Report”, 2010.

Value Pyramid

InSciTe Advanced (2011)

InSciTe Adaptive (2012)

OntoFrame (2005~2009)

Big Data & Decision Making

http://lithosphere.lithium.com/t5/Lithium-s-View/Big-Data-Analytics-Reducing-Zettabytes-of-Data-Down-to-a-Few/ba-p/36378

� Reducing Zettabytes of Data Down to a Few Bits

Data help us make better decisions.

The primary function of analytics is to support decision making.

The challenge of big data analytics isto reduce a lot of data down to a few bits.

Strategic Foresight

R. Rohrbeck, H. Arnold, and J. Heuer, “Strategic Foresight in Multimedia Enterprises”, 2007.

Quantitative Analytics

TI Projects

� FUSE

� Funded by IARPA (early 2011 ~ early 2016)

� Kick off meeting in summer, 2011

� Foresight and Understanding from Scientific Exposition Program

� Seeks to develop automated methods that aid in the systematic, continuous, and comprehensive assessment of technical emergence using information found in the published scientific, technical, and patent literature

� Partners

� BAE Systems, Brandeis Univ., New York Univ., 1790 Analytics, …

TI Projects

� FUSE

TI Projects

� CUBIST

� Funded by the European Commission (late 2010 ~ late 2013)

� 1st CUBIST workshop in July, 2011

� Combining and Uniting Business Intelligence with Semantic TechnologiesProgram

� Aims to develop new ways to interrogate not only the massive volume data on the Internet, but also analyze the different formats it exist in – such as blogs, wikis, and video

� Partners

� SAP, Ontotext, Sheffield Hallam Univ., …

TI Projects

� CUBIST

TI Projects

� Common Technologies

� Semantic technologies

� Ontology, reasoning, URI scheme

� Analytics model

� BYOM (e.g. technology opportunity discovery model, technology evolution model, formal concept analysis model)

� Information extraction (InSciTe, FUSE)

� Named entities and events/relations in textual documents

Our Vision & Architecture

InSciTe Adaptive (2012)

Data Fact Sheet

� InSciTe Adaptive (2012)

� Articles: 22.6 millions (9.8 millions for papers, 7.6 millions for patents, 5.3 millions for Web data)

� All technical areas (2001~2011)

� Named entities: 1.9 millions

� Authority dictionary: 1.5 millions entries

� LOD data: 290 GB (are being connected)

Supporting Decision Making

http://4.bp.blogspot.com/-Pf1hkccZZh4/TWDJahBpL2I/AAAAAAAAASU/JHLpXi8d9AQ/s640/meetings.jpg

Data Scientist

http://philanthropy.com/blogs/innovation/matching-data-scientists-and-nonprofits/778

Evidence-based Decision Making

� Advantages

� Ensures that policies are responding to the real needs of the community

� Highlight the urgency of an issue or problem which requires immediate attention

� Enables information sharing amongst other members of the public sector

� Reduces government expenditure which may otherwise be directed into ineffective policies or programs

� Produces an acceptable return on the financial investment that is allocated toward public programs

� Ensures that decisions are made in a way that is consistent with our democratic and political processes which are characterized by transparency and accountability

http://www.abs.gov.au/ausstats/abs@.nsf/lookup/1500.0chapter32010

InSciTe Project

http://semantics.kisti.re.kr

Thank you

jhm@kisti.re.kr

“A lot of times, people don’t know what they want until you show it to them.”

by Steve Jobs

“Many people won’t be convinced until they’ve seen it for themselves.”

by Jakob Nielsen

Big Data - Where from Where to

Technology

Where is my big data: security, privacy and jurisdictions in the cloud

Big Data is Every Where - University Of Illinoispublish.illinois.edu/frontiers-big-data-symposium/files/2016/10/CarleyUIUCslides.pdfTopic Networks in Twitter Data During the Arab Spring”

GPU accelerated Cray XC systems: Where HPC meets Big Data

Big Data & Analytics Innovationie.theinnovationenterprise.com/eb/BigDataLondon-Brochure2015.pdf · was a Data Scientist at Feedzai, where he performed R&D in big data and machine

Big Data is changing abruptly, and where it is likely heading

BIG DATA EN SANTÈ KALRA - Big Data en santè.… · Big Health Data: where are the opportunities? ... Genomics, proteomics, metabolomics... Advanced mathematical modelling Provide

A holistic approach to Big Data · 19/09/2013 · Big Data – A holistic approach Big Data is Not Only Hadoop! Examples where Hadoop is not entirely applicable: – Cyber security,

Show me the data. Where big data meets PR #PRshow13

Where Linked Data meets Big Data: Applying standard data models to environmental data streams

“Big Data in Marketing: Where Do We Go From Here?”cdwcmojo.com/pdfs/big-data-big-brains/gartner-presentation-04-07... · “Big Data in Marketing: Where Do We Go From ... DESIGN

With the Emergence of Big Data, Where do Relational ... · With the Emergence of Big Data, Where do Relational Technologies Fit? VP Product Marketing CA Technologies DAMA Chicago

DATA-POP ALLIANCE and the Sustainabledatapopalliance.org/wp-content/uploads/2016/03/... · with and by Big Data—where Big Data here is used not as big data alone, nor as big data

Big Data Spatial Analytics – An Introduction · Geospatial in Big Data • Big Data with location in Enterprise Big Data warehouse • Run spatial queries on data where it resides

BIG DATA TRANSFORMS BUSINESS - Dell EMC Areas Where Companies Are Using Big Data How Companies Are Using Big Data McKinsey Global Survey of 1,469 C-level executive respondents at a

Big Data - INCITS · 2015-04-15 · Big Data Soutions Big Data Solutions are those which address problems in data analytics where the Volume, Velocity, or Variability of the source

Internet of Things: Where Fast Data Meets Big Data: Couchbase Connect 2015

Big Data Roundtable. Why, how, where, which, and when to start doing Big Data

MALAYSIA’S NATIONAL BIG DATA ANALYTICS …calabarzon.neda.gov.ph/.../2016/10/02-Big-Data-Analytics-MDEC.pdf · BIG DATA ANALYTICS (BDA) ... and eco-system where hype is separated

Next Generation Credit Decisions: Where big data outperforms the credit bureaus

With the Emergence of Big Data, Where do Relational Technologies Fit? · PDF file · 2016-10-30With the Emergence of Big Data, Where do Relational Technologies Fit? Donna Burbank