NoSQL Technology and Real-time, Accurate Predictive Analytics

www.Objectivity.com

© Objectivity Inc 2013

Welcome!

Webinar: Big Data – NoSQL Technology and Real-time,

Accurate Predictive Analytics


Agenda

Market Overview• Presented by Matt Aslett, Research Director at 451 Group

Big Data Use Case• Presented by J.C. Smart, Director Global Insight Laboratory at Georgetown

University

Q&A• Presented by

• Matt Aslett, Research Director at 451 Group• J.C. Smart, Director Global Insight Laboratory at Georgetown University• Leon Guzenda, Founder at Objectivty, Inc.

© 2013 by The 451 Group. All rights reserved

Matthew Aslett• Research Director, Data Management and Analytics [email protected] www.twitter.com/maslett

Responsible for data management and analytics research agenda

Focus on operational and analytic databases, including NoSQL, NewSQL, and Hadoop

With 451 Research since 2007


Company Overview

One company with 3 operating divisions

Syndicated research, advisory, professional services, datacenter certification, and events

Global focus

200+ staff 1,300+ client organizations:

enterprises, vendors, service providers, and investment firms

Organic and growth through acquisition


Unique combination of research, analysis & data

Emerging tech market segment focus

Daily qualitative & quantitative insight

Analyst advisory & Go-to-market support

Global events


What has driven the development and adoption of NoSQL?

NoSQL, NewSQL and Beyond• Assessing the drivers behind the development and

adoption of NoSQL and NewSQL databases, as well as data grid/caching technologies• Released April 2011• Role of open source in driving innovation• [email protected]

MySQL vs NoSQL and NewSQL• Released May 2012

Next-generation Operational Databases• Released July 2013


SPRAINED RELATIONAL DATABASES

Photo credit: Foxtongue on Flickrhttp://www.flickr.com/photos/foxtongue/4844016087/


Database SPRAIN

The traditional relational database has been stretched beyond its normal capacity by the needs of high-volume, highly distributed or highly complex applications.

There are workarounds – such as DIY sharding – but manual, homegrown efforts can result in database administrators being stretched beyond their normal capacity in terms of managing complexity.

Scalability Performance Relaxed consistency Increased willingness to look towards Agility emerging alternatives Intricacy Necessity


Necessity is the mother of NoSQL

Hadoop and NoSQL innovation did not come from existing relational database and storage suppliers

It came from Google, Amazon, Facebook, Yahoo, LinkedIn and open source communities…

This has significantly altered the relationship between customer and vendor, and changed the database landscape enormously

And also generated a new breed of database vendors and database products

“We couldn’t bet the company on other companies building the answer for us.”

– Werner Vogels, Amazon CTO


The NoSQL database landscape

Wide-column stores

Data is mapped by a row key, column key and time stamp.

Key Value Stores

Store keys and associated values.

Graph databases

Store data and the relationships between data.

Document stores

Store all data related to a specific key as a single document.

DATA MODEL COMPLEXITY



Wide-column stores

Data is mapped by a row key, column key and time stamp.

Key Value Stores

Store keys and associated values.

Graph databases


Document stores

Store all data related to a specific key as a single document.

Multi-model databases

Support a combination of the various individual NoSQL data models.

DATA MODEL COMPLEXITY



Graph databases not only store data in a collection of key-value pairs, known as nodes and properties, but also store the relationships – or edges – that connect nodes to other nodes, or nodes to properties.

Users can navigate – or traverse – the resulting graph by nodes, properties or edges to identify and analyze relationships between nodes and properties.

This is inherently more flexible than traditional approaches that would require cross-table joins in relational databases.

Graph databases




Graph databases are more than just a new way of storing data

Graph databases enable analysis of not just individual or aggregate data, but also the relationships between data

Graph databases potentially provide new opportunities for generating business intelligence by highlighting new patterns in data

Graph databases



Graph analytics

The rise of graph databases is closely linked to the rise of social networking

It could be argued that the most valuable assets that Facebook, Twitter and LinkedIn own are the graphs that represent the relationships between their users and their users’ interests

However, the roots of graph analytics can be traced back much further, all the way to Leonhard Euler’s Seven Bridges of Königsberg, published in 1736

Graph databases



Seven Bridges of Königsberg (now Kaliningrad)

Find a route crossing each bridge once, and only one• Euler proved there was no solution

Source: Wikipedia http://en.wikipedia.org/wiki/File:Konigsberg_bridges.png


Seven Bridges of Königsberg (now Kaliningrad)

Relevance today:• Google uses graph theory to find the most efficient routes for Street

View cars to capture images for Google Maps


Other applications

Less obvious applications include customer management• E.g. Financial services firm with multiple business units

PARENT CO

LOANBANKING

CHECKING CREDIT CARD

INSURANCE PENSION

HOUSE INSURANCE CAR INSURANCE


Other applications

Less obvious applications include customer management• E.g. Financial services firm with multiple business units• What happens when an individual has multiple customer relationships?

PARENT CO

LOANBANKING


INSURANCE PENSION



Other applications

Less obvious applications include customer management• E.g. Financial services firm with multiple business units• What happens when an individual has multiple customer relationships?• Graph analysis to identify multiple services related to an individual

PARENT CO

LOANBANKING


INSURANCE PENSION



Other applications

Less obvious applications include customer management• E.g. Financial services firm with multiple business units• What happens when an individual has multiple customer relationships?• Graph analysis to identify multiple services related to an individual• And provide a customer-centric relationship perspective

CUSTOMER

PENSIONLOANCHECKING HOUSE INSURANCE


Exploratory analysis/discovery

While BI involves analyzing data for answers to existing questions, exploratory analytics/discovery involves exploring patterns in data to prompt new questions

This search for patterns requires a platform that offers more flexibility than the schema-on-write approach of the EDW and traditional analytics• Statistical analytics• Predictive analytics• Machine learning

The search for patterns also lends itself to analyzing not just data, but relationships between data• Graph analysis


Conclusion

NoSQL development was driven by the need for new approaches to scalability, performance, consistency, agility and intricacy

Initiated by Web startups, it has generated a new breed of database vendors and database products

Graph databases enable analysis of not just individual or aggregate data, but also the relationships between data

While the rise of graph databases is closely linked to the rise of social networking, use-cases include anything that involves relationships between entities

Graph databases are expanding the market for analytics


Questions? [email protected]@maslett


Big Data Use Case: Georgetown University

J. C. Smart, Ph.D.Georgetown University

August 2013

Global Insight

The world is an important place…...and it has a few problems

7 billion people, 40,000 cities, 5 billion cell phones, 800 million vehicles, 12 million miles of paved roads, 50,000 airports, ...

The world is a complex system ofinterdependent complex systems

Climate Population Political Energy

Social Poverty Transportation Trade

Communications Terrorism Crime Health

There is an enormous diversity of topics,scales, fidelity, time, duration, …

Geospatial, cyberspatial, real-time, historical, predictive, hypothetical, virtual, on and on….

Data exists in many different forms….

Real-time Feeds Applications Databases Spreadsheets

Files Photos Audio Sensors

Websites Models Systems Plans/Maps

The “High-Yield” Knowledge Phenomena

Knowledge Density

(#Related Facts / Domain)

High-YieldPotential

Low-YieldPotential

?

Information Inferiority Information Superiority

“Anything,Anytime,

Anywhere”

“Some things,Some of the time,

Somewhere”

IntelligenceSaturation

Knowledge Gap

“Critical Mass”

IntelligenceStarvation

An

alyt

ic P

oten

t ial

/

An

alyt

ic Y

iel d

04/11/2023

Why is “connecting-the-dots” so hard?

• Plumbing: Massive logistics problem to integrate thousands of government/non-government data systems at scale

Different standards, models, security, infrastructure, procedures, policies, networks, access, compartments, applications, tools, protocols, etc. … all at immense scale!

• Protection: Large-scale integration of data resources increases cyber security risks

Prevention of adversary exploitation of strategic national assets.

• Patterns: Lack of analytic algorithm techniques to automatically detect data patterns and alert

Transition from “analytic dumpster diving” to early-warning indication and real-time notification

• Privacy: Significant tension between security and libertyWho trusts the “watchers”?Who watches the watchers?

04/11/2023

The FOUR-Color FrameworkOverview

Black Layer

Black Layer

Analytic

AnalyticKnowledge Space

Analytic

Analytic

Analytic

Analytic

Analytic

Analytic

AnalyticEngine

AnalyticEngine

AnalyticEngine

AnalyticEngine

API

API

API

API

Global insight is now possible!

• Techniques derived from innovations at LLNL, DoD, Raytheon, Georgetown, [many others] – enabled by HPC

• Extremely powerful, very effective, not for the timid

• Represents global systems as trillions of interacting objects

• Scaling, privacy, and protection achieved through a unique data to information transformation (overlay) technique

04/11/2023


Q&A

A copy of the webinar including QA will be available online at www.Objectivity.com.

A follow up email incorporating answers to questions that may not have been answered live will be sent out following the webinar.

Thank you for joining us!

http://www.objectivity.com/