© 2015 IBM Corporation

Bring Data Analytics to the EdgeTon Machielsen - Customer Technical ProfessionalCloud Data Services – IBM Digital Sales

© 2015 IBM Corporation2

Powerful DBaaS For apps that need:• Elastic scalability• High availability• Data model flexibility• Data mobility• Text search

• Geospatial Available as:

• Fully managed DBaaS• On-premises private cloud• Hybrid architecture

With custom coding that enables unique data needs to be developed in days, not weeks

Cloudant – NoSQL Database as a Service

© 2015 IBM Corporation3

dashDB – Analytics Warehouse as a Service

For apps that need:• Elastic scalability• High availability• Data model flexibility• Data mobility• Text search

• Geospatial

Available as:• Fully managed DBaaS• On-premises private cloud• Hybrid architecture

Announced in October 2014 DB2 BLU columnar technology

+ Netezza in-database analytics

• Rapid deployment of large scale data warehouses

• Flexible options for both volume and processing speed

• Unified architecture that enables hybrid data processing on-premises & cloud

BLU Acceleration

Netezza In-Database

Analytics

Cloudant Database

as a Service

© 2015 IBM Corporation4

Fully AutomatedIntelligence

Natural LanguageDialogue

Guided AnalyticDiscovery

Single AnalyticsExperience

IBM Watson AnalyticsSelf-service analytics capabilities in the cloud

© 2015 IBM Corporation5

Use Case / What we will see today

Management of a ficticious Airline company XYZ inotices a stagnation in growth.

Marketing is asked to conduct a Customer Satisfaction Survey to investigate which areas to improve.

1.Survey results are delivered to be analized

2.Survey data is imported in Cloudant for storage

3.Cloudant data is exported to DashDB for analysis

4.Watson Analytics is used to perform the analysis on the data

© 2015 IBM Corporation6

Survey data delivered as CSV file

© 2015 IBM Corporation7

CSV file gets imported into Cloudant

python csv-import.py -f <importfile.csv>-u <username> -d <database name>

© 2015 IBM Corporation8

Sample JSON document in Cloudant

8

{ "_id": "0940405adf4b22beab341d90e4000034", "_rev": "1-a727ce80b1e01d1e7b8365874d4f64a8", "Origin State": "Texas", "Airline Code": "AA", "Type of Travel": "Business travel", "Shopping Amount at Airport": "0", "Airline Name": "Paul Smith Airlines Inc. ", "Scheduled Departure Hour": "8", "No of Flights p.a.": "8", "Airline Status": "Blue", "Flight Distance": "1172", "Orgin City": "Dallas/Fort Worth, TX", "Price Sensitivity": "1", "Arrival Delay greater 5 Mins": "no", "Class": "Eco Plus", "Arrival Delay in Minutes": "0", "Gender": "Male", "Age": "40", "Flight cancelled": "No", "No of Flights p.a. grouped": "1 to 10", "% of Flight with other Airlines": "19", "Departure Delay in Minutes": "5", "Year of First Flight": "2003", "Flight date": "02-14-2014", "Age Range": "40-49", "Destination State": "Virginia", "No. of other Loyalty Cards": "2", "Satisfaction": "4.0", "Day of Month": "14", "Destination City": "Washington, DC", "Flight time in minutes": "140", "Eating and Drinking at Airport": "60"}

© 2015 IBM Corporation9

Integration between Cloudant and DashDB

Create Datawarehouse from Cloudant Dashboard.

Datawarehouse is automatically created on IBM Bluemix using an automatically generated DashDB instance.

Automatic Schema Discovery Process models the NoSQL database into a relational database structure and data is loaded into the DashDB database.

© 2015 IBM Corporation10

Database structure in DashDB

© 2015 IBM Corporation11

Migrating Cloudant JSON into dashDB

Cloudant’s Schema Discovery Process (SDP) translates JSON documents into a schema (or set of tables) that dashDB understands

© 2015 IBM Corporation12

Sample JSON document in Cloudant{ "_id": "019b716168d45be2c2bd8371d4000d5c", "_rev": "1-4b4994f0dd1ddc56b96bd2a16cb080e7", "Received": "2010-10-18T11:36:10.057", "Period": "3rd Quarter (July 1 - Sep 30)", "Amount": "20000", "Client": { "ContactFullname": "JOHN MARK TRUMBORE", "ClientPPBCountry": "USA", "ClientID": "1000634", "GeneralDescription": "Non-profit organization dedicated to restoring the Chesapeake Bay and its tributary rivers", "ClientPPBState": "DISTRICT OF COLUMBIA", "ClientCountry": "USA", "IsStateOrLocalGov": "0", "ClientState": "DISTRICT OF COLUMBIA", "ClientName": "CHESAPEAKE BAY FOUNDATION", "ClientStatus": "1" }, "GovernmentEntities": [ { "GovEntityName": "HOUSE OF REPRESENTATIVES" } ], "Lobbyists": [ { "LobbyistName": "BORSKI, ROBERT A JR", "LobbyisteIndicator": "0", "LobbyistStatus": "0", "OfficialPosition": "Member of Congress (1983-2003)" }, { "LobbyistName": "TRUMBORE, JOHN MARK", "LobbyisteIndicator": "0", "LobbyistStatus": "0", "OfficialPosition": "Leg Asst (Rep. McHale), Leg Dir (Rep. R. Brady)" } ], "Year": "2010", "Type": "THIRD QUARTER REPORT", "Registrant": { "RegistrantPPBCountry": "USA", "RegistrantName": "Borski Associates", "RegistrantCountry": "USA", "RegistrantID": "84376", "GeneralDescription": "Government Relations Consulting", "Address": "4015 Fitler Street\r\nPhiladelphia, PA 19114" }, "Issues": [ { "Code": "ENVIRONMENT/SUPERFUND", "SpecificIssue": "Chesapeake Clean Water and Ecosystem Restoration Act (HR 3852)\nChesapeake Bay Program Reauthorization and Improvement Act (HR 5509)" } ]}

© 2015 IBM Corporation13

Database structure in DashDB

LOBBY-SEARCH• _ID• _REV• AMOUNT• CLIENT_CLIENTCOUNTRY• CLIENT_CLIENTID• CLIENT_CLIENTNAME• CLIENT_CLIENTPPBCOUNTRY• CLIENT_CLIENTPPDSTATE. . . . . .

LOBBY-SEARCH_AFFILIATEDORGS• ARRAY_INDEX• AFFILIATEDORGCOUNTRY• AFFILIATEDORGNAME• AFFILIATEDORGPPBCCOUNTRY• _ID

LOBBY-SEARCH_FOREIGNENTITIES• ARRAY_INDEX• FOREIGNENTITYCONTRIBUTION• FOREIGNENTITYCOUNTRY• FOREIGNENTITYNAME• FOREIGNENTITYOWNERSHIPPERCENTAGE• FOREIGNENTITYPPBCOUNTRY• FOREIGNENTITYSTATUS• _ID

LOBBY-SEARCH_ISSUES• ARRAY_INDEX• CODE• SPECIFICISSUE• _ID

LOBBY-SEARCH_OVERFLOW• EXCEPTION• WARNING• _ID

LOBBY-SEARCH_GOVERNMENTENTITIES• ARRAY_INDEX• GOVENTITYNAME• _ID

LOBBY-SEARCH_LOBBYISTS• ARRAY_INDEX• LOBBYISTINDICATOR• LOBBYISTNAME• LOBBYISTSTATUS• OFFICIALPOSITION• _ID

LOBBY-SEARCH_GOVERNMENTENTITIES• ARRAY_INDEX• GOVENTITYNAME• _ID

Keys with array-values are stored in separate tables.

"Lobbyists": [ { "LobbyistName": "BORSKI, ROBERT A JR", "LobbyisteIndicator": "0", "LobbyistStatus": "0", "OfficialPosition": "Member of Congress (1983-2003)" }, { "LobbyistName": "TRUMBORE, JOHN MARK", "LobbyisteIndicator": "0", "LobbyistStatus": "0", "OfficialPosition": "Leg Asst (Rep. McHale), Leg Dir (Rep. R. Brady)" } ], "Year": "2010",

© 2015 IBM Corporation14

Watson Analytics connection into DashDB

© 2015 IBM Corporation15

Datasets in Watson Analytics

© 2015 IBM Corporation16

Analysis in Watson Analytics

© 2015 IBM Corporation17

Analysis in Watson Analytics

© 2015 IBM Corporation18

Information delivery from Watson Analytics

© 2015 IBM Corporation19

Q&A and more info

Cloudant – http://www.cloudant.comDashDB – http://www.dashdb.comWatson Analytics – http://www.watsonanalytics.comBluemix – http://www.bluemix.net

Me! – http://ibm.biz/ton_machielsen

Try it out for free!

© 2015 IBM Corporation20