Monitoring and Analyzing London's Air Quality with Pentaho ... · Monitoring and Analyzing...

MonitoringandAnalyzing London'sAirQualitywithPentahoMarkSemenenko,PentahoSalesEngineer,HitachiVantaraAnyaRumyantseva,PentahoDataScientist,HitachiVantara

Pentaho

Agenda

DevelopingPentahoDemoshowcasingtheplatform’send-to-endcapabilities

• Pentahoasanend-to-endplatform• UseCaseOverview• Demonstration

• Summary

Pentahoprovidesflexiblemanagementforend-to-enddataflows…

DataPrepDataEngineering Analytics

Ingestion Processing Blending DataDelivery DataDiscovery/Analysis

Analysis&Dashboards

Administration Security LifecycleManagement

DataProvenance

DynamicDataPipeline Monitoring Automation

…andMachineLearningcapabilities!

DataTransformationandMachineLearning

Workflows

Fullautomatedproduction

Dashboarddecisionsupport

ypred

DataEngineering DataPreparation Analytics

X

Dataco

nnectors

DataSource1

DataSource2

DataSourcen

Aim:developademoshowingafullspectrumofPentahocapabilities

OurUseCase“LondonAirQualityMonitoring”

Why?

• Airqualityaffectswell-beingofresidentsandvisitorsofthecity• 9000earlydeathsayearinLondonduetoairpollution• Airpollutionmonitoringandmitigationarecrucialforcitycouncils

• Theusecaseiseasytounderstandandtofollow

DataSources

• LondonAirQualityNetwork(96stationsacrossthecity)• RoadTrafficCounts(numberandvehicletype)

• MeteorologicalParameters(wind,temperature,humidity)

Collecting,PreparingandBlending

Demonstration

Collecting,PreparingandBlending

AirQualityPrediction

PentahoMachineLearningOrchestration

1 2

4

3

1

PredictivemodelforAirQualitywasaddedtothedemo.

*Long-ShortTermMemoryneuralnetworkwasusedtomakepredictions

Demonstration

OperationalisingPredictiveModelinPentaho

Whatwehaveshowninthedemo:

• Featureengineeringanddatapreparation• Scheduledmodeltrainingandupdate

• Modelapplicationonnewdatastreams

• Trackingandstoringmodelperformancemetrics

• DisplayingpredictionsinPentahodashboards

OperationalizemachinelearningmodelsinPentaho!

Summary

Whatwecoveredtoday:

• Pentahoprovidesasingleconsistentexperiencefordevelopingdataproducts• Theplatformallowstoputmachinelearningalgorithmsintoproduction

• Informativedashboardsfordecisionmakingbasedonadvancedanalyticsanddatatransformationinthebackend