When Two Seconds Is Too Long
A look at low latency, real-time analytics
VP Product [email protected]
@daiclegg
dai clegg
@daiclegg
2
The new data sources driving big data analytics
2
mobile marketingsocial appsinfrastructure monitoring
batch reportingexploratory analysisdata discovery
Batch AnalyticsOperational Intelligence
infrastructure fabric/logssmart grids/smart metersdfid tags, etc
social mediamobile appsweb clicks
Machine DataSocial Data
@daiclegg
TITLE HERE
3
the Big Data technology landscape
data mining/ data warehousing
quantitative analytics
data access
@daiclegg
Subtitle
TITLE HERE
4
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
complexity of analytics
use case
money-offcoupons
data mining/ data warehousing
quantitative analytics
data access
‣ Identifies items that shoppers are likely to buy in future visits
‣ Coupon redemption rates as high as 24%
@daiclegg
Subtitle
TITLE HERE
babies
5
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcoupons
data mining/ data warehousing
quantitative analytics
data access
complexity of analytics
use case
‣ Neo-natal infant monitoring‣ 120 babies monitored‣ 120k messages /second
@daiclegg
Subtitle
TITLE HERE
babies
6
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
data mining/ data warehousing
quantitative analytics
data access
complexity of analytics
use case
‣ 2.5 petabytes in Hadoop‣ weather data, turbine operational data‣ model weather to optimise wind farms
@daiclegg
Subtitle
TITLE HERE
babies
7
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
data mining/ data warehousing
quantitative analytics
data access
‣ Neo-natal infant monitoring‣ 120 babies monitored‣ 120k messages /second
‣ Global e-store‣ Shopping cart session store‣ 000s of transactions per second
shopping carts
complexity of analytics
use case
@daiclegg
Subtitle
TITLE HERE
babies shopping
carts
8
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
data mining/ data warehousing
quantitative analytics
data access
Hi-tech Mftg
‣ automated hi-tech assembly
‣ ‘000s of test readings per second‣ comparing results to historic metrics‣ monitoring the test stations’ performance
complexity of analytics
use case
@daiclegg
Subtitle
TITLE HERE
babies shopping
carts
9
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
data mining/ data warehousing
quantitative analytics
data access
Hi-tech Mftg
taxis
complexity of analytics
use case
‣ Neo-natal infant monitoring‣ 120 babies monitored‣ 120k messages /second
‣ Real-time visibility of infrastructure‣ Insight delivered into the cab‣ Caught competitor ‘stealing’ web data
@daiclegg
Subtitle
TITLE HERE
babies shopping
carts
10
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
data mining/ data warehousing
quantitative analytics
data access
Hi-tech Mftg
taxisSMS Mktg
‣ ingesting 300m SMSs daily‣ over 5000 events per second
‣ maintaining 90 days history per campaign‣ Oracle/NetApp only supported 45 days
complexity of analytics
use case
@daiclegg
Subtitle
TITLE HERE
babies shopping
carts
11
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
data mining/ data warehousing
quantitative analytics
data access
Hi-tech Mftg
taxisSMS Mktg
X-factor
complexity of analytics
use case
‣ Neo-natal infant monitoring‣ 120 babies monitored‣ 120k messages /second
‣ 1000s of votes/boos/applauds per second‣ “we were able to handle our problems with
$5,000 and a credit card.” - CTO, Tellybug
@daiclegg
Subtitle
TITLE HERE
babies shopping
carts
12
the Big Data technology landscape
hours minutes seconds milli-secondsimmediacy of results
money-offcouponswindmills
latest data, historic context, analytic insight
deep analyticsworth the wait
data mining/ data warehousing
quantitative analytics
data access
Hi-tech Mftg
taxisSMS Mktg
X-factor
just need put & get
just need recent data
complexity of analytics
use case
@daiclegg
the Acunu Analytics
13
Acunu
Analytics
delay text
delay text
Prod 4Prod 3Prod 2
Ventas Ron Rate
Ventas BSF
Ventas x VendedorMes en Curso
Cuentas Por Cobrar
Cuentas Por Pagar
Contabilidad
Prod 1
Acunu Analytics can ingest data, at very high velocity, from any source
The data is pre-processed, as it arrives, to filter, transform and enrich it with other corporate data
And aggregated into roll-up cubes of sums, averages, top k, etc, so query answers are already stored
Then, when a dashboard query is executed, the answer is there for instant response
But not just dashboards; there’s a JSON API so queries can be embedded in other apps
And the original data is stored for further analysis and to share with other analytic tools
Recommended
