PistonHead's use of MongoDB for Analytics

Haymarket Media Group

Building a reporting and analytics suite for dealers and administrators

Pete Dignan - Platform Director@RacingPete

PistonHeadsPistonHeadsStarted in 1998Original social networkAlways irreverentClassifiedsNewsForums

Classifieds140,000 cars6.5 million users100 million page views9 minute average visit durationMore 17k+ vehicles than competitors

Vision - PistonHub

“To create a stock insights and trading screen, providing secure access to the statistics behind the classifieds, and opening up the true performance of our

classifieds to Dealer Groups, Individual Dealers and Internal Staff”

1st Problem - View Data

2nd Problem - Stock and Classifieds Data

MongoDB

Advert Data

Category

Make/Model

History

Ad Status

MySQL

Dealer Groups

Individual Dealers

3rd Problem - Lead Data

PistonHub Architecture Solution

Abstract Data Aggregator

Google Data API

HTTP/SOAP

.Net Driver

ADO .NetAPI AngularJS App

{ "_id" : ObjectId("564a9892ad2e970fac7f6fff"), "CDRId" : "ac61259e35bacc174f3e9ff0afd4c0c58ff6ffff", "NTSNum" : "01234567890", "FromNum" : "0771234567", "ToNum" : "01234567891", "DialledNum" : "+44771234567", "ConnectTime" : ISODate("2015-11-11T02:18:14Z"), "CallDuration" : "3", "TalkTime" : "3", "TimeToAnswer" : "1975", "DisconnectText" : "Answered", "DisconnectType" : "Answered", "Recorded" : "N", "CallId" : "i1447208272.153117.xxxx", "LoadDate" : ISODate("2015-11-11T03:00:01.177Z")}

{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}

{ "DealerAltPhone" : "01234567891", "DealerGroupId" : 0, "DealerGroupIdStr" : "0", "DealerGroupName" : null, "DealerId" : 61056992, "DealerIdStr" : "61056992", "DealerName" : "Dealer LTD", "DealerPhone" : "01234567890", "TrackingAltPhone" : null, "TrackingPhone" : "01234567890", "_id" : ObjectId("555ef398e816135315639fff")}

Raw Gamma Data Raw Dealer Data

Phone Data

Abstract Data Aggregator

{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}

Phone Data

Abstract Data Aggregator

{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}

{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}

{ "CallDate" : ISODate("2015-11-10T22:27:00Z"), "CallDuration" : NumberLong(3), "CalledTrackingNumber" : "01234567890", "CallerNumber" : "0771234567", "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "RingTime" : NumberLong(1975), "Status" : "Answered", "_id" : ObjectId("564a9892ad2e970fac7f6fff")}

View Data

Email Data

Stock Data

{ "AdViews" : 119, "CallLeads" : 0, "Calls" : 1, "Date" : ISODate("2015-11-10T00:00:00Z"), "DealerGroupId" : 0, "DealerGroupName" : null, "DealerId" : 61056992, "DealerName" : "Dealer LTD", "DeepLinkClicks" : 1, "EmailLeads" : 1, "Emails" : 1, "NormalizedDealerName" : "DEALER LTD", "Phone" : "01234567890", "Stock" : 2, "TrackingPhone" : "0771234567", "WebsiteClicks" : 14, "_id" : "61xxxx_2015-10-10"}

Dealer Stats

The Dashboard

The Dashboard

The Dashboard

The Dashboard

The Dashboard

The Dashboard

The Dashboard

Conclusions

Pipeline aggregation is super fast to collate data…

… and run queries for AngularJS

Storing day by day data means easy to do custom date formats

Dealer feedback has been very positive on the extra insights

Sales teams have used the system to proactively help dealers

Wish we were using 3.2

Questions?