Big Data & the Enterprise

  • Published on

  • View

  • Download


Given at the European Trading Architecture Summit 2012


<ul><li> 1. Welcome</li></ul> <p> 2. It used to be easy 3. they all looked pretty much alike 4. NoSQL BigDataMapReduce GraphDocument SharedColumn EventualBigTable CAP Nothing OrientedConsistencyACIDBASE Mongo CouderaHadoopVoldemort CassandraDynamoMarklogic Redis VelocityHbaseHypertable Riak BDB 5. Now its downrightc0nfuZ1nG! 6. What Happened? 7. we changed scale 8. tackge dwe ch an 9. thebig dataconundrum 10. thebig dataconundrum ? 11. The Internet 12. Which isnt mostly textEverything (500,000) Web Pages (40) ~0.01% is web pages Words (0.6) ~1% of that is textSizes in Petabytes 13. And there is lots of other stuff out theremobileweathersensors SocialLogsdataaudiovideo 14. Gartner80% of business is conducted on unstructured information 15. Big Data is now a new class ofeconomic asset* *World economic forum 2012 16. Yet 80% Enterprise Databases &lt; 1TBReferencefrom 2009 17. so what does Big Data mean forthe enterprise? 18. Insight data&gt;Data beats Algorithms 19. Backing up a bit 20. We live in a world of, largely, private data Where data is often changed and forwarded on 21. Sometimes were a bit more organized! 22. But most of our data is not generally accessibleCore Operational ExposedData 23. Sharing is often an afterthought CoreOperational Exposed Data 24. How do we process, acquire, reason about and act upon information? 25. The Brain Reptilian Primitive operations: balance, temperature regulation, breathing Mammalian Emotion, short-term memory, flee of fight etc (limbic) Neocortex Plan, innovate, problem solve etc 26. Our intelligence is segregated indisparate worlds 27. could our corporations bemore intelligent? 28. Siloed, closed, bespoke data makes our organisations opaque andunresponsive 29. What if we exposed it all? 30. So what might that look like? Single data store Federated, homogenous stores Federated, heterogeneous stores 31. The Google ApproachMapReduceGoogle FilesystemBigTableTenzingMegastoreF1DremelSpanner 32. The Ebay Approach 33. so is one approachbetter? 34. Data Volume?TB 0 1 10100 100010,000 We live well within the overlap region 35. Academic acumen? 36. Performance Trade-Off Curve Volume (pure physical size) Velocity (rate of change) Variety (number of different types of data, formats and sources) Static &amp; Dynamic Complexity (do you need to interpret the affect one message has on another) 37. ProblemOur ability to model data is much more of agating factor than raw size, particularly when considering new forms of data Dave Campbell (Microsoft VLDB Keynote) 38. Gravitate around a single data modelGlobally AccessibleApplicationSpecific Core DataModelsCore Data Model ModelViews /linkages 39. The data itself follows a similar patternGlobally AccessibleApplicationSpecific Core DataCore Data Model dataViews /linkages 40. Compose Solutions (for now) 41. Big Data is more than the opportunity forbetter insight over new data sources 42. It is the opportunity to make the organisation smarter, simply bymaking data more accessible 43. But the harder job, for us, is unifyingthe various domains to make all that data intelligible 44. Thanks @benstopford</p>


View more >