Big Data in 6 easy pieces

Business & media intelligence solutions

So, Big Data. It’s a subject that’s been on the lips of the business world for a while now and, if Gartner’s recent hype study http://onforb.es/1p5Pipj is to be trusted, the buzz is beginning to subside as everybody knuckles down to actually make use of it. We know the term Big Data can be confusing; metrics analysts see it as a buzzword for marketing types who, in turn, see it as the key to unlocking results in identifying new markets. At M-Brain, we love elegant simplicity. So, in the words of the late, great Richard Fenyman, here’s ‘Big Data in 6 easy pieces’.

1. What is Big Data?The crux of any subject and the start and end of all arguments is definition. With that in mind, let’s ask “what does big data mean?” Does it refer to volume or technology? Well, it’s both. The term ‘big data’, from a pure analytics POV, is used to describe a massive volume of structured and unstructured data that’s so big it is difficult to process using traditional database and software techniques. From a vendor POV, the term can be used to describe the technology that an organisation needs to store and process what can amount to petabytes (1024 terabytes) or exabytes (1024 terabytes) of raw data. To put that in perspective, and to give you an idea of just how massive big data actually is:

In this age, the human race creates as much information in 48 hours as was created from the beginning of time up to 2003.

2. Why now? (and, also why not)Unless you’ve truly had your head in the sand, you won’t have been able to escape the fact that big data as a concept has been on the rapid rise for a while. This is down to a number of interconnected factors, the first being that data, alongside labour and capital, has now become an important pillar of production. It’s become so central that, according to a 2011 study by MGI, nearly all companies in the US with more than 1,000 employees held at least an average of 200 terabytes of stored data. The rapid and wide proliferation of smartphones, mobile data, aerial sensor networks and social media have all had a massive impact on big data through increasing scope and ease of collecting data. This rising trend has been countered by the fact that the majority of businesses still use traditional data storage and analysis methods, leading to a rise in the need for pure big data analysis solutions.

3. What does this mean for business?The applications for big data in today’s business world are as myriad as the types of data collected, which can range from eBay buyer patterns to twitter data produced by the world cup (which broke world records for 672 million tweets during the competition) to data generated by emergencies like Hurricane Sandy. What this means for business is a more responsive way of doing things. By harnessing the power of big data through analysis solutions, companies and brands can more easily find untapped demographics and offer consumers and clients more intelligent, tailored and relevant solutions.

4. Big data and brands

So, which brands out there are making big data actionable to their benefit? To comprehensively list them all would be the subject of a whole blog post on it’s own but, for brevity’s sake, I’ve chosen two household names who stand out in this field.

Netflix

We all know and love Netflix; their on-demand TV and film service has become something of a cultural trope in recent years and their use of big data is something that everyone sees, but few notice. It can be recognised in their algorithm based recommendation service that uses customer data to drive recommendations and continues to drive customer actions into delivering a better experience.

eBay

One of eBay’s biggest initiatives to date has been the introduction of their new, data driven, home page, which they call ‘the feed’. Using a social media approach to the feed (the ability to follow categories is very reminiscent of Twitter and it’s layout just screams pinterest) eBay’s marketing team keeps a close eye on analytics and any way they can use these to optimise customer experiences. In the words of their CMO, Richelle Parham, "I like when I have the opportunity to understand what inspires their path to purchase and what matters to them".

5. Problems & Bias

When there’s a new kid on the block, there’s usually teething problems. The problems with big data stem from what could be called ‘data fundamentalism’ which is essentially the belief that large data sets and predictive analytics always reflect objective truth. This is partly true but, due to the way data sets are collected, it isn’t the whole truth. The problem with data sets is that they are, at their core, human creations and that is where the hidden biases start to make themselves apparent. A prime example of this a recent study that combined twitter and foursquare data generated by Hurricane Sandy. The majority of data about the Hurricane was generated in Manhattan, which leads to the natural assumption that Manhattan was the worst hit area. It wasn’t and areas like Coney Island and Rockaway were far worse affected.

Take also Boston’s pothole problem; the city has a massive problem with potholes, with over 20,000 a year to deal with. To combat this they released the very well received StreetBump smartphone app that combines accelerometer and GPS data to locate potholes. The problem here is that people in lower income brackets, whose locales are more likely to have potholes, are less likely to own a smartphone, leading to compromised data. As I mentioned above, these problems are essentially teething issues, which as analytics, algorithms and understanding of them becomes more intelligent and complex, are likely to become a thing of the past. This also stresses the importance for companies in choosing a data analysis solution that’s intelligent enough to see through hidden biases.

6. The FutureWhat lies in store for big data? Well, if Gartner’s recent hype cycle study is to be believed, big data buzz is past it’s peak. This doesn’t mean it’s going anywhere however, just that the hype is now dying as companies begin to get their teeth into it and start to find intelligent big data solutions.

There’s no question about it, big data is here to stay and if companies continue to use traditional data storage and analysis methods, the need for powerful big data analysis solutions will become ever more prevalent in coming years. Paul ClarkeM-Brain UK

www.m-brain.com 0118 956 5834 lee.west@m-brain.com

Follow us for more Big Data scoops @MBrainUk

M-Brain Uk