Painting a Social & Mobile Picture in Real Time

Painting a Social & Mobile Picture in Real Time

© 2015, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

Abstract

Abbreviations

Market Trends and Challenges

Solution

Requirements

Architecture

Key Features Key Features

Solution in Detail

Case Study

Twitter Real Time Analysis

E-Commerce Real Time Analysis

Best Practices

Conclusion

ReferenceReference

Author Info

3

3

3

6

6

6

66

7

8

8

10

12

13

1313

13

Table of Contents


Data has been around since time unknown; our books, records, journals and scriptures all contain data in some form or the other. As we entered the computer era, more data points were generated and ware-housed. The size of data soon tipped over grasping power and ‘analysis’ came to the rescue with graphs and charts. Analysts had a desire to predict the future or take decisions in real time based on the data but there were constraints, such as computing, limited data sets and the cost of computation. The computing part was made simpler with ‘Map Reduce’ and ‘Real Time Processing’ that helped us view the present as it happened. DataData changed its nature too; Social Networks brought in the challenge of utilizing huge amounts of unstruc-tured data for business value. Meanwhile, the silent Mobility revolution has materialized with the advent of smartphones.

A simple solution that can utilize data, tap into social sentiments and provide business value to mobile users is much desired. Social data can be tapped for both society and business, and everyone is looking for an application that can address both. In this paper, we analyze a working solution, its tenets and features, and also indulge in a bit of future gazing.

Sl.No

1 RTB Real Time Bidding

Full FormAcronyms

Abstract

The Figure 1 infographic highlights that: The YOY growth of mobile devices at 5% is higher than the growth of the worldwide population; hence, we can now reach more people using mobiles than are born every year. The Social Media YOY growth is 12%, which reiterates Aristotle’s ‘Man is a Social Animal’; moreover, it showcases the mutual dependencies of humans. 23% YOY growth of Social Mobile Accounts highlights the fact that not only are humans dependent on their fellow beings but also want to keep in touch closely

Market Trends and Challenges

Abbreviations

Painting a Social & Mobile Picture in Real Time | 3


The reach of mobility and the utility of social networks are further strengthened by the latest UN Report on ‘Natural Disasters’ during 2014 in the Asia Pacific region. It claims that a Mobile App developed by NSRC, India was of great use as it allowed viewers access to post-disaster damage, and thus enabled quicker relief and saved many lives. Mobile devices were also useful for spreading early warnings for evacuation through SMS and Social Networks during cyclone Hudhud. However, the key challenges are:

In quickly processing and assimilating data Cheap alternatives of existing COTS solutions for Real Time Data Processing Monitoring an event in real time

Figure 1. How the Digital World has evolved over the past 12 months

Source: http://wearesocial.net/blog/2015/01/digital-social-mobile-worldwide-2015/


Gartner’s Hype Cycle talks about the ‘Nexus of Forces’ (mobile, social, cloud and information/data) during the Digital Marketing Stage. It claims that “focus is on new and more sophisticated ways to reach consumers who are more willing to participate in marketing efforts to gain greater social connection, or product and service value”.

The key points that emerge from Figure 2 and the detailed report, are:

Data Science, Content Analytics and In Memory Analytics are some of the key trends in the hype cycle. Big Data has become a commodity and is no longer the most hyped technology trend. IoT and Wearable Interfaces are the most hyped trends now and have probably reached a plateau. Challenges

Can an advertisement be pushed to a user depending on his location and his ‘information need’ via social networks? Can a CEO monitor the launch of a new product in the market in real-time? Can we estimate the popularity of a new chartbuster?

Figure 2. Hype Cycle for Emerging Technologies, 2014

Source: http://www.gartner.com/newsroom/id/2819918



Requirements

Data has to be processed in real time to reach people faster. Save lives, greater business value The technology should be cheap, simple and easy to use. A common man should be able to use it It can be accessed from any device and installed in commodity hardware The analytical value of the technology should be impactful. Analysts, CEOs and the common man should find real value

Architecture

Solution

Figure 3. Architecture



Key Features

The background application is defined entirely on lightweight JavaScript, which is platform independent The visualizations have both real time and batch flavors Options for persisting important data and throwing away other data Real Time Social Sentiment Analysis

Solution

The streaming API of Social Networks was used to tap into unlimited social data. In some cases, the keyword ‘Search’ option is also provided and used The streaming data was ingested using Node.js api The data was mined for keywords and sentiments using Node.js modules Micro-batch processing was used to run Map Reduce, to aggregate the data A combination of Angular.js and Express was used to create the web-application framework A great combination of HTML5, JSON, CSS and SVG was used to create dynamic and static data visu alization Socket.io was used to enable real time event based communication. This can be used to further push notifications to mobile devices


Figure 4. A part of the Server Code


Twitter Real Time Analysis

Twitter provides two feeds for tweets Firehose: Commercial (millions of tweets per hour), Public: Provides 1% of the tweets ONLY. This is available for developers/public to make apps on twitter. It also has two services streams: Real Time, Search: This gives results from the Twitter database and is based on OLD DATA

TheThe case study tapped into the real time PUBLIC feed of Twitter and received tweets based on certain keywords in real time. The tweets received were used to create visualizations, each of which had a different use case as follows:

Real Time Sentiment & Popularity Analysis

Figure 5. Real time Tweet Sentiment Flow


Case Study


The graph in Figure 5 shows the speed of tweets and sentiment flow in Twitter the day the Apple Watch was launched. The graph moves in real time from right to left, and up and down. The speed of the graph depicts the number of tweets, and the up and down movement depicts the sentiment flow from positive to negative. This feature can be used by any company or marketing team to monitor a newly launched product. It can also be used by disaster monitoring/health teams all around the globe to gauge the damage or outbreak of a disease.


Figure 6. Real time Tweet Location Tracking Figure 7. Real time Tweet Location Tracking

Real Time Tweet Location Tracking – Geo Tagging

The icons in the map in Figure 6 show people tweeting about the Apple Watch. Using the above feature we can actually track a person in real time. This can be used by RTB teams to push real time advertisements to a person’s mobile in relation to the local establishments. On the other hand, this feature can be used to track down a person in distress. The Indian Army used a feature similar to this to find a concentration of people stranded in the recent Jammu and Kashmir floods.

Heat-Map Analysis of Sentiments in Social NetworksHeat-Map Analysis of Sentiments in Social Networks

This diagram depicts the buckets, or tweet counts of sentiments ranging from -15 to 15 for the ebola disease over a month. By the end of the month, the sentiments shifted from negative to positive due to the introduc-tion of medicines / shots, etc. This feature enables enterprises/businesses to highlight certain issues and high-light the social spread of these issues across various sentiment segments. Also, as depicted in this example, this can be used to find whether any disaster/pandemic is slowly loosening its grip over human lives and suf-fering.

Real Time Word Cloud – Popular Feature Identification

The word cloud is created every 100 tweets in real time. It consists of top N terms. This can be used to identify the popular features of a newly launched product/ merchandise like a mobile or a game. This will also help to understand what the greatest impacted areas or items are in times of a disaster, as captured in this diagram during the 2014 earthquake in California where the wine industry was greatly impacted. You can see the term ‘winemakers’ and ‘California’ in the word cloud.



Figure 8. Real Time Word Cloud

Figure 9. Products bought and sold by Online Shoppers in India

E-Commerce Real Time Analysis

The India E-Commerce market will reach $6 billion in 2015 - a 70% increase over 2014 revenues of $3.5 billion, according to Gartner, Inc. Digital commerce facilitates a purchasing transaction over the Web, and supports the creation and continuing development of an online relationship with a consumer or business customer across multiple retail, wholesale, mobile, direct and indirect sales, call center, and digital sales channels.



Figure 10. Smartphone Vendor Share in India

Challenges in E-Commerce sites

How to identify changing product requirements with respect to customer needs How does the social impact of an existing user (may be a post or a review) impact a customer when buying a similar product Gauging the impact of offers and discounts during a specific campaign or event Gauging a customer’s opinion and attitude towards a particular feature of a product

TheseThese challenges show that social media plays a vital role in persuading a customer to buy something addi-tional or more expensive, from an ecommerce site.

The smartphone market has been trending a lot recently. We’ve done a study by crawling the reviews of a par-ticular smartphone from one of the e-commerce sites, performed the process for the extraction of features, and summarized the score of the entire product, feature-wise, together with the trending features.

When a product is purchased, it has the following influencing factors based on price, geographical location, key features the customer was looking for, technology, and more.

1. Extraction of customer enlisted specific features / feature descriptors from a bulk repository of features enlisted for a particular product, categorized in a particular segment of technology.


For more details contact: [email protected] us on twitter: http://twitter.com/hclers andOur blog http://www.hcltech.com/blogs/engineering-and-rd-servicesVisit our website: http://www.hcltech.com/engineering-rd-services

Hello, I’m from HCL’s Engineering and R&D Services. We enable technology led organizations to go to market with innovative products and solutions. We patner with our customers in building world class products and creating associated solution delivery ecosystems to help bring market leadership. We develop engineering products, solutions and platforms across Aerospace and Defense, Automotive, Consumer Electronics, Software, Online, Industrial Manufacturing, Medical Devices, Networking and Telecom, Office Automation, Semiconductor and Servers & Storage for our customers.

This whitepaper is published by HCL Engineering and R&D Services.

The views and opinions in this article are for informational purposes only and should not be considered as a substitute for professional business advice. The use herein of any trademarks is not an assertion of ownership of such trademarks by HCL nor intended to imply any association between HCL and lawful owners of such trademarks.

For more information about HCL Engineering and R&D Services,Please visit http://www.hcltech.com/engineering-rd-services

Copyright@ HCCopyright@ HCL TechnologiesAll rights reserved.

Kinnar Kumar Sen HCL Engineering and R&D Services

Gopi Krishna Durbhaka HCL Engineering and R&D Services

Reference

Conclusion

Author Info

[1] http://wearesocial.net/blog/2015/01/digital-social-mobile-worldwide-2015/

[2]http://www.thehindu.com/features/education/research/man-as-a-social-animal/article2988145.ece

[3] http://en.wikipedia.org/wiki/Homo_sapiens

[4] http://www.gartner.com/newsroom/id/2819918

[5]http://pluznews.com/11152/early-warning-evacuations-saved-lives-during-cyclone-hudhud-un.htm

Real Time Analysis is here to stay; businesses and people would love to see more values from data that floats around. A unified platform for this purpose has a lot of potential, and may find its applicability elsewhere too. Technology is used for churning money out of the business melting pot, but it achieves its true value only when it is useful in solving some of the social problems as well. This was a simple attempt to highlight that lightweight, free, and easy to use libraries can be put together to develop a solution that can claim to touch base with both.


Business

Painting a Social & Mobile Picture in Real Time