Big Data Analytics: Facts and Feelings

Big Data Analytics: Facts and Feelings

Seth GrimesAlta Plana Corporation

@sethgrimes

TDWI – Washington DCJune 21, 2013


Theses• We gain knowledge when we make

connections.• Data analysis is a process of

connection discovery.• The more data, the greater the

possibilities.• The more data, the greater the need

to filter, reduce, and contextualize.• Timeliness counts.


The World of Big DataMachine data (e.g., logs, sensor outputs,

clickstreams).Actions, interactions, and transactions:

geolocation and time.Profiles: individual, demographic &

behavioral.Text, audio, images, and video.

Facts and feelings.


A 3-slide reprise...


Imperatives for the 2010s:Do more with more.

“It’s Not Information Overload. It’s Filter Failure”: Clay Shirky, 2008.

• More sources & types of data.• Greater data volumes.• New hardware and methods.

Automate more, more intelligently.• Analytics.• Semantics.

Engage. Socialize.


I see three categories of data:1. Quantities, whether measured,

observed, or computed.2. Content, which I’ll characterize as

non-quantitative information.3. Metadata (semantic & structural)

describing quantities and content.

• Our concern is content, analytics & fusion.

• Structured/unstructured is a false dichotomy.

• Where do relationships fit?


http://www.businessweek.com/magazine/content/04_19/b3882029_mz072.htm

En route.


End of reprise. So, is this Big Data?

44


Of course not. It’s a number, not data.

Size (Volume) is only one Big Data factor.

Other factors (standard definition) are Velocity and Variety.

I reject reVisionist 3Vs extensions such as:

Variation/Variability.Veracity.Value.

These factors are the province of analytics.


Gary King, Harvard Univ. –“Big Data isn’t about the data. It’s about analytics.”

Me – Analytics is a collection of tools and techniques that extract insights from data.

I’d argue –The Value in Big Data is in content, patterns, and connections, derived via analytics.


Variability is an interpretive property. I say –The sense of Big Data is in context and intent…of both the data producer and the data consumer, captured in metadata and (also) derived via analytics.


As for Veracity, data is data. Consider:“The Iraqi regime… possesses and produces chemical and biological weapons.” – George W. Bush, October 7, 2002.


Data… and more. Is this Big Data?

No, it’s a screen of aggregated query results.


The Big Data is behind it.

http://www.newyorker.com/online/blogs/culture/2012/05/google-knowledge-graph.html


And behind comparably-scaled/structured systems.


Comparably-scaled/structured systems?

http://www.cambridgesemantics.com/semantic-university/semantic-search-and-the-semantic-web


Graphs model language models relationships.


Another view, using an old GATE image.

http://gate.ac.uk/hamish/talks/ibot-slidy.html


… for instance, social language.


Text analytics applies natural-language processing (NLP) techniques to discern –

EntitiesRelationships

Context Identity

– and get at the sense of “unstructured” online, social, and enterprise information.

Semantic identity unites data of all types.


http://searchuserinterfaces.com/

Sensemaking:“It is convenient to divide the

entire information access process into two main components: information retrieval through searching and browsing, and analysis and synthesis of results. This broader process is often referred to in the literature as sensemaking. Sensemaking refers to an iterative process of formulating a conceptual representation from of a large volume of information.”

– Marti Hearst, 2009


Intelligent computing – sensemaking –involves:Big (and little) Data.• Quantities.• Content.• Metadata.

Analytics.Semantics.Integration.Facts and feelings.


Feelings: Sentiment detection, classification.


http://techpresident.com/news/21618/politico-facebook-sentiment-analysis-bogus


“Sentiment analysis is the task of identifying positive and negative opinions, emotions, and evaluations.”

-- Wilson, Wiebe & Hoffman, 2005, “Recognizing Contextual Polarity in Phrase-

Level Sentiment Analysis”

“Sentiment analysis or opinion mining is the computational study of opinions, sentiments and emotions expressed in text… An opinion on a feature f is a positive or negative view, attitude, emotion or appraisal on f from an opinion holder.”

-- Bing Liu, 2010, “Sentiment Analysis and Subjectivity,” in Handbook of Natural Language

Processing


Sentiment may be of interest at multiple levels.Corpus / data space, i.e., across multiple sources.

Document.Statement / sentence.Entity / topic / concept.

Human language is noisy and chaotic!Jargon, slang, irony, ambiguity, anaphora, polysemy, synonymy, etc.

Context is key. Discourse analysis comes into play.


Emotion and effect.


Emotion and understanding.


Prediction/Feeling/Wish... and Intent.

http://www.aiaioo.com/whitepapers/intention_analysis_use_cases.pdf

http://sentibet.com/


Many options (text).


(Accessible) Data Everywhere


Beyond Text:• Audio including speech.• Images.• Video.

http://www.geekosystem.com/facebook-face-recognition/

http://www.sciencedirect.com/science/article/pii/S0167639312000118

http://flylib.com/books/en/2.495.1.54/1/


Or just ask: Swipp mobile polling (example).


Surfaced.


A Big Data analytics architecture (example).

http://www.geeklawblog.com/2011/12/lexis-advance-platform-launch-two.html


Complementary.


Seth GrimesAlta Plana Corporation

@sethgrimes

TDWI – Washington DCJune 21, 2013

Documents

Big Data Analytics: Facts and Feelings