Modeling Temporal Intention in Resource Sharing

Modeling Temporal Intention in Resource Sharing

Hany SalahEldeen & Michael Nelson Modeling Temporal Intention. WADL2013

Hany M. SalahEldeen & Michael L. NelsonOld Dominion University

Department of Computer ScienceWeb Science and Digital Libraries Lab.

WADL 2013

All tweets are equal…

…but some are more equal than the others

Hany SalahEldeen & Michael Nelson 01 Modeling Temporal Intention. WADL2013

Preliminary research questions:

1. How long would these last?2. And if lost, is there backup somewhere?3. Is this what the author intended?


Since tweets are considered the first draft of history… the historical integrity of the tweets could be compromised.


Historical integrity

People rely on social media for most updated information


The life cycle of a social post



tweets



tweets Links to



tweets

What the reader

receives

Links to

Same state the author intended



tweets

What the reader

receives

Links to



The resource has disappeared


tweets

What the reader

receives

Links to



The resource has changed



Resource’s possibilitiesWhat the

reader receives





Resource’s possibilities

a bigger problem since the reader might not know.

What the reader

receives




We could lose the linked resource


The attack on the embassy was in February 2013

Or the resource could change


Why do we want to detect the Author’s Temporal Intention?

• Match: and convey the intended information.• Notify:– the author that the resource is prone to change.– the reader that the resource has changed.

• Preserve: the resource by pushing snapshots into the archive automatically.

• Retrieve: the closest archived version to maintain the consistency.


Our investigation angles

1. The state of the archived content2. The age of the shared resource 3. The states of the resource:

1. Missing from the live web2. Changed from what the author intended to share

4. Detect the author’s intention and collect a dataset5. Model this intention6. Create a time-based navigation tool to match the predicted

intention


Estimating web archiving coverage• Goal: Estimate how much of the public web is present in the public archives and how

many copies are available?• Action:

– Getting 4 different datasets from 4 different sources:• Search Engines Indices• Bit.ly• DMOZ• Delicious.

• Results: *

• Publications: – How much of the web is archived? JCDL '11

* Table Courtesy of Ahmed AlSum JCDL 2011






intention


The timeline of the resource


Timestamps accumulation






intention


• From Twitter, Websites, Books:• The Egyptian revolution.

• From Twitter Only:• Stanford’s SNAP dataset:• Iranian elections.• H1N1 virus outbreak.• Michael Jackson’s death.• Obama’s Nobel Peace Prize.

• Twitter API:• The Syrian uprising.

Six socially significant events


Resources missing & archived


Revisiting after a year…


Measured vs. predicted


Interesting phenomenon: reappearance on the live web and disappearance from

the archives


Reappearing and disappearance predictions






intention


Temporal Intention Relevancy Model( TIRM)

Between ttweet and tclick:

The linked resource could have:• Changed• Not changed

The tweet and the linked resource could be:• Still relevant• No longer relevant


Resource is changed but relevant

• The resource changed• But it is still relevant

Intention: need the current version of the resource at any time


Relevancy and intention mapping

Current


Resource is changed and not relevant

Intention: need the past version of the resource at any time

• The resource changed• But it is no longer relevant


Past


Current


Resource is not changed and relevant

Intention: need the past version of the resource at any time

• The resource is not changed• And it is relevant


Past


Current

Past


Resource is not changed and not relevant

Intention: I am not sure which version of the resource I need

• The resource is not changed• But it is not relevant


Past


Current

Past Not Sure






intention


Feature extraction

• For each tweet we perform:– Link analysis– Social Media Mining– Archival Existence– Sentiment Analysis– Content Similarity– Entity Identification


• To remove confusion we removed the close calls

898 instances remaining

Relevant Assignments 929 82.65%

Non-Relevant Assignments 195 17.35%

5 MT workers agreeing (5-0 split) 589 52.40%

4 MT workers agreeing (4-1 split) 309 27.49%

3 MT workers agreeing (3-2 close call split) 226 20.11%

Modeling and classification using Mechanical Turk


The trained classifier

• From the feature extraction phase we extracted 39 different features to train the classifier.

• Using 10-fold cross validation, the Cost Sensitive Classifier Based on Random Forests gave the highest success rate = 90.32%


Testing the model10-Fold Cross-Validation Testing

Classifier Mean Absolute Error

Root Mean Squared Error

Kappa Statistic

Incorrectly Classified %

Correctly Classified %

Cost sensitive classifier based on Random Forest

0.15 0.27 0.39 9.68% 90.32%

Classifier Precision Recall F-measure Class

Cost sensitive classifier based on Random Forest

0.930.53

0.960.37

0.950.44

RelevantNon-Relevant

Weighted Average 0.89 0.90 0.90






intention


TimeLord Navigator


Thanks!


Hany SalahEldeenWeb Science & Digital Libraries Old Dominion University

Email: [email protected]

@hanysalaheldeen

Hany SalahEldeen

mailto:[email protected]

TimeLord Navigator


Demo:

www.cnn.com

www.bbc.com

http://128.82.7.39:10000/reader/http:/t.co/ryivzJWrfG

http://128.82.7.39:10000/reader/http:/t.co/wPLTp7d06B

Evaluation


Actual Vs. Estimated Dates


Resources Missing & ArchivedCollection Percentage Missing Percentage Archived

23.49%H1N1 Outbreak 41.65%

36.24%Michael Jackson 39.45%

26.98%Iran 43.08%

24.59%Obama 47.87%

10.48%Egypt 20.18%

7.04%Syria 5.35%

31.62% 30.78%

24.47% 36.26%

25.64% 43.87%

26.15% 46.15%


First Attempts to Shared Content Replacement


Link analysis

• Since the tweets have embedded resources shortened by Bit.ly we can extract:– Total number of clicks– Hourly click logs– Creation dates– Referring websites– Referring countries.

• We calculate the depth of the resource in relation to its domain (either it is a leaf node or a root page)– We calculated the number of backslashes in the resource’s URI


Social Media Mining

• Twitter:– Using Topsy.com’s API to

extract:• Total number of tweets.• The most recent 500.• Number of tweets by

influential users.

The collection of tweets extracted provided an extended context of the resource authored by users in the twittersphere.


Social Media Mining• Facebook:– Mined too for likes, shares, posts, and clicks related to each

resource.


Archival Existence• Using Memento Time

Maps we get:– Total mementos

available– Different archives count.– The closest archived

version to the tweet time.


Sentiment Analysis• Using NLTK libraries of natural language text processing• Extract the most prominent sentiment in the text


Content Similarity• Steps:– We download the content HTML using Lynx browser.– We apply boilerplate removal algorithm and full text extraction.– Calculate the cosine similarity between the two pages.

70% similarity


Entity Identification• By visual inspection we observed that the majority of tweets about

celebrities are related to current events.• We harvested Wikipedia for lists of actors, politicians, and athletes.• Checked the existence of a celebrity mention in the tweets.

Actor: Johnny Depp


Documents

Modeling Temporal Intention in Resource Sharing