Embed Size (px)
Citation preview
Real Time Sentiment AnalysisAssociated Mentor:- Gopichand Kotana
MLG55Mithlesh Kumar Faizanurrahim Ansari Prince Gaurav
Puneet Kumar Verma
Department of Computer Science & EngineeringIIT Kanpur
CS771 Project Presentation, 2017
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 1 /
11
Outline
1 Introduction
2 Previous Work
3 Our Work
4 Configuration Graph
5 Experimental Results
6 Examples
7 Possible Reasons for low accuracy
8 Future Work
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 2 /
11
Introduction
Our aim is to perform real time sentiment analysis on videos tooutput a sentiment value corresponding to 5 labels - happiness, love,sadness, violence and fear.
Focus was on the classification of the general scene not on humanfacial expressions.
Our model takes input a video and outputs the real time probabilityof that frame belonging to the 5 labels.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 3 /
11
Introduction
Our aim is to perform real time sentiment analysis on videos tooutput a sentiment value corresponding to 5 labels - happiness, love,sadness, violence and fear.
Focus was on the classification of the general scene not on humanfacial expressions.
Our model takes input a video and outputs the real time probabilityof that frame belonging to the 5 labels.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 3 /
11
Introduction
Our aim is to perform real time sentiment analysis on videos tooutput a sentiment value corresponding to 5 labels - happiness, love,sadness, violence and fear.
Focus was on the classification of the general scene not on humanfacial expressions.
Our model takes input a video and outputs the real time probabilityof that frame belonging to the 5 labels.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 3 /
11
Introduction
Our aim is to perform real time sentiment analysis on videos tooutput a sentiment value corresponding to 5 labels - happiness, love,sadness, violence and fear.
Focus was on the classification of the general scene not on humanfacial expressions.
Our model takes input a video and outputs the real time probabilityof that frame belonging to the 5 labels.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 3 /
11
Previous Work
Xu et. al. paper on ”Visual Sentiment Prediction with DeepCNN”
Uses a CNN pretrained on a very large scale dataset.
The learned parameters are then transferred to the task of sentimentprediction.
Campos et. al. paper ”From pixels to sentiments: Fine tuningCNNs for visual sentiment prediction”
Uses fine tuned CNN to classify images into positive and negativesentiments.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 4 /
11
Previous Work
Xu et. al. paper on ”Visual Sentiment Prediction with DeepCNN”
Uses a CNN pretrained on a very large scale dataset.
The learned parameters are then transferred to the task of sentimentprediction.
Campos et. al. paper ”From pixels to sentiments: Fine tuningCNNs for visual sentiment prediction”
Uses fine tuned CNN to classify images into positive and negativesentiments.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 4 /
11
Previous Work
Xu et. al. paper on ”Visual Sentiment Prediction with DeepCNN”
Uses a CNN pretrained on a very large scale dataset.
The learned parameters are then transferred to the task of sentimentprediction.
Campos et. al. paper ”From pixels to sentiments: Fine tuningCNNs for visual sentiment prediction”
Uses fine tuned CNN to classify images into positive and negativesentiments.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 4 /
11
Previous Work
Xu et. al. paper on ”Visual Sentiment Prediction with DeepCNN”
Uses a CNN pretrained on a very large scale dataset.
The learned parameters are then transferred to the task of sentimentprediction.
Campos et. al. paper ”From pixels to sentiments: Fine tuningCNNs for visual sentiment prediction”
Uses fine tuned CNN to classify images into positive and negativesentiments.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 4 /
11
Previous Work
Xu et. al. paper on ”Visual Sentiment Prediction with DeepCNN”
Uses a CNN pretrained on a very large scale dataset.
The learned parameters are then transferred to the task of sentimentprediction.
Campos et. al. paper ”From pixels to sentiments: Fine tuningCNNs for visual sentiment prediction”
Uses fine tuned CNN to classify images into positive and negativesentiments.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 4 /
11
Previous Work
Xu et. al. paper on ”Visual Sentiment Prediction with DeepCNN”
Uses a CNN pretrained on a very large scale dataset.
The learned parameters are then transferred to the task of sentimentprediction.
Campos et. al. paper ”From pixels to sentiments: Fine tuningCNNs for visual sentiment prediction”
Uses fine tuned CNN to classify images into positive and negativesentiments.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 4 /
11
Our Work I
Used transfer learning to take a model pretrained on ImageNetdatabase and fine tune on to our database for emotion classification.
Database was obtained using Flickr’s API.
Experimented with VGG16, ResNet50 and AlexNet.
Used OpenCV to capture frames from the video and run our modelwith these frames as input.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 5 /
11
Our Work II
Figure: A pictorial explanation of training. Pic credit:- Xu et. al paper
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 6 /
11
Configuration Graph
Figure: Configuration Graph of our model
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 7 /
11
Experimental Results
AlexNetGot an accuracy of 34% on test set.
VGG16Got an accuracy of 37% on test set.
ResNet50Got an accuracy of 27% on test set.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 8 /
11
Examples I
(a) Love (b) Violence (c) Fear
(d) Happiness (e) Sadness
Figure: Some examples of true positives
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 9 /
11
Possible Reasons for low accuracy
Model classifies positive and negative sentiments with decent accuracybut finds it difficult to further classify into different emotions.
71.1 % accuracy on positive and negative sentiment classification.
Models learns colors :-
Pink → LoveBlack → SadnessBright → Happiness
Poor dataset.
Poor correlation.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 10 /
11
Possible Reasons for low accuracy
Model classifies positive and negative sentiments with decent accuracybut finds it difficult to further classify into different emotions.
71.1 % accuracy on positive and negative sentiment classification.
Models learns colors :-
Pink → LoveBlack → SadnessBright → Happiness
Poor dataset.
Poor correlation.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 10 /
11
Possible Reasons for low accuracy
Model classifies positive and negative sentiments with decent accuracybut finds it difficult to further classify into different emotions.
71.1 % accuracy on positive and negative sentiment classification.
Models learns colors :-
Pink → LoveBlack → SadnessBright → Happiness
Poor dataset.
Poor correlation.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 10 /
11
Possible Reasons for low accuracy
Model classifies positive and negative sentiments with decent accuracybut finds it difficult to further classify into different emotions.
71.1 % accuracy on positive and negative sentiment classification.
Models learns colors :-
Pink → LoveBlack → SadnessBright → Happiness
Poor dataset.
Poor correlation.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 10 /
11
Possible Reasons for low accuracy
Model classifies positive and negative sentiments with decent accuracybut finds it difficult to further classify into different emotions.
71.1 % accuracy on positive and negative sentiment classification.
Models learns colors :-
Pink → LoveBlack → SadnessBright → Happiness
Poor dataset.
Poor correlation.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 10 /
11
Possible Reasons for low accuracy
Model classifies positive and negative sentiments with decent accuracybut finds it difficult to further classify into different emotions.
71.1 % accuracy on positive and negative sentiment classification.
Models learns colors :-
Pink → LoveBlack → SadnessBright → Happiness
Poor dataset.
Poor correlation.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 10 /
11
Future Work
Seq2Seq model can be used to pass the parameters of the previousframe to predict the current frame.
Audio could also be used to classify videos into emotions.
Can use an ensemble of models that predict the facial expressions ofpeople (if present) and the model that classifies the general scene.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 11 /
11
Future Work
Seq2Seq model can be used to pass the parameters of the previousframe to predict the current frame.
Audio could also be used to classify videos into emotions.
Can use an ensemble of models that predict the facial expressions ofpeople (if present) and the model that classifies the general scene.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 11 /
11
Future Work
Seq2Seq model can be used to pass the parameters of the previousframe to predict the current frame.
Audio could also be used to classify videos into emotions.
Can use an ensemble of models that predict the facial expressions ofpeople (if present) and the model that classifies the general scene.
MLG55 Mithlesh Kumar, Faizanurrahim Ansari, Prince Gaurav, Puneet Kumar Verma (IIT Kanpur)Real Time Sentiment AnalysisCS771 Project Presentation, 2017 11 /
11
