Sentiment Analysis of Social Media for Evaluating Universities · Sentiment Analysis of Social Media for Evaluating Universities ... Twitter as one of the most popular microblog-

Sentiment Analysis of Social Media for Evaluating Universities

Anas Abdelrazeq Daniela Janssen Christian Tummel Sabina Jeschke Anja RichertIMA/ZLW & IfU at RWTH Aachen UniversityDennewartstrasse 27, 52068 Aachen, Germany

[email protected], [email protected],[email protected], [email protected],

[email protected]

ABSTRACT

In the age of digitalization, a huge amount of sen-timents are expressed daily on university relatedtopics using social media platforms. Particularly,posted statements from students and teachers canprovide a potential source for evaluating universi-ties. Twitter as one of the most popular microblog-ging platforms is a rich data resource for opinionmining. Stimulated by this fact, ways to analyzeTwitter for information in the context of universi-ties are sought. This paper looks at the analysis ofsocial media sentiment as a complementary sourcefor evaluating universities. The extracted resultscan support university rankings that experience crit-icism in terms of measuring vital indicators. Thispaper relays on sentiment analysis methods to an-alyze opinions published on Twitter. For this pur-pose, at first, tweets that are related to selected uni-versities in Germany were collected. Second, thetweets were classified based on their sentiment into“Positive” and “Not Positive” tweets. At last, theresults were analyzed providing information aboutthe communicative topics at the universities. Thispaper gives an outlook to further research in contextof an automated analysis of social media content inorder to support the evaluation of universities.

KEYWORDS

Social Media, Universities, Higher Education,Twitter, Sentiment Analysis, Data Analysis, Natu-ral Languages Processing, Learning Environments

1 INTRODUCTION

Nowadays, social media platforms form a sub-stantial data source for opinions on any topic.This include statements that are related to uni-versities’ topics and events. Particularly, mi-croblogs appeared as one of the most common

social media tools, in which, countless userscan participate and interact at any time. Us-ing microblogging platforms, content from ev-ery user can be published, read, commented,linked or forwarded as desired. As a result, anopen communication medium is established.This open exchange with the participation ofdifferent users helps to bridge and interlockformal and informal learning contexts. Conse-quently, microblogging is a medium that offersreal-time communication of its users thoughtsand ideas.Twitter is one of the most opinion-rich re-sources, where huge amounts of opinions ondifferent topics are expressed. Such opinion-rich data can be used for extracting and analyz-ing the opinions in terms of specific questions.In this context, data mining techniques havethe potential to detect opinions from largeamounts of data. With the help of existing toolsfrom machine learning and natural languagesprocessing fields, concepts of sentiment analy-sis can be applied on the collected data fromsocial media. This can provide vital mea-surements and insights over the communica-tive topics in the university environment.Analyzing students and teachers opinions aswell as comparing and evaluating universitiesis important on different scales. On the onehand, universities would like to have a perfor-mance measurement mechanism for improve-ment and adjusting plans. On the other hand,students relay on these comparisons to supporttheir decision for joining a specific universityto proceed with their studies.The standardized process of comparing andevaluating universities is based on universityrankings on a national and international level.It operates on the basis of certain criteria (i.e.

ISBN: 978-1-941968-26-0 ©2015 SDIWC 49

Proceedings of Second International Conference on Digital Information Processing, Data Mining, and Wireless Communications (DIPDMWC2015), Dubai, UAE, 2015

indicators). Recently, university rankings ex-perienced a lot of criticism in terms of: mea-surement accuracy, measuring the university asa whole institution, and the way data is beingcollected for measuring specific indicators atuniversities.Besides the official standardized universityrankings, alternative and complementary ap-proaches must be investigated. Standard rank-ings try to search for statements from teach-ers and students that are related to their experi-ences at the universities. A promising additionlies in extending the search to include new plat-forms such as social media.The main contribution of this papers is to bringsentiment analysis tools in analyzing socialmedia content - especially microblogs- to actas a supportive indicator for evaluating univer-sities.In section 2, related work is presented. The pa-per concept of using social media as supple-mentary tool for evaluating university is dis-cussed in section 3. Afterwards, the papermethod used for data processing is presented insection 4. Later on, the results and discussionon using sentiment analysis for Twitter data inuniversity context are shown in section 5. Fi-nally, a conclusion and outlook are presentedin section 6.

2 RELATED WORK

This paper aims at investigating new indica-tors for universities evaluation and compari-son. Such indicators can be provided by ap-plying sentiment analysis concept and methodson the available data from social media. In theupcoming sections, the related work for usingsocial media platform at universities, evaluat-ing universities by university rankings and sen-timent analysis concepts are presented.

2.1 Social Media at Universities

Recently, social media platforms have becomea main medium for people to express theirdaily activities, reactions and emotions. Blogsand microblogs are the most common form ofsocial media [1]. Blogs (the abbreviated form

of Weblog) are informal sites on the world-wide web where users are used to post (i.e.publish) ideas, discussions, thoughts, etc. [2].While, microblogs are smaller blogs with shortposts up to a limited number of signs [3]. Onething they have in common is that they consistof entries that are listed in a chronologicallydescending order (i.e. the latest news is ontop). They are tools that enable discussions andcomments on information shared with otherusers. They are characterized by its dynamicand up-to-datedness.

Students use social media platforms and in-ternet on a daily basis, often times more fre-quently than other mass media such as news-paper or television. Therefore, they are expe-rienced in how to use social media [4]. Basedon this fact and the increasingly usage of so-cial media, it seems reasonable to use socialmedia in university environments for the pur-pose of engaging the social media tools in theteaching process, as well as for the purposeof analyzing opinions regarding university top-ics like teaching and learning. On universitylevel, microblogs are tools for simultaneouscommunication, knowledge management andpublication service. In formal and informalteaching and learning contexts, microblogs cansupport individual and cooperative communi-cation, knowledge management, reflection andfeedback processes.

Teachers use microblogs to encourage motiva-tion and participation of students. This is jus-tified by two features [5]: First, it raises in-teractivity of students and creates the oppor-tunity to implement social and team buildingaspects even in large classes. Second, it makesuse of the media usage of smartphones, tabletsand notebooks of today’s students in learn-ing contexts [1]. The fields of application ofmicroblogs in teaching scenarios can be di-vided into three parts: collaboration, feedback-giving/discussion and public scientific commu-nication. By adding the ability to share infor-mation and knowledge via social media, men-tioning Twitter specifically, collaborative workand learning is enabled with an open feedback-giving space.

ISBN: 978-1-941968-26-0 ©2015 SDIWC 50


Furthermore, general and personal questionsas well as lecture or seminar contents can bereflected. Twitter can be utilized to collectquestions and feedback of students and discussthem [6]. One of the interesting use cases is the“Twitter wall”, where the concept is realizedby projecting Twitter’s posts on a big screenduring the class time [7]. In addition to dis-cussing the most relevant questions at the endof the lecture, other questions and feedback canalso be answered after the lecture via Twitter[6].Besides the well-known applications of mi-croblogs in different teaching and learning sce-narios, semantic technologies are often put touse in order to analyze and evaluate automati-cally opinions made in social media, e.g. Twit-ter or Facebook. Brauer and Bernroider [8]conduct an international study analyzing theusage of Facebook within higher education in-stitutes in Germany, Austria and Switzerland.In this case, the social media strategy that isrelated to Facebook of selected universities isanalyzed. Another field of social media ana-lytics in universities is the analysis of studentsand teachers opinions which are made in e.g.Twitter or Facebook [1].

2.2 Evaluating Universities via UniversityRankings

The comparable analysis of universities world-wide has been established over the past fewdecades. National and international universityrankings try to compare and evaluate univer-sities through various criteria and try to makedifferences in quality measurable.According to a user study from the Centerfor Higher Education Development (CHE), themost common reason to make use of universityrankings is to acquire information about uni-versities current or potential status [9]. There-fore, the main goal of any university rankingis to process and present data in order to makecomparing universities with each other clearer.It also helps in the decision making process to-wards the future of teaching and learning envi-ronment based on the data collected and infor-mation provided [10].

Via comparing the available study programsand conditions of universities, the transparencyof performance is improved [11]. Universityrankings have to meet certain methodical stan-dards to be helpful for the decision makingprocess of students, as well as an orientationtool for university’s improvements. At first,university rankings have to be disciplinary be-cause universities are versatile and have differ-ent foci [11]. Another important requirementfor university rankings is being multidimen-sional. Thus, other indicators, such as timeta-bles, external funding or the state of the uni-versity’s library as examples, have to be put incomparison.On an international level, the most popularrankings in Germany are the “Ranking ofWorld Universities” by the magazine “TimesHigher Education (THE)”, as well as the “QSWorld University Ranking” [9]. The THE re-views research-led universities across all theircore missions in teaching, research, knowledgetransfer and international outlook using 13 in-dicators. The “QS World University Ranking”compares universities worldwide on a basis ofeight indicators. In addition to the state of re-search, publications and Nobel prizes are alsoemphasized in the QS. The QS ranking doesnot only compare universities on a global level,but also individually for each country or re-gion.Even though rankings have become an increas-ingly popular way to compare higher educationperformance and productivity, university rank-ings have been criticized over the recent yearsin three major aspects [12][13].First, most rankings measure universities aswhole institution. Thereby, only the averagequality of a university is measured. Individ-ual subjects are not taken into account. Basedon weighing single indicators a total value (i.e.“composed indicator”) is calculated [12].The second aspect relates to the way differentaspects and dimensions have been measured insome indicators, and the way the related datais collected. In total, some indicators (e.g.university reputation) have a higher influencevalue than some others. [12].

ISBN: 978-1-941968-26-0 ©2015 SDIWC 51


Third, the measurement of educational qualitysuch as the quality of teaching and learning orthe quality of student experience is underrepre-sented in the existing ranking [13].

2.3 Sentiment Analysis

Sentiment analysis is a method that analyzeshow opinions, reactions, impressions, emo-tions and perspectives are expressed in a lan-guage. Its algorithms can extract evaluative in-formation from large text databases and sum-marize it [14].

In order to analyze the opinion of people andcustomers, sentiment analysis appears as themain tool in different contexts. As an exam-ple, sentiment analysis has been used to mea-sure customers satisfaction via statements theycomment on a specific product they boughtor a service they were delivered [15]. It alsoappeared in the extend of detecting differ-ent opinions regarding political events such aselections [14].

Sentiment analysis methods are well developedin the domain of blogs and product reviews[16] [17]. Researchers have been working ondetecting sentiment in text via presenting dif-ferent algorithms for detecting semantic orien-tation [18].

In favor of producing meaningful informationfrom tweets, sentiment analysis has been alsoused [14] [15] [19]. Different features selec-tion techniques have been investigated, estab-lishing a comparison between different onessuch as n-grams, part of speech, lexicons, etc.[20]. Besides, different classifiers with theirlearning performance have been tested in dif-ferent contexts [21].

This paper applies the existing developed ap-proaches in sentiment analysis to microblog-ging platforms data such as Twitter in orderto explore complimentary resources for univer-sity evaluation and comparison.

3 TWITTER SENTIMENT ANALYSISFOR EVALUATING UNIVERSITIESIN GERMANY

This paper suggests that social media contentis a vital source for collecting feedback and re-actions on the daily events and activities thatis related to universities. To prove this hypoth-esis, a case study is established which evalu-ates the reactions and feedback from the so-cial media data that is related to nine universi-ties in Germany that are part of the TU91 Ger-man Institutes of Technology. The TU9 uni-versities are: RWTH Aachen University, TU2

Berlin, TU Braunschweig. TU Darmstadt, TUDresden, Leibniz Universitat Hannover, Karl-sruher Institut fur Technologie, TU Munchenand Universitat Stuttgart.

3.1 Data Collection

Out of many social media platforms, Twitter isthe most popular microblogging platform [22].Therefore, it was chosen to be the source of thesocial media data for this paper.Students in universities message about any-thing and everything in their day-to-day lives.Twitter users post about their reactions andfeedback in a form of microblogs, each is atext up to 140 letters that is called a tweet. Thetweets that are related to the TU9 form the dataset.The Twitter API3 enables different ways tosearch and filter the tweets. In the literature,different ways for extracting the tweets areadapted. For example, Pak and Paroubek [23]and Bifet and Frank [24] have extracted thetweets based on specific mentioned emoticons.Others, such as Davidov et al. [25], have ex-tracted the tweets based on a list of mentionedhashtags in the tweets.To collect tweets which are related to the TU9,all tweets which have a matching word froma list of keywords were extracted via Twitter

1TU9 is an incorporated society of the nine mostprestigious, oldest, and largest universities focusing onengineering and technology in Germany.

2TU: Technical University3API: Application Programming Interface

ISBN: 978-1-941968-26-0 ©2015 SDIWC 52


API. The keyword list contains combinationsof the universities names and titles.The data extraction script began running onOctober 1st, 2014 till March 31st, 2015. In theGerman academic calendar, this time span isthe entire winter semester of 2014/2015. Thescript catches every tweet that matches any ofthe keywords and saves it in a database. As aresult, the script collected 16488 tweets.The collected tweets were posted in both En-glish and German. Also, it included originaltweets and the re-posted (i.e. retweeted) ones.Table 1 shows how tweets are divided over theTU9 4.

Table 1. Tweets count for each of the TU9.4

University CountRWTH Aachen 8073

TU Dresden 4075Universitat Stuttgart 1014

TU Darmstadt 951Karlsruher Institut fur Technologie 767

Leibniz Universitat Hannover 450TU Braunschweig 445

TU Berlin 401TU Munchen 300

Table 2. Tweets count for each language.

Language CountGerman 12906English 3582

Considering the fact that German universities -especially the TU9 - rely on multiple languagesfor communication and enroll a large num-ber of international students, it makes senseto keep the tweets in both English and Ger-man. Table 2 shows the number of collectedtweets in each language. Retweets begin with”RT” and are mostly copies of other originaltweets, with some possible text at the begin-ning [26]. Still, retweets are useful to be con-sidered as they emphasize the statement andfacts included in the original tweet. Table 3shows the number of original and retweetedtweets in the data set.

4Few tweets belong to more than one university, thatleads to have the tweets summation larger than the totalcount of collected tweets.

Table 3. Number of original and retweeted Tweets.

Tweet Type CountOriginal Tweets 10189

Retweets 6287

3.2 Defining Tweets Sentiment

This paper’s approach classifies the sentimentof each tweet to be either “Positive” or “NotPositive”. This is known as a two way senti-ment classification [15].A “Positive” tweet refers to text that indicatesa positive statement regarding an event such asa lecture, class or activity that is related to oneof the TU9. A “Not Positive” tweet can be ei-ther a negative statement regarding an event,or a neutral one, such as an announcement oradvertisement regarding an event in the univer-sity.Adapting two way classification can be consid-ered as a limitation. Nevertheless, it is easierto process in the classifier learning step. Thesame approach was adapted by Go et al. [20]who consider that the “Not Positive” tweets areactually “Negative” ones, ignoring the neutralnature of some tweets. Pak and Paroubek [23]proved that adapting a three way classificationleads to bad performance which can be avoidedby the two way classification [21].

3.3 Tweets Sentiment Analysis Challenges

Dealing with social media as a source of infor-mation - especially microblogging platformssuch as Twitter - adds extra difficulties to thesentiment analysis process [19]. Tweets areplain text written in an informal manner andits processing face challenges such as:

• Length: Tweets have a limited text length,which is 140 characters. This forces usersto start using some common and uncom-mon abbreviations and phrases. As anexample, abbreviations such as OMG5,WTH6, DKDC7, TY8, etc. appears oftenin Twitter.

5OMG: Oh My God!6WTH: What the hell!7DKDC: Don’t know, don’t care8TY: Thank you

ISBN: 978-1-941968-26-0 ©2015 SDIWC 53


• Informality: Twitter is mostly used asa non-formal communication medium.This leads to many informal statementswhich probably contain errors such asmisspellings, unstructured sentences andslang. Informality may also infer sar-casm, which adds an extra layer of dif-ficulty in guessing the right sentiment ofeach tweet.

• Credibility: This paper’s approach ofgathering the tweets is based on a list ofkeywords. This does not guarantee thecredibility of who and what tweets aregenerated on Twitter. This leaves the pos-sibility that one anonymous user has gen-erated all the content about a specific uni-versity with different usernames, ratherthan the students.

• Data availability: collecting the right datais always a challenge, but having enoughdata is another critical issue. The tar-get data for this study is very specific,which can be problematic for collectingdata over a six-month period of time. Itis significant to note that more data leadsto more trusted results.

4 DATA PROCESSING

Figure 1 illustrates the main steps of the dataprocessing. The starting point is a set of tweetswhich was extracted via Twitter API. Based onsentiment analysis approach, a sentiment clas-sifier will be build by learning from previouslyannotated subset of tweets in order to classifythe rest of tweets. The classifier to be built willbe able to learn the defined sentiment: “Posi-tive” and “Not Positive”. The processing stepsare divided into three main steps: Tweets’ textfiltering, feature extraction, and sentiment clas-sification.

4.1 Tweets Text Filtering

As previously mentioned, tweets are informalsentences that have to pass through a filteringstage before it can be processed for the up com-ing steps. Filtering is the process of cleaning

Start

Collecting Data Twitter API

Filtering

Feature Extraction

5000 RandomTweets Selecttion

Sentiment Man-ual Annotation

Classifier Training Cross Validation

Sentiment Classification

Results

16488 Tweets

5000 Tweets

Training Set

4000 Tweets

The

rest

oftw

eets

Figure 1. Data processing flow

the tweets text removing all irrelevant text forthe sentiment classifier learning step. The fol-lowing are tweets filtering steps mentioned inthe order they were performed:

1. All text is switched to lowercase includingthose words which are completely capital-ized. Despite the fact that some users tendto emphasize specific words with capi-talization, this was not the general casewith the collected tweets. Many namesand sentences are found completely cap-italized indicating no emphasizing on themeaning. Taking the capitalization intoaccount in such cases might lead to falseresults. Therefore, all text is changed tolowercase.

2. All hyperlinks are removed. Tweetsmostly contain hyperlinks to other sitesand photos which does not contribute tothe sentiment of the tweet.

3. All mentioned usernames (identified bywords that start with @) are removed and

ISBN: 978-1-941968-26-0 ©2015 SDIWC 54


all hashed words with the # symbol are re-placed with the word itself. These specificsymbols and markups mentioning user-names or include hashed words that tag aplace, name, etc. are so general to con-tribute to a specific tweet sentiment.

4. The “RT” text which indicates a retweet isremoved.

5. Repeated lettered are filtered. Often, usersemphasize words by repeating letters suchas: “I am Happyyyy”. Go et al. [20] sug-gest to remove out repeated letters leav-ing only two of them. This also guaran-tees that words such as “cool” with origi-nal double letters are left unaffected.

6. Common emoticons are replaced withtheir semantic. Emoticons are often usedin social media language to indicate theusers’ emotions [23]. The found emoti-cons are classified as:

• Happy emoticons: “:)”, “:-)”, “:D”,“;)”, “;’]”, etc. which are replacedby “HAPPY FACE”.

• Sad emoticons: “:-(”, “:(”, “=(”,“;(”, “:[”, etc. which are replaced by“SAD FACE”

7. Negations are detected in the tweet. De-pending on the language, negation ap-pears in different forms. Accordingly, thesentiment of the words appear before andafter the negation are changed. For exam-ple, “I don’t like exams” is changed to “INOT do NOT Like exams”.

8. All words which do not start with a let-ter are removed. This eliminates all phonenumbers and dates included in the tweet.

9. Extra spaces and punctuation marks areremoved.

10. All stop words and keywords (includ-ing the universities’ names) are removedbased on the language of the tweet.

After these steps, every tweet text is left onlywith words that can play a role in indicatingthe sentiment of the tweet. Here is an exampleof a tweet after applying all filtering steps:

“I would tell you. That I loved you.If I thought ... � Boys Don’t Cry byGrant-Lee Phillips (at RWTH Bibliothek 2)https://t.co/qGoJm1bmV8”

“tell loved thought boys NOT do NOT crygrantlee phillips bibliothek”

4.2 Features Selection and Extraction

An important part of the sentiment analysisprocess is features selection. Features are thesentence properties that are analyzed in an at-tempt to correlate it to the tweet sentiment (i.e.“Positive” or “Not Positive”). A feature can bethe fact that the tweet contains a word, emoti-con, a combination of words, etc. The selectionof the features is very important as they act asthe input for the classifier in the next step.Different features selection approaches appearin the literature in the context of tweets senti-ment analysis. Some algorithms use unigrams(i.e. single words), some others use bigrams ortrigrams (i.e. two or three consecutive words,respectively) [20]. Also, many algorithms usepart-of-speech tags and lexicons [19]. Hash-tags and emoticons also appeared as featuresin some algorithms [25].Unigram features give a wide coverage for thetweet’s text. On the other hand, n-grams withn ≥ 2 show a higher ability to capture the sen-timent expression patterns. As larger as n gets,the sentiment is more specific. N-grams withn ≥ 2 are used to find out the domain specificlanguage. This is useful for this research, asthe aim is to build a set of vocabulary that isrelated to the universities context.Depending on the context, the chosen sizeof n-grams affects differently. For example,unigrams are a better choice than bigramswhen performing the sentiment classificationof movie reviews [16]. On the contrary, bi-grams and trigrams worked better for the prod-uct review polarity classification [27]. In gen-

ISBN: 978-1-941968-26-0 ©2015 SDIWC 55


eral, using only bigrams as features is not use-ful because the feature space becomes verysparse [20].The best settings of the tweets features has tobe determined. This paper adapts a combi-nation of uni- and bigrams features to benefitfrom the unigrams’ coverage of the data andthe bigrams’ ability to capture the sentimentexpression patterns. The same approach wasadapted by Pak and Paroubek [23].Using part-of-speech features leads to a dropin the sentiment classification performance.These results were proven by Go et al.[20] andKouloumpis et al. [19]. Therefore, the part-of-speech features were neglected.The sentiment of the emoticons can be part ofthe n-grams features. On the one hand, strip-ping out the emoticons from the tweets leadsthe classifier to learn from other words thatforms unigrams and bigrams [20]. On the otherhand, including the emoticons data showed aperformance improvement referring to the re-sults of Kouloumpis et al. [19].Based on the filtering step, emoticons are re-placed by their sentiment (i.e. “HAPPYFACE” and “SAD FACE”). In the features ex-traction step, they take part in forming the uni-grams and bigrams features of the tweets. Thisleads to a better performance for the classifier[20].

4.3 Sentiment Classification

Different classifiers have been presented in theliterature for Twitter sentiment analysis. Su-pervised classifiers are the focus. They requirea training set to be prepared forehead.The training set has to be annotated. Pak andParoubek [23] did this automatically based onthe fact that all tweets in their dataset containsemoticons. They labeled each tweet based onthe emoticon sentiment to be either “Positive”or “Negative”.For the training set, it got annotated manuallyby different people based on their own feelingwhether the tweet indicate a “Positive” or “NotPositive” sentiment. Automatic labeling wasnot possible at this stage as the tweets lack acommon feature for sentiment labeling.

Several classifiers can be used when it comesto the tweets sentiment analysis. Mainly,three common classifiers in the field of ma-chine learning have been used in the litera-ture: Naive Bayes classifier, Support VectorMachines (SVM), and Maximum Entropy.Naive Bayes and SVM have been compared byPak and Paroubek [23] and Go et al. [20] [21];Naive Bayes has performed better.Theoretically, Maximum Entropy performsbetter than Naive Bayes as it handles featureoverlap better. However, in practice, NaiveBayes showed better performance on a varietyof problems [20].Naive Bayes classifier is adapted by this pa-per’s approach. It is a common method fortext categorization. It appeared often for solv-ing the problem of determining the category orclass of documents that belongs to using wordfrequencies as the features.In machine learning, Naive Bayes classifier be-longs to the family of probabilistic classifiersbased on applying Bayes’ theorem with theassumption that features are conditionally in-dependence from each other given a specificclass.

P (s|f) = P (s).P (f |s)P (f)

(1)

Equation 1 shows the basic formula of Naivetheorem where s is the sentiment class (i.e.“Positive” or “Not Positive”) and f is a specificfeature. This equation computes the probabil-ity of having a tweet with the sentiment s whenit contains the feature f . It is calculated basedon the probability of having a specific senti-ment, probability of the feature existence in alltweets, and the probability of finding the fea-ture in the tweets that belongs to that specificsentiment.

5 RESULTS AND DISCUSSION

The data set holds 16488 tweets. Each tweetcontains a statement regarding a university ormore from the TU9 in Germany. For the train-ing set, 5000 tweets were chosen randomly andgot annotated manually by one sentiment eithera “Positive” tweet or “Not Positive” tweets.

ISBN: 978-1-941968-26-0 ©2015 SDIWC 56


From the 5000 tweets, 4000 tweets where cho-sen randomly divided equally between 2000“Positive” and 2000 “Not Positive” tweets.They are the input for the training step of theNaive Bayes classifier (see Figure 1).The results section evaluates three main as-pects of the presented method:

1. Measuring the classifier efficiency basedon the suggested filtering and features ex-traction steps.

2. Establishing a comparison between theTU9 based on each university’s tweetstrying to prove the hypothesis that socialmedia content may act as an indicator foruniversity comparisons.

3. Investigating the tweets sentiment ondaily basis for each university to obtainfeedback on different events and activi-ties.

Each is presented in the following sections.

5.1 Classifier Efficiency

Each tweet has been processed through the fil-tering step. Then, its unigram and bigram fea-tures were extracted. For the purpose of crosschecking validation, 25% (i.e. 1000) of thetraining set tweets were chosen randomly leav-ing out 3000 tweets for training. The 1000tweets are used for testing the classifier perfor-mance, which were excluded from the training.A script was written in Python using the Natu-ral Languages Tool Kit (NLTK) python library.The NLTK classifier was used for perform-ing this paper’s approach. Part of the resultsof the trained classifier are shown in Table 4.The table shows the most informative featureswhich were learned by the classifier. In otherwords, each listed feature was more found sev-eral times for the corresponding feature class.The results show that some of these featureshave a dominant effect on the classificationof the tweet. As an example, whenever theword “Pegida9” is mentioned, tweets are prob-ably “Not Positive” 27 more times than being

9Patriotic Europeans Against the Islamisation of theWest

Table 4. The most informative features learned by theNaive Bayes classifier.

Feature SentimentClass

×Times

Pegida Demonstrant Not Positive 27.0Gluckwunsch Positive 16.3Mittelschicht Not Positive 14.6

Ojeu Not Positive 13.0Pegida Studie Not Positive 13.0Berufstaetig Not Positive 12.3

Studie Pegida Not Positive 12.1Maennlich Jahre Not Positive 11.7

Ausgebildet Not Positive 11.0Ausgebildet berufsttig Not Positive 11.0

Schreiben Positive 11.0NOT gut ausgebildet Not Positive 11.0

Kritik Pegida Not Positive 11.0Klausuren Positive 10.3

Willkommen Positive 10.3Mittelschicht gut Not Positive 9.7

Herzlichen Positive 9.7Demonstrant mittelschicht Not Positive 9.7

Geheimdienst Not Positive 9.0Wunschen Positive 8.3

“Positive”. On the other hand, whenever theword “Gluckwunsch10” is mentioned, tweetsare probably “Positive” 16.3 more times thanbeing “Not Positive”.It can be also noticed that both unigrams andbigrams played a role as the most informativefeatures.The classier performance was verified withthe rest of the annotated tweets (i.e. 1000tweets). 73.6% of the tweets sentiment wereguessed correctly by the classifier. This resultis considered to be good considering the datasize and compared to the other developedclassifier by Pak and Paroubek [23] and Go etal. [20], which achieved around 80% successrate.

5.2 Tweets Sentiment for UniversitiesComparison

The rest of the tweets have been classified bythe learned classifier. Each tweet belongs toone sentiment class. Tweets were divided overthe TU9, showing how many “Positive” and

10Congratulations

ISBN: 978-1-941968-26-0 ©2015 SDIWC 57


RWTHAachen

TU Berlin TU Braun-schweig

TU Darm-stadt

TU Dresden LeibnizUniversitatHannover

KarlsruherInstitut fur

Technologie

TU Munchen UniversitatStuttgart

0

50

100

3,688(46%)

161(40%)

204(46%)

453(48%) 2,576

(63%)

176(39%)

166(22%)

178(59%)

411(41%)

4,385(54%)

240(60%)

241(54%)

498(52%) 1,499

(37%)

274(61%)

601(78%)

122(41%)

603(59%)

University Name

Twee

tsPe

rcen

tage

Positive Tweets Not Positive Tweets

Figure 2. Percentage of “Positive” and “Not Positive” tweets per university in the time span from October 1st, 2014 tillMarch 31st, 2015

“Not Positive” tweets each university had inthe specified time span (see Table 5).

Table 5. Classified tweets counts per university of theTU9.

University Posi-tive

NotPosi-tive

RWTH Aachen 4385 3688TU Berlin 240 161

TU Braunschweig 241 204TU Darmstadt 498 453TU Dresden 1499 2576

Leibniz Universitat Hannover 274 176Karlsruher Institut fur Technologie 601 166

TU Munchen 122 178Universitat Stuttgart 603 411

The number of the total tweets per universityvaried depending on how active Twitter usersare at that university. This makes it difficultto establish a comparison between the univer-sities. Nevertheless, to get an overview onhow positive the given feedback was by Twitterusers on each university, the percentage of the“Positive” tweets is considered for carrying outthe comparison. Results are shown in Figure 2.Karlsruher Institute fur Technologie had thehighest percentage of “Positive” tweets form-ing 78% out of its total 767 tweets. RWTHAachen University got the highest number ofpositive tweets with 4385 tweets. Meanwhile,TU Dresden had the lowest percentage of thepositive tweets (37%).The results might act as an indicator on howthe higher education environment at each uni-

versity is perceived by Twitter users. Such in-dicator which is supported by the social mediacontent can play a role in enhancing the univer-sities rankings.

5.3 Daily Scale Tweets Sentiment

The next goal is to build a more detailed eval-uation by having a closer look into the tweetsentiments regarding daily events and activi-ties at each university. The classified tweetsare divided over time on daily basis, where the“Positive” and “Not Positive” tweets count andpercentages are considered.In order to present the case, the two universitieswith the highest number of tweets were cho-sen, namely: RWTH Aachen University andTU Dresden.Figure 3 shows how RWTH Aachen Univer-sity tweets are spread over the six month period(i.e. winter semester 2014/2015). The uppertwo figures show the number of “Positive” and“Not Positive” tweets respectively. The thirdfigure shows the ”Positive” tweets percentageon each day.The “Positive” tweets in RWTH Aachen Uni-versity data varied over the semester in differ-ent frequencies. The graph in Figure 3 cantrack when and what events or activities are in-teresting for the students.Few local maxima points are investigated. Thetweets at each point were extracted and wordfrequencies are analyzed. For RWTH AachenUniversity, the highest “Positive” tweets count

ISBN: 978-1-941968-26-0 ©2015 SDIWC 58


0

20

40

60

80 Erstsemester (First Semester)Wissenschaftsnacht (Science Night)

Beliebtesten Unis (Fav. Uni)

Posi

tive

Cou

nt

0

20

40

60

Not

Posi

tive

Cou

nt

Oct 1st6th Nov 1st Nov 14th Dec 1st Jan 1st Feb 1st Feb 10th Mar 1st Apr 1st

0

50

100

Time Period

Posi

tive

Twee

tsPe

rcen

tage

Positive Not Positive

Figure 3. RWTH Aachen University tweets sentiment on daily bases over the period from October 1st, 2014 till March31st, 2015

occurred on October 6th, 2014, when the wordfrequency analysis shows that many users thattweeted were excited about the beginning ofthe new semester. Table 6 shows the most fre-quent words appeared in that day’s tweets. OnOctober 6th the orientation sessions and wel-coming event took place, in which many newand old students were tweeting “positively”about it.Looking at other points similarly, on Novem-ber 14th, 2014 another peak was experiencedwhere the event of the “Wissenschaftsnacht”(Science Night) took place. It is an annualevent where students from different institutespresent their work and achievements. Manyusers tweeted about the event, and the activitieswhich took place at it. One more example is onFebruary 10th, 2015, when the RWTH AachenUniversity was chosen as one of the top three“most favorite” universities in Germany.The same analysis steps were applied to TUDresden tweets. In total, TU Dresden tweetsexperienced the most of the “Not Positive”sentiments. Figure 4 shows the tweet senti-ments on a daily basis over the semester. The

Table 6. Most frequent words which were mentioned inthe “Positive” tweets for RWTH Aachen University onOctober 6th, 2014.

Word TranslationErstsemester First Semester

Heute TodayErsti Newbie

:) :)Alle AllViel A lot

Filmstudio FilmstudioWunschen Wish

neuen NewErstis Newbies

Semesterstart Semester StartBietet Offers

Herzlich [Willkomen] Warmly Welcomed

maximum number of the “Not Positive” tweetsappeared on January 14th, 2015. As shownin Table 7, the word frequency analysis indi-cates that many tweets were related to “Pegidademonstrations” that took place on January2015 in the city of Dresden. This was a rea-son for many users to tweet “Not Positively”about the activities that happened regardingthese demonstration.

ISBN: 978-1-941968-26-0 ©2015 SDIWC 59


0

20

40

60

Posi

tive

Cou

nt

0

200

400

600

Pegida

Not

Posi

tive

Cou

nt

Oct 1st Jan 1st Jan 14th Apr 1st

0

50

100

Time Period

Posi

tive

Twee

tsPe

rcen

tage

Positive Not Positive

Figure 4. TU Dresden University tweets sentiment on daily bases over the period from October 1st, 2014 till March 31st,2015

Table 7. Most frequent words which were mentioned inthe “Positive” tweets for TU Dresden on January 14th,2014.

Word Translation CountPegida Pegida 1095Studie Study 895

Mannlich Male 243Mittelschicht Middle Class 196Demonstrant Demonstrator 187

Demonstranten Demonstrators 181Gut Good 168

Typische Typical 158Typischen Typical 155

Such analysis for the daily tweet sentiments inthe context of higher education can give an in-sight over what activities and events which at-tract the attention of the Twitter users, and howthey react based on it (i.e. positively or not).This analysis can be used by the university’sadministration to gather feedback. Based on it,actions can be delivered.

6 CONCLUSION AND OUTLOOK

Students nowadays use different social mediaplatforms including Twitter to express their re-

actions, and tell about their daily activities.Such platforms can be a vital source for opin-ion mining related to the universities.Standard university rankings are quitewidespread in order to compare and evaluateuniversities. University rankings requirecomplementary sources specially those whichidentify statements from students and teachersexpressing their experiences.This paper established a case study to provethe hypothesis that Twitter sentiment can po-tentially support the universities ranking sys-tem by analyzing posted statements and opin-ions of students and teachers in higher educa-tion institutions context.The case study used sentiment analysis meth-ods to analyze 16488 collected tweets aboutthe TU9. 4000 annotated tweets between “Pos-itive” and “Not Positive” were used to train aNaive Bayes classifier. The classifier perfor-mance achieved 73.6% success rate. The clas-sifier was used to guess the sentiment for therest of the collected tweets.The tweets sentiment was analyzed on two dif-ferent scales. First, the percentage of “Posi-

ISBN: 978-1-941968-26-0 ©2015 SDIWC 60


tive” tweets was calculated to establish a com-parison between the TU9. Second, the tweetswere analyzed on daily basis for the RWTHAachen and TU Dresden in order to discoverthe communicative topics and events at the uni-versities.The analysis established a new approach of us-ing social media in comparing universities.As a future improving steps, there is a lot ofroom in improving the classifier performanceregarding the filtering, features selection andchosen classifier type. The sentiment classifi-cation can be extended to include a third neu-tral class. Also, the tweets can be classifiedbased on topics to give a better insight on whatattracts Twitter users in the context of highereducation institutes.

REFERENCES

[1] Ilona Buchem, Ralf Appelt, Sascha Kaiser, San-dra Schon, and Martin Ebner. Blogging undmicroblogging-anwendungsmoglichkeiten im bil-dungskontext. booklet, 2011.

[2] Danah Boyd. A blogger’s blog: Exploring the def-inition of a medium. Reconstruction, online book-let, 6(4), 2006.

[3] Akshay Java, Xiaodan Song, Tim Finin, and BelleTseng. Why we twitter: understanding microblog-ging usage and communities. In Proceedings ofthe 9th WebKDD and 1st SNA-KDD 2007 work-shop on Web mining and social network analysis,pages 56–65. ACM, 2007.

[4] Harald Gapski and Lars Graßer. Medienkompe-tenz im web 2.0–lebensqualitat als zielperspektive.Praxis Web, 2:11–34, 2007.

[5] Julia Hisserich and Jasmin Primsch. Wissens-management in 140 zeichen: Twitter in derhochschullehre. webSquare, online booklet, 2010.

[6] Michael Kerres and Annabell Preussler. Sozialenetzwerkbildung unterstutzen mit microblogs(twitter). Handbuch E-Learning, Koln: DeutscherWirtschaftsdienst, online booklet, 2009.

[7] Jana Herwig, A. Kittenberger, M. Nentwich, andJ. Schmirmund. Microblogging und die wis-senschaft. das beispiel twitter. Steckbrief IV imRahmen de Projekts ”‘Interactive Science”’ ITA-Reports, 54-4, 2009.

[8] Claudia Brauer and Edward WN Bernroider. So-cial media analytics with facebook-the case ofhigher education institutions. In HCI in Business,pages 3–12. Springer, 2015.

[9] Isabel Roessler. Hochschulentwicklung, was war?was bleibt? was kommt? 15 jahre erfahrungen mitrankings und indikatoren im hochschulbereich, ar-beitspapier nr 167. booklet, 2013.

[10] Carmen Lebherz, Cordula Mohr, Matthias Hen-ning, and Peter Sedlmeier. Wie brauchbar sindhochschul-rankings? eine empirische analyse. InHochschullandschaft im Wandel., pages 188–208,2005.

[11] Sonja Berghoff, Gero Federkeil, Petra Giebisch,Cort-Denis Hachmeister, Detlef Muller-Boling,and Daniel Rolle. Che-hochschulranking. vorge-hensweise und indikatoren, arbeitspapier nr 119.booklet, 2006.

[12] Gero Federkeil. Internationalehochschulrankings–eine kritische bestand-saufnahme. Beitrage zur Hochschulforschung,35(2):34–48, 2013.

[13] Ellen Hazelkorn. World-Class Universities orWorld Class Systems?: Rankings and Higher Edu-cation Policy Choices. Book of Dublin Institute ofTechnology, 2013.

[14] Diana Maynard and Adam Funk. Automatic de-tection of political opinions in tweets. In The se-mantic web: ESWC 2011 workshops, pages 88–99.Springer, 2012.

[15] Apoorv Agarwal, Boyi Xie, Ilia Vovsha, OwenRambow, and Rebecca Passonneau. Sentimentanalysis of twitter data. In Proceedings of theWorkshop on Languages in Social Media, pages30–38. Association for Computational Linguistics,2011.

[16] Bo Pang and Lillian Lee. Opinion mining and sen-timent analysis. Foundations and trends in infor-mation retrieval, 2(1-2):1–135, 2008.

[17] Bo Pang and Lillian Lee. A sentimental education:Sentiment analysis using subjectivity summariza-tion based on minimum cuts. In Proceedings of the42nd annual meeting on Association for Computa-tional Linguistics, pages 271–279. Association forComputational Linguistics, 2004.

[18] Peter D Turney. Thumbs up or thumbs down?:semantic orientation applied to unsupervised clas-sification of reviews. In Proceedings of the 40th

ISBN: 978-1-941968-26-0 ©2015 SDIWC 61


annual meeting on association for computationallinguistics, pages 417–424. Association for Com-putational Linguistics, 2002.

[19] Efthymios Kouloumpis, Theresa Wilson, and Jo-hanna Moore. Twitter sentiment analysis: Thegood the bad and the omg! in Proceedings of theFifth International AAAI Conference on Weblogsand Social Media (ICWSM-2011), pages 538–541,2011.

[20] Alec Go, Richa Bhayani, and Lei Huang. Twittersentiment classification using distant supervision.CS224N Project Report, Stanford, 1:12–18, 2009.

[21] Alec Go, Lei Huang, and Richa Bhayani. Twittersentiment analysis. Final Project Report, 2009.

[22] Farzindar Atefeh and Wael Khreich. A survery oftechniques for event detection in twitter. Compu-tational Intelligence, 31:132–164, 2013.

[23] Alexander Pak and Patrick Paroubek. Twitter asa corpus for sentiment analysis and opinion min-ing. In In Proceedings of the Seventh Conferenceon International Language Resources and Evalua-tion, pages 1320–1326, 2010.

[24] Albert Bifet and Eibe Frank. Sentiment knowledgediscovery in twitter streaming data. In DiscoveryScience, pages 1–15. Springer, 2010.

[25] Dmitry Davidov, Oren Tsur, and Ari Rappoport.Enhanced sentiment learning using twitter hash-tags and smileys. In Proceedings of the 23rdInternational Conference on Computational Lin-guistics: Posters, pages 241–249. Association forComputational Linguistics, 2010.

[26] Amy Mollett, Danielle Moran, and Patrick Dun-leavy. Using twitter in university research, teach-ing and impact activities. in Impact of Social Sci-ences, online booklet, 2011.

[27] Kushal Dave, Steve Lawrence, and David M Pen-nock. Mining the peanut gallery: Opinion extrac-tion and semantic classification of product reviews.In Proceedings of the 12th international confer-ence on World Wide Web, pages 519–528. ACM,2003.

ISBN: 978-1-941968-26-0 ©2015 SDIWC 62


Documents

Sentiment Analysis of Social Media for Evaluating Universities · Sentiment Analysis of Social Media for Evaluating Universities ... Twitter as one of the most popular microblog-