User research through codeLanguage analysis of Amazon reviews of fitness trackers

Coding for Humanities | Prof Ishizaki Fall 2016Shruti AdityaChowdhury

Proposal 1

Fitness tracker manufacturers

Technology magazines

Academic journals User reviews

Analyse the use of language across different source• Number of words (complexity of argument)• Sentiment analyses to find variance within a source• Compare the nature of the content (no clue of what or how)

Proposal 2

• What features people think are important• How attitudes change over time• Possibly creating customer segments

User reviews Sentiment analysis

Appraisal framework

(Martin & White)

Pivot

Using the Attitude aspects of the Appraisal framework• Manually categorize words as affect, judgement or appreciation (create a dictionary)• Find words that correspond to the sub-categories and study how they are used in

reviews• See if there are co-relations between the use of ‘Attitude’ words and number of stars

Appraisal framework(Martin and

White)

Engagement

GraduationAttitude Judgement

Affect

Appreciation

Pivot

Categorizing words was time consuming and not helpful – needed context to understand

Bottomline• It’s very difficult to do since it involves analyses of text by word, sentence and

paragraph.

Version 1

Helping people find the ‘right’ fitness tracker for them – based on what people have written • Create a database of keywords - features and synonyms of the features• Parse reviews for keywords• Extract sentences• Analyze sentiment• Normalize the score with the star rating• Ask for user input and return the best option

User reviewsSentiment

analysis per product, review,

line, featureTop 7 products

Master features list

Related key words

Version 2

User reviews

959 pages

Sentiment analysis per

product,review and line for words related to

Master features list

Top 7 products Related key words

FeaturesWords related to behaviour

changeBehavior change

Details of words related to changeCode name List of wordsstate of mindintent intent,intention,intended,intending,mean, meant,meaning,hope,hoped,hopinghealth health,healthy,fit,fitness,strength,strong,stronger,weighthabit habit,routine,practice,practiselifestyle lifestyle,regimebuy buy,bought,purchased,gotbecause cos,because,since,therefore,hencemotivate motivate,motivates,motivated,motivating,motivationalaware aware,awareness,notice,noticed,concious,conciousness

changetime since,earlier,now,used toamount more,less,increase,decrease,increased,decreased,increasing,decreasing,samechange change,changing,changed,turned,transformed,transformation,improve,improved,improving,worse,worsen,worseningnotice notice,noticed,noticing

usegoal goal,aim,target,aimed,targetted,objectiveactivity yoga,tai-chi,meditation,aerobic,weight,weights,lift,lifting,lungebehavior behavior,behaviour,behaviors,behaviours

relationshipencourage encourage,encouragement,encouraging,encouraged,encouragementshelp help,helps,helped,helping

emotions happy,sad,angry,guilty,unhappy

Overview of code

1. Creating Sql databases instead of files – so that the data can be stored and analyzed (through queries) in multiple ways

2. Define functions for1. Finding the number of pages per product2. Getting different products’ reviews3. Getting all reviews per star rating4. Splitting the reviews into sentences5. Doing sentiment analysis6. Reading the database table for features and product7. Reading the database table for features and line 8. Populating the database tables

3. Running the code and populating the database

Findings

• Positive and negative reviewsPositive – change and motivation. Negative – features (inaccuracy), customer service, product design, app

• Time periodPeople use pedometers longer than fitness trackers – a year v/s 3 monthsKeywords of ‘hours, days, weeks, months’ was found regarding fitness trackers and ‘months’, ‘years’ with pedometers

• Language usedMuch more extreme than with one on one interviews – lack of judgement? Altruistic motive?

• CompetitionCompetition was perceived as positive

• DisplayPeople like seeing a detailed display with graphics and numbers rather than a general sense of activity

• GiftPeople buy them for others and in pairs

• Supplement to recoveryPeople use it to recover from illness – chemotherapy, knee replacementUsage among older people is high

• Emotion scaleVery happy > Happy > Sad > Not happy > Angry

Reflections

• Key wordsBalance between the right word (“Change”) and too many variations

• Quantitative data is not as helpful for design research “It motivates me to walk more.” x 50 times

• On Amazon – Have to rate before you write a review which may impact how people write

Reflections

• Limitations of sentiment analysis - “I've used it on my runs, lifting, hikes” gets a neutral score when it should be positive

• Helped me discover new features that were important to people but not mentioned on the websites – flower and water tracking

• A lot of first-hand data but still biased – how and why people write it, how I wrote the key words and how I did the qualitative analysis

Points of investigation – Features

CaloriesSteps TimeDistance Sleep Silent alarm

Multiple sports

Goals ActivitiesCardio Backlight CallText

Waterproof RemindersActivity switch LEDCalendar Timer

Points of investigation – Behavior change

ChangeState of Mind RelationshipUse Emotion