22
HAL Id: hal-03132233 https://hal.archives-ouvertes.fr/hal-03132233 Submitted on 4 Feb 2021 HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Evaluating Users’ Experiences (UX): a case study approach to improving I-Doc UX Design Samuel Gantier, Michel Labour To cite this version: Samuel Gantier, Michel Labour. Evaluating Users’ Experiences (UX): a case study approach to improving I-Doc UX Design. I-docs: The Evolving Practices of Interactive Documentary„ 2017, 978- 0-231-85107-7. hal-03132233

Evaluating Users' Experiences (UX): a case study approach

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Evaluating Users' Experiences (UX): a case study approach

HAL Id: hal-03132233https://hal.archives-ouvertes.fr/hal-03132233

Submitted on 4 Feb 2021

HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.

Evaluating Users’ Experiences (UX): a case studyapproach to improving I-Doc UX Design

Samuel Gantier, Michel Labour

To cite this version:Samuel Gantier, Michel Labour. Evaluating Users’ Experiences (UX): a case study approach toimproving I-Doc UX Design. I-docs: The Evolving Practices of Interactive Documentary„ 2017, 978-0-231-85107-7. �hal-03132233�

Page 2: Evaluating Users' Experiences (UX): a case study approach

Pre-print version, chapter from I-docs: The Evolving Practices of Interactive Documentary, edited by Judith Aston, Sandra Gaudenzi and Mandy Rose, Columbia University/Wallflower Press, New York (USA), 2017, p. 101-116. ISBN: 978-0-231-85107-7

1

EvaluatingUsers’Experiences(UX):acasestudyapproachtoimprovingI-DocUXDesign

SamuelGantierandMichelLabour,UniversityofValenciennes(France)

Keywords:UXDesignEvaluation,InteractiveDocumentary(i-doc), Informational architecture, User-centred design,Usability,Sense-making.Introduction

Hundreds of interactive documentaries (i-doc) havebeen published on the Internet in the 2005-2015 decade.Despitetheapparentsuccessofi-docs(Gaudenzi2013:247,Gantier2012), itappears thata greatmajority of themdonot keep theirusersbeyonda fewminutes1.Howcanoneexplainthis?Oneresponsetothequestionisthati-docsareparticularly challenging as an emerging media mixing thelanguage of documentary filmswith that of computationaland interactive ‘computerisedmedia’ (Jeanneret 2007). Ineffect, an i-doc is of a hybrid nature. It merges a graphicinterface with an audiovisual flow (Gantier 2016). In thiscontext, the aim of our study was to establish an i-docevaluationmethodologyofusers’experiences.

To present the study, first, archetypal ‘Model User’drawn from the minds of professionals (filmmakers,producers, web developers, graphic artists, televisionbroadcasters) involved in the designing of the i-doc ‘B4,Fenêtressurtour’2(2012,‘B4,Windowsofablockofflats’)isportrayed. Second, we describe a novel tri-dimensionalevaluation model. Third, in an empirical case study, weexamine how two user groups experienced differentviewing modes, usability problems and the informationalarchitectureofB4.Fourth,weexaminehowusers’soughtto

Page 3: Evaluating Users' Experiences (UX): a case study approach

2

make sense of their i-doc experience. Finally, the gapbetween the Model User and Empiric User is advanced,alongwithrecommendationstoimprovei-docdesign

1 ArchetypalModelUser

The study began when one of the researchers of thechapter, SamuelGantier,was employedasa film-editoronthe six-month long design of the i-doc B4. It provided anopportunity toobservehow thedesign team imagined thearchetypal Model User3 (Gantier and Labour 2015). TheidentificationofaModelUsereffectivelycrystallisesasetofsociotechnical negotiations (Akrich 2006) and semio-pragmatics elements (Jeanneret and Souchier 2005) thatconditions both the informational architecture and theinteractiondesignofthei-docandits‘film-interface’4.

1.1 InformationalarchitectureofB4B4 is an adaptation of the experimental novel of

GeorgesPerec,LaVie,moded’emploi(1978),whichexploresthe fictional possibilities of algorithmic logic (cf. Hartje,Magné and Neefs 1993). B4 re-interprets Perec’sexploration on how 12 inhabitants of a virtual, 12-floor,inner-cityblockof flats feelabout theirneighbourhood.Attheheartofthei-docare96‘documentaryhaïkus’oftwotothreeminutes,totalling180minutes.

If one views, for example, six videos (for about 15minutes), and if one takes into account all the possiblesequencesofwatchingthehaïkus,usershaveachoiceof667billion5 different viewingpaths. TheB4 architecture offersthreenavigationmodes.- A vertical viewing mode presents a series of 12

charactersinaspecifichaïkuspattern.Themodeisfoundbyclickingonthecolumnsofthevirtualblockofflats,orbyclickinghaphazardlyonthedécor.

- AhorizontalviewingmodefocussesoneachcharacterofB4.Usersclickontherowsofwindowsofthebuilding.

- Atransversalviewingmodeoffersthedifferentthemesofthevideos.Thisissituatedatthebottomofthescreen.

Page 4: Evaluating Users' Experiences (UX): a case study approach

3

Figure1.B4informationalarchitecture

(translatedfromFrench)

Users can switch on or off the default backgroundneighbourhood sounds of children playing and passingtrains.

1.2 InteractiondesignanduserchoiceDuring the participant-observation of the B4 design

team (see in the Introduction, above), a central issue wasdecidingwhichactionsshouldbesystem-driven,andwhichshouldbeuser-driven.Thisshowshowwhenviewingani-doc, users are placed in a dialectical tension between thecognitive ease, by letting the i-doc system decide what iswatched, and thecognitive effort of lettingusersdecide as‘spec-actors’(BarbozaandWeissberg2006)whattowatch.User-driven decisions depend, however, on knowing fromwhatoptionscanbechosen.

Regardingthequestionofprovidinguserswith(choicegiving) ‘clickable’ possibilities, Serge Bouchardon (2009)distinguishes between ‘conventional engagements’ (wheretheergonomicsofasystemseek toincrease thecoherenceofthenarrative)and‘non-conventionalengagements’(usersdonotfindwhattheyexpect).AccordingtoYvesJeanneretandEmmanuelSouchier (2005)whenusers findconfusingon-screenfeatures,theydisengage.

Page 5: Evaluating Users' Experiences (UX): a case study approach

4

In thiscontext, theB4design teamdifferedabouthowmuch leeway to give tousers. Short of conducting tests tofind out what users thought about the B4 design, the in-teamdiscussionsfocusedaroundthreekeypoints.

1. The author-director and the graphic artistwanted to giveusersminimalguidance.Userswereexpectedtodiscoverinanopen-ended,trial-and-errorapproachhowthei-docwasorganised via a series of non-conventional engagements(Bouchardon2009)

2. The producer and development engineers pushed for asemi-guideduserapproach.Theyarguedforvisualdecision-aiding graphics that would suggest how the videos wereorganised.Theyproposed thatwhenauser clicks inside awindowoftheblockofflats,thefloors(horizontalline)andcolumns (vertical line) of the building simultaneously andmomentarily light up to create an intersecting cross. Thisapproachsoughttoreduceusercognitiveeffortasawaytolimitdisengagement.

3. The department director of France Télévisions - NouvellesÉcritures (co-producerofB4)arguedtoembedestablishedweb design conventions. This design decision can beconsidered as providing conventional engagement(Bouchardon2009).Itreinforcedthedecisiontolightupthehorizontalandverticallinesoftheblockofflats.

1.3 ModelUser’sviewingpathDuring the official launch of B4, its author-director

createda39-minutefilmfocussedexclusivelyontheverticalmodeofviewingB4.Thefilmrepresentsaneffectiveportrayof a Model User. In this light, it would seem useful tocomparetheverticalviewingmodeB4ModelUsertoadata-basedEmpiricUserinferredfromastudyofusers’declaredexperiences.

2 Evaluationofusers’experiences

2.1 ResearchmethodologyFocussingonuserexperiencecanhighlightordiminish

different aspects of an i-doc depending on the domain ofreference(technological,ergonomical,emotional,cognitive,aesthetical, semio-pragmatic, etc.). In interaction design,

Page 6: Evaluating Users' Experiences (UX): a case study approach

5

notably in the ‘new documentary writing’6 domain, userexperienceisoftendescribedas‘anindividual’sperceptionsof his/her interaction with a product, service, orenvironment’ (Drouillat 2013). For researchers such adescription lacksprecision,hence limiting itseffectiveuse.It is thus necessary to explain what is meant by ‘userexperience’ in the B4 context, namely concerning theevaluationdimensions.

Followingtheworkonthemetricsofinterfaceusability(e.g. Baccino, Bellino and Colombi 2005) and multimediadocuments (e.g. Huart, Kolski and Bastien 2008), theempiricalanalysisofB4users’experiencewere focusedonthesystemic7interactionofthreedimensions(seeFigure2,below):1) Viewing frame. Given that i-docs are relatively ill-

defined (Gaudenzi 2013: 26), viewers do not knowwhat to expect or what to do. This can negativelyimpactonestablishingtheappropriateviewing‘frame’8toenableusers’adapttheirinterpretive‘beliefsystem’to a document (Pignier and Drouillat 2005). In thislight, one hypothesis of our hypothetico-deductivestudy is that an i-doc viewing frame involves asimultaneous ‘documentarising’ (Odin 2011,) and‘ludic’viewingmode(Genvo2013,seebelow).

2) Usability. This is described as the ‘degree to which aproduct canbeusedbyspecifically identifiedusers toattain adefined objectivewith efficacy, efficiency andsatisfaction in a given context of use’ (Bonnier 2013:147).Forani-doc,usabilitycanbeseenastheextenttowhichafilm-interfacebecomesmeaningfulforusersinagivenreceptioncontext.

3) Sense-making. This draws on the work of researcherslike Brenda Dervin and Patrica Dewdney (1986), andmorerecentlythatofMichelLabour(2011,seebelow).Sense-making research attempts to understand theconstruction of context-bound problem-solvingconstructs.Figure 2, below, summarises the tri-dimensional B4

evaluation model based on mutually reinforcing (viaiterativeloops)top-downandbottom-updynamics.

Page 7: Evaluating Users' Experiences (UX): a case study approach

6

Figure2.Thesystemictri-dimensionali-doc

UXevaluationmodel

TheModel formally identifies threekeydimensions inanapparenttop-downlinearlogic.Inpractice,however,thedimensions function as an interdependent emergingfunctionalwhole.An evaluationof i-docusage should thustakeintoaccounttherealitiesofactualusewhichgobeyondformally identified ‘dimensions’ conceived for researchpurposes.

2.2 WorkinghypothesesAn advantage of a quantitative approach, like web

metrics,isthatitindicatesthefrequencyandthedurationofvisits to i-docs, but it is not enough. We contend that aqualitative and empirical experimental approach is alsoneededtounderstandhowdifferentusers‘makesense’ofi-docs.

In this context, our evaluation was trialled in a pre-studythatcrystallised fourworking ‘falsifiablehypotheses’(Popper 2007 [1935]) of users’ experiences, taking inconsideration the three dimensions identified earlier (seeTable1,below).

Dimension 3. Sense-making processes based on:users' sense-making constructs link up different

meaningful elements into a coherent whole

Dimension 2. Usability based on :'meaningful' signs emerging from users' interaction with

the film-interface

Dimension 1. Viewing frame based on :a pragmatic reception context that induces documentarising and ludic viewing modes

Page 8: Evaluating Users' Experiences (UX): a case study approach

7

Userexperienceevaluated

Underlyingconcepts(seebelow)

Workinghypotheses(H)

1.Viewingframe

Ludicanddocumentarisingviewingmode.

H1:Asuccessfulviewingframe‘balances’theludicanddocumentarisingviewingmodessuchthatbothemergesimultaneously.

2.Usability

Affordanceandgo-betweensigns.Architectureoftheinformationalandparatext.

H2:Notgraspingsignsembeddedinafilm-interfacecreateproblemsofusability.H3:Thepresenceofdecision-aidingparatextsincreasestheuseofdifferentviewingmodes.

3.Sense-makingprocesses

Sense-makingconstructs,Senselessness,Sense-makingbreak.

H4:Thetypeofviewingmodeimpactsonsense-makingprocesses.

Table1.HypothesesregardingUsers’Experiences

These hypotheseswere tested by using a set researchmethod.

2.3 ResearchmethodIn 2014, 12 users, aged from 20 to 30 years, were

recruited on the basis that they had never met theresearchers, before the study, and were media and webliterate. Users were randomly divided into two equalgroups: A and B. Themaindifference between the groupswasthatonlyGroupBreceivedadecision-aiding‘paratext’(explicative text about a text) that explained theinformationalarchitectureofB4inthefollowingway:‘Thei-docgivesyouthepossibilityofmeeting12inhabitantsofaninner-cityblockofflats.Asyoucanseefromthepicture(seeFigure1,above)youcanview96videosinthreeviewing

Page 9: Evaluating Users' Experiences (UX): a case study approach

8

modes:Maincharactersmode,Typeoffilmsmode,ThemesofB4mode’.(translatedfromFrench)

Before viewingB4, groups A (withoutparatext) andB(withparatext)weregiventhesameverbalinstructions:‘IwouldlikeyoutodiscoverthesiteB4,WindowsofaBlockofFlats. Have you already seen theweb site? (If the answer is‘Yes’,theuserisaskedtoleavetheroom).Feelfreetoviewthesiteinanywayyouwish.Afterwards,Iwouldliketospeaktoyouabouthowyouviewedthesite’.(translatedfromFrench)Viewers spent about 30 minutes viewing the i-doc (seeTable4,below).Inordertomonitorusers’viewingpaths,atracking software of users’ clicks was installed oncomputers used in the experiment. After the viewingsession,usersagreedtofillinaself-evaluatingVisualAnalogScale (VAS) questionnaire (preferences are indicated on asliding scale of 0-100) and participated in a semi-guidedinterviewabouttheirB4experience.

3 EmpiricUser

3.1 Viewingframe(Hypothesis1)

DocumentarisingviewingRogerOdin(2011)defines‘documentarisingviewing’as

a user interpretative (‘reception’) skill that gives a text its‘documented value’. This rests on context-bound semio-pragmatic elements allowing users’ to infer thedocumentaryvalueofatextfromitsviewingsituation.Onefunctionofthedocumentarisingmodeisthatitallowsuserstoestablishwhoisresponsibleforthediscursiveandmoodaspects of a text. In this way, users can symbolicallyquestion issues of ‘identity of action and of truth’ (Odin2011:56).Theviewer is thus in aposition to criticize the‘enunciator’(theonewhoappearstoberesponsibleforthediscursive and mood aspects of a text). In short, thecredibility of a documentary is never guaranteed as itinvolvesauser’sinterpretativeprocess.‘Attheendtheday,thei-docsaysthereisnotonesolewayofspeakingabout inner-cities. So I said tomyself,why is therenoinhabitantinadifficulteconomicsituationinthefilms?Isthis a deliberate choice of the filmdirector? Iwonder if the

Page 10: Evaluating Users' Experiences (UX): a case study approach

9

situation is not a bit distortedbecause the i-doc only showsmoreor lesswell-off inhabitants. Inmyopinion, thedirectorshouldhaveshownwhat‘really’exists,thatistosaythesocio-economic reality of the inner-city block of flats’ (interviewextractofUser12B9).Our study showed that overall, groups A and B accepted,without question, that the characters really lived in aParisian inner-city. From this, one can infer that userssuccessfully adopted a documentarising viewing mode inperceivingtheinhabitantsofB4ascredibleenunciators.Inthis sense, the B4 characters and the users shared acommon communication space based on an underlying‘agreement of trust’ (Soulez 2004). This agreement is theoutcomeofadialecticrelationshipbetweenwhattheauthorstates about the world and what the viewer thinks of theworld.GuillaumeSoulez (2004)describes this situationasthe inevitable ‘communicational challenge’ of adocumentaryfilmdirectorwantingviewerendorsement.

LudicviewingFor Sébastien Genvo (2013) the idea of a ludicmode

involves playability, a ludic ethos, and aModel Player thatreinforces user involvement. By ludic ethos is meant ‘astructure that seeks topersuade its user that ‘this is play’(Genvo 2013: 134) in echo to Gregory Bateson (2000[1954]:178).

When users were asked to self-evaluate (using a VASquestionnaire,seeabove)theirludicattitudewhenviewingB4, they indicated an overall average level of 73 per cent(Table2,seebelow).The44percentdifferencebetweenthehighest (92 per cent) and the lowest score (48 per cent)indicates that users’ experiences of B4 was in generalplayfully ludic. In this vein, it appears that the trial-and-error exploration of the film-interface provided somepleasure, notably concerning the parallax effect of thebuildings and the random graphic animation of the i-docdécor.

Evenifthetrial-and-errormodeisoftenencouragedininteractivemedia, it does not appear to trigger off lastinguser involvement (i.e. being ‘engrossed, caught up,

Page 11: Evaluating Users' Experiences (UX): a case study approach

10

enthralled’,cf.Goffman1986[1974]:345)withi-docs.ThiswasillustratedintheanalysisofUser11A’sviewingpaths.ThisUserdeclaredhavingadopteda ludicattitude levelof78percent,yethadnotviewedasinglevideoinitsentirety.Given the limitedtimeconstraintsof theviewingsituation,the plethora of viewing choices, and the absence ofcomment from the viewers about wanting to go back andrelook at videosduring thepost-viewing interviews, it canbeinferredthatthedeterminingfactorappearsnottobethe‘ludic attitude’, but the cognitive effort needed tounderstandtheplayabilityrulesofani-doc.

TheeffortrequiredtograspthethreeviewingmodesofB4appearedtobehighforusers(seeUser9B’scomments,below). Itwas only once the informational architecture ofB4hadbeenassimilatedthataludicattitudebecameuseful.

‘IfindtheconceptofablockofflatsinterestingbutIdidnotanswer ‘completelyludic’becauseittookmesometimetounderstandhowthingswork.Ididnot always understand the different categories.WhenIamontheInternet,Idon’twanttosearchhow to do things. I prefer to have all that infoquickly.Iwanttolearnstuff,andsearchinghowtounderstand how the site is built is too tedious’(interviewextractofUser9B).Table2, below, summarises thedegree towhich

usersperceived the lucidity level of the film-interface.Theoverall resultshowsthattheperceptionofplayfulludicitywasseenasrelativelyhigh(73.7%),withonlyathreepointdifferencebetweenGroupA(75.5%)andB(72%).

Page 12: Evaluating Users' Experiences (UX): a case study approach

11

Table2.PerceptionsoftheludicitylevelofB4

3.2 Usability(Hypothesis2)In evaluating B4, the analysis of on-screen actions

indicated how users perceived, understood and used the‘affordances’and‘go-between’signs(Table3,seebelow)ofthefilm-interfaceandattendantproblemsofusability.

HighlightedaffordancesDonald Norman (2013 [1988]: 10) defines an

affordances as ‘the relationship between the properties ofan object and the agent’s capacity to determine how theobject could possibly be used’. The effectiveness of anaffordance rests on informing userswhat possible actionscanbetakenwithoutneedingfurtherguidance.

InthecaseofB4,afterusersclickinsideawindow,thehorizontal and vertical lines light-up as a momentary,yellow intersecting cross (see Figure 1, above) as anintended affordance. The data show that 92 per cent ofusersdidnot see that thehighlighted crossasa visual aidindicatingtheorganisationofvideos(seeTable3,below).Inaddition,77percentofusersdidnotgraspthattheflashing

Users %

Group

A(w

ithout

paratext)

U2 70U5 73U6 89U8 79U10 64U11 78average 75,5

Group

B (withparatext) U1 80

U3 80U4 92U7 56U9 60U12 64average 72

overallaverage 73,7

Page 13: Evaluating Users' Experiences (UX): a case study approach

12

cross indicated an entrance hall of the building that wasclickableonwhattowatch,ornot.‘I saw the lighted sign,but it didnot strikeme that it couldhavebeenuseful.Ididnotseeitassomethingsignificant.Forme,itwasdecorative’(interviewextractofUser6A).

Atleasttworeasonscanexplainwhymostusersdidnotclickontheintersectingcross.First,whenauserclicksonavideo,avideoplayerspringsupinthemiddleofthescreen,effectively blocking out the flashing light of the cross.Second,thelackofalabeltoexplainthecross,differsfromthe logic of other parts of the film-interface that links thenameofeachcharactertoavideo(seeUser5A’scomments,below).‘Ididnotseethelightinthehallwaybecausethevideoplayerwas in front of it […]. I didn’t click on it probably becausetherewasn’t a text to it. In the otherwindows, I know theywere links because they had titles.When the lines flashed, Ithought it was an animation – like permanent hazardwarning lights at (French) railway crossings– thatwasnotclickable.WhenIseeatextthatchanges,Isaytomyselfthatsomethingisclickable,buthereitjustflashedwhenmymousewas not over it, so I thought it was decorative’ (interviewextractofUser5A).

Go-betweensignsInthetheoryof‘screenwritings’(écritsd’écran)ofYves

Jeanneret and Emmanuël Souchier (2005) ‘go-betweensigns’ (signes passeurs) are described as sign-tools (icons,words,buttons,etc.) indicatingwhattheviewercandecidetouse.Thego-betweensignsthusaddaperfomativeaspectto an i-doc. The design of such signs creates a dialecticaltension between the graphic image of the sign and thenorms of on-screen usage. In this way, clicking on an on-screen signdependsonauser’s capacity tomakesenseofthe document asmuch, if not more so, as the ergonomicsaspectsofthetext.

B4 users understood the go-between signs indicatingthe on/off sound track and the sharing with Facebook(Figure1middle right of the screen, see above). However,92 per cent of users did not grasp theRestarting viewing

Page 14: Evaluating Users' Experiences (UX): a case study approach

13

path sign. In effect, users did not consider the option ofchanging viewing paths because they did not see how thefilm-interfacewas‘playable’indifferentways.

Table3.Understandingaffordancesandgo-betweensigns

Thehyperlinks sendingusersoutsideof the i-doc (topof theB4 screen, Figure 1, see above)were considered asconfusinglygraphicallysimilartootherhyperlinksofferingparatextualinformationinsidethei-doc(Figure1,bottomofthe screen, see above). This seemed to have disorientedsomeusers.‘I thought thatthe thumbnailson topof thescreenwerethemenus of Pluzz (catch-up TV). I waswondering why it hadbeenplaced there. I didnot dare click on itbecause I didn’twant to leaveB4. Iwasn’t certainwhether themenuat thebottomofthescreenmadeyoustayinB4,ornot’(interviewextractofUser10A).

Page 15: Evaluating Users' Experiences (UX): a case study approach

14

3.3 Informationalarchitecture(Hypothesis3)For Group A, who had viewedB4 without a decision-

aiding paratext, 66 per cent used the horizontal viewingmode,comparedto100percentofGroupB(withparatext,seeTable4,below).

ForGroupAtheverticalviewingmodewas100percentinvisible,whereasforGroupB,66percentofusersuseditin their alternative viewingpaths. It appears thatGroupBfirstneededtocomparedifferent theviewingpathsbeforeoptingforaverticalviewingmode.

Thetransversalviewingmodeactedasareferencepointforusers, inwhat canbe calleda reassuring ‘conventionalengagement’ (Bouchardon 2009). A problem with thetransversalmode is that it repeatedvideos found inothermodes(seeUser8A’scomments,below).‘Iwouldhave liked toavoid therepetitionofvideosbetweenthosepresentedinthewindowsoftheblockofflatsandthoseinthemenuofthemesbecauseitwasarightmesswhenyouviewed the same video twice (…). I wanted to stop viewingbecauseIhadtheimpressionofhavingseenallthevideos,orthatitwasnotworththeeffortoflookingforvideosnotseen(interviewextractofUser8A).

Table4.Viewingmodesofgroups10AandB

Page 16: Evaluating Users' Experiences (UX): a case study approach

15

4.Sense-makingprocesses(Hypothesis4)

Giventhattherearenumerousviewingpaths,B4usershad to take a series of problem-solving decisions inchoosingagivenviewingpath.Theproblem-solvingprocessrests on a series of coherence-seeking connections ofperceivedmeaningfulelements(Figure2,seeabove)guidedbyusers’preoccupations.

In the case ofB4, users’ central preoccupationwas toexplore inner-city ‘togetherness’ (vivre-ensemble). It ledusers to create information-seeking quests expressed inidentifiable ‘search modes’. These search modes allowedusers to connect, what they considered as meaningfulelements into ‘sense-makingconstructs’(Labour2011:96-99).

Table5.Sense-makingsearchmodes

There are times, however, when the top-down andbottom-up dynamics of the sense-making process do notmutuallyreinforceeachother(seeFigure2).Usersthenfindthemselves in a ‘sense-making break’, or ‘senselessness’situation.

Sense-makingbreakA break in the sense-making process (Labour 2011:

101-102) occurs when users can identify apparentlymeaningfulelementsbutcannotconnectthemupcoherently

User Examplesofusersense-makingsearchmodes

U8A Getting an overall view of the neighbourhoodbeforezoomingontotheblockofflats.

U5A Focusingonindividualportraits.U9B Itemsofinteresttoyoungadultsandchildren.U12B Discoveringtheintimacyofeverydaylife.

U6A Items that may interest a given (‘female’) publicabouttheNorthAfricanurbanspaces.

U11A

Curiousabouttechnicaldetails.

Page 17: Evaluating Users' Experiences (UX): a case study approach

16

in terms of the perceived situation-problem (cf. Goffman1986[1974]:345-377).Thiscaninciteuserstolookfornewepistemological connections in order to resolve theproblem-situation. The sense-making break leads users tore-frame what they perceive. This can accord with anauthor’s wish to challenge how users see their world.However, attempts to question users’ value system arechallenging because users can disengage at any time,namelyiftheyfeeltheyarewastingtheirtime,orifthetaskisseenastooarduous.

InthiscasetheB4author’sintentionwastochangehowpeopleperceiveinner-cities,asUser9Bexplainsbelow.‘There is an even bigger gap than I thought between thehumanwarmthof inhabitantsandtheexternalenvironmentof the flats. I’ve never lived in subsidised housing; I havealwayslivedinahouseinthecountryside(…).Irealisedthatbeyondthecliché,peopleininnercitieshadastorytotell.B4opened my eyes about something I had never thought ofbefore. I said to myself that people are there because theyescapedawarintheircountry,ortheywantabetterqualityof lifeoverhere[…].Therewasanexample,ofapersonwhoworkedforaMemberofParliament.Thissurprisedme,butitmakessensethatnoteveryoneiswhiteinParliament.Isaidtomyself that when one takes the time to get involved in theinhabitants’ lives, theyareverydifferent towhatyouseeonTV(interviewextractofUser9B).

SenselessnessSenselessness happens when users cannot identify

meaningfulelements(words,sounds,andpictures,etc.)inaproblem-situation(Labour2011:99-101).Thiscanleadtoadiscouraging feeling of being in a dead-end, anddisengagementcanfollow.

Inthecontextofani-doc,senselessnesscanbeinducedbydisorienting(usability)problemsthatproducecognitivenoise. Our data show that Group A (without paratext)declared a higher feeling of senselessness than Group B(withadecision-aidingparatext).

Page 18: Evaluating Users' Experiences (UX): a case study approach

17

5.EmpiricUserandModelUsergap

The findings of the study, focussed on four workinghypotheses about a data-based constructed Empiric User,showthat:

- Users did indeed use a combination of ludic anddocumentarisation viewing modes when using the film-interface(Hypothesis1confirmed).

- Unclear or patently absent go-between signs andaffordanceseffectivelycausedusabilityproblemsformanyusers(Hypothesis2confirmed).

- Users’ ability to make sense of their viewing pathsincreased with the help of explicit decision-aidingguidance(Hypothesis3confirmed).

- Sense-making constructs were effectively personalisedthrough a series of information-seeking quests linked todifferent viewing modes. The data showed that users’modified their initial viewpoints about inner-cities afterhavinginteractedwithB4(Hypothesis4confirmed).The comparison between the Model User and the

Empiric User leads us to advance three key points aboutlessonslearntfori-docdesign.

First, if B4 users deploy simultaneously adocumentarisation and ludic viewing mode, it did notencourage ‘playability’ as would have wished the author-director’sModelUser.

Second, the archetypal Model User was aimed atencouraging an intuitive viewing of B4. This was notconfirmedinthestudy.Itturnedoutthatmostusersneededto be provided with clear ‘conventional engagements’(Bouchardon 2009). For example, in the film of the trial-and-error Model User, the vertical viewing mode waspromoted.Yet,inourstudy,nousertooktheverticalmodeunlesstheyhadbeenexplicitlyinformedofitsexistence.Ina nutshell, the design of i-docs could be significantlyimproved if users were given explicit guidelines about itsinformationalstructure.

Finally,ourstudyrevealedthatsenselessnessincreasedwhen the viewing was conducted without an explicitdecision-aidingparatext.Thelackofsuchguidelinescreates

Page 19: Evaluating Users' Experiences (UX): a case study approach

18

cognitive noise that hampers users’ information-seekingquests.

Conclusion

A linear television (pre-production, production, post-production)modelwas used in the design of theB4 i-docexamined in this study. This resulted in the i-doc beingdesignedaroundaModelUserbasedontheperceptionsofthedifferentauthor/designer/TVcommissionerinvolvedinthe i-doc design. However, the qualitative UX researchapproach of our study shows the gap between what theprojected Model User should have experienced, and whatthe observed Empirical User experienced actuallyexperienced. Given this, we advance an iterative user-centered design approach to ensure that empiric usersgrasps (sense-making) of the intentionality of authors isboth fully experienced (usability) and taken seriously(viewingframe).Suchaniterativedesignapproachinvolvescreating auser-validatedprototypephase about the initialdesign (paper prototyping) before moving on to theimplementationphases(dynamicprototyping).

One limit of the studywas that itwas conducted in anear-laboratoryconditionandnotinamorenaturalviewingcontext.Futureresearchcouldalsocompareverydifferenti-docs as a way to improve i-doc design from a userperspective. More broadly speaking, the challenge ofemerging i-docs formats lies in evaluating iterative user-centreddesign.Anexample,of thisapproachcanbe foundin theManifesto forAgileSoftwareDevelopment (DeuffandCosquer 2013), with its capacity to integrate authoringissues through self-organization, team work and flexibleresponsestochangewhenimprovingi-docdesign.Bibliography

Akrich, Madeleine (2006) ‘Les objets techniques et leursutilisateurs. De la conception à l’action’. In MadeleineAkrich,MichelCallon,etBrunoLatour(Eds.)Sociologiedelatraduction:Textesfondateurs.Paris:PressesdesMines.79-199.

Page 20: Evaluating Users' Experiences (UX): a case study approach

19

Barboza, Pierre & Weissberg, Jean-louis (2006) L'imageactée. Scénarisations numériques, parcours du séminaire‘L'actionsurl'image’.Paris:L'Harmattan.Bateson, Gregory (2000 [1954]) Steps to an Ecology ofMind.Chicago:UniversityofChicago.Bouchardon, Serge (2009) Littérature numérique: le récitinteractif.Paris:Hermès/Lavoisier.Dervin,BrendaandDewdney,Patricia (1986) ‘Informationseeking neutral questioning’, Research Quarterly, 25, 506-513.Deuff,DominiqueetCosquer,Mathide(2013)MéthodeAgilecentréeutilisateurs.Paris:Hermès.Di Crosta, Marida (2009) Entre cinéma et jeux vidéo:l’interface-film, Métanarration et interactivité. Bruxelles:DeBoeck/INA.Drouillat, Benoît (2013) ‘Expérience utilisateur’. In LeDesign des interfaces numériques en 170 mots-clés: desinteractions homme-machine au design interactif. Paris:Dunod.57-59.Eco, Umberto (2010 [1985]) Lector in fabula, Le rôle dulecteur.Paris:Grasset.Gantier, Samuel (2016) ‘Scénariser le rôle et le pouvoird’agirdel’utilisateur:versunetypologieinteractionnelleduweb-documentaire’,Entrelacs,13.Gantier, Samuel and Labour, Michel (2015) ‘UserEmpowermentandthei-docModelUser’.InDavid,Bihanic(Ed.). 2015. User Empowerment: Interdisciplinary studiesand combined approaches for technological products andservices.London:SpringerVerlag.231-254.Gaudenzi, Sandra (2013) The Living Documentary: fromrepresenting reality to co-creating reality in digitalinteractive documentary. Doctoral thesis, Goldsmiths,UniversityofLondon.Genvo, Sébastien (2013) Penser la formation et lesévolutions du jeu sur support numérique. Post-doctoralthesis ‘Habilitation àdiriger des recherches’. University ofLorraine,France.Goffman,Erving(1986[1974])Frameanalysis.AnEssayofthe Organization of Experience. Boston: NorthwesternUniversityPress.

Page 21: Evaluating Users' Experiences (UX): a case study approach

20

Hartje,Hans,Magné,BernardetNeefsJacques(Ed.)(1993)CahierdeschargesdeLaViemoded’emploi.Paris:CNRS.Huart,Julien,Kolski,ChristopheetBastien,Christian(2008)‘Évaluation de documents multimédias. État de l’art’. InObjectiver l’humain. Sylvie Leleu-Merviel (Ed.). Paris:Hermès/Lavoisier.211-240.Jeanneret,Yves(2007)Ya-t-il (vraiment)des technologiesde l’information? Villeneuve d’Ascq: Presses duSeptentrion.Jeanneret, Yves et Souchier, Emmanuël (2005)‘L’énonciation éditoriale dans les écrits d’écran’,Communicationetlangages,145,1,3-15.Labour,Michel (2011)Média-Repères : uneméthode pourl'explicitation des construits de sens au visionnage. Post-doctoral thesis ‘Habilitation à diriger des recherches’.UniversityofValenciennes,France.Norman, Donald (2013[1988]) The Design of EverydayThings.NewYork:BasicBooks.Odin, Roger (2011) Les Espaces de communication:introduction à la sémio-pragmatique. Grenoble: PressesUniversitairesdeGrenoble.Perec, Georges (2010 [1978]) La Vie mode d’emploi:romans.Paris:Hachette.Pignier, Nicole et Drouillat, Benoît (2005) Penser lewebdesign: modèles sémiotiques pour les projetsmultimédias.Paris:L’Harmattan.Popper, Karl (2007 [1934]) La Logique de la découvertescientifique.Paris:Payot.Soulez, Guillaume (2004) ‘ "Qu’y croire?" Le crédit del’auteurdanslafiction, lereportageetledocumentaire’. InTélévision:notiond’œuvre,notiond’auteur.RenéGardiesetMarie-ClaudeTaranger(Eds.).Paris:L’Harmattan.119-151.

1 There appears to be little reliable studies on this subject. Theobservationisbasedonprofessionalgrassrootspractice.2TheI-Doc,whichwillhenceforthbereferredtoasB4,wasdirectedbyJean-Christophe Ribot and co-produced byMosaïque Films and France

Page 22: Evaluating Users' Experiences (UX): a case study approach

21

Télévisions (French state television). Available at:http://www.francetv.fr/nouvelles-ecritures/banlieue-b4/ (accessed 30October2015).3The ideaofModelUser is inspiredfromUmbertoEco’s(2010[1985])conceptofModelReaderinhistheoryoftextualcooperation.4MaridaDiCrosta (2009)definesa film-interfaceas a seriesofhybridobjects found in the intersection between cinematographic fiction andvideogames.5i.e.96x95x94x93x92x91=667,474,778,880.6SeetheinterviewsconductedbySandraGaudenzi.availableat:http://i-docs.org/?s=UX+series&x=0&y=0(accessed30October2015).7Asystemicapproachinvolvescircularcausalrelationships–viaaseriesof feedback loops operating in different temporal frames – linking updifferentdimensionsintoafunctionalwhole.8 For Erving Goffman (1986 [1974]:345)a ‘frame’ organises ‘meaning’and ‘involvement’, linked to normative expectations, such that‘participantswillnotonlyobtainasenseofwhatisgoingonbutwillalso(in some degree) become spontaneously engrossed, caught up,enthralled’.9AlloftheinterviewextractsaretranslatedfromFrench.10Timegiveninminutesonanindicativebasis.