1

WallTalk: Connecting Elderly with Their(Grand)Children in a Way That Is Suitable for All

Generations InvolvedKayleigh Beard (2521477) and Nathalie Post (2526555)

Faculty of SciencesVU University Amsterdam

k.l.beard@student.vu.nl, n3.post@student.vu.nlJuly 1, 2015

F

Abstract—A high quality relationship between elderly and their(grand)children can be beneficial for all generations involved. However, acommunication barrier can exist between the younger and older genera-tions, because generally each generation prefers different communicationmethods. This can negatively influence the grandparent-(grand)childrelationship. To address this problem, WallTalk is developed: a systemthat aims to connect elderly with their (grand)children, in a way that issuitable for all generations involved. WallTalk projects (grand)children’sFacebook content on the wall of elderly. The Facebook content of each(grand)child is projected around a physical photo frame containing apicture of that (grand)child. Elderly can ‘like’ and ‘comment’ on thisFacebook content in an intuitive manner: by smiling and speaking. Itwas evaluated whether WallTalk is a suitable system for both grand-children and grandparents. A questionnaire was conducted among 254grandchildren, and a field experiment with semi-structured interviewswas carried out among 5 grandparents. The sampling methods usedwere non probabilistic (convenience sampling), therefore the results ofthis study are not generalizable to a larger population. The findingsof this research indicate that the developed prototype of WallTalk isnot completely suitable for grandchildren and grandparents. For bothgroups, suggestions are made to enhance the suitability of WallTalk infuture research.

1 IntroductionModern communication technologies could enhance the con-nectedness between elderly and their (grand)children [1]. Yet,communication preferences of younger generations mostlydiffer from the preferences of elderly people [2]. According toLindley et al. (2012) [1], younger generations regularly usemodern communication methods, such as text-messaging andsocial media platforms. However, the design of these moderntechnologies is mostly focused on younger generations, andoften involves asynchronous (‘lightweight’) communication,such as posting Facebook messages. In general, elderly havemore affinity towards traditional synchronous (‘heavyweight’)communication, for instance calling and face-to-face contact.Older generations often experience difficulties when adaptingto new technologies, or are simply not interested in usingthem. This can create a communication barrier betweengrandparents and their children and grandchildren. Therefore,

the research goal of this paper is to develop and evaluate a sys-tem that connects grandparents with their (grand)children, ina way that is suitable for all generations involved.

Previous research [3][4] has emphasized the importanceof the grandparent-(grand)child relationship. High qualityrelationships between these generations can be beneficialfor the health and well-being of both parties. Several stud-ies [5][6][7] have focused on developing communication sys-tems to enhance the relationship between grandparents andtheir (grand)children. However, these systems establish onecommunication method for all users, and are therefore eithersuitable for elderly or for (grand)children, but rarely for allgenerations involved. Text-messaging, posting a status updateon Facebook, and tweeting on Twitter, are all communi-cation methods that are suitable for the busy lifestyles ofyounger generations. On the other hand, elderly often haveless busy lifestyles; therefore, communication methods suchas calling and face-to-face contact are more suitable for theirgeneration. This is the main challenge of developing a sys-tem for grandparent-grandchild communication: addressingthe different needs of all generations involved. Therefore,‘WallTalk’ is proposed to improve upon current grandparent-(grand)child communication technologies, and meet the needsof both elderly and their (grand)children.

WallTalk is a system that projects (grand)children’s Face-book content on the wall of elderly (Figure 1). The Facebookcontent of each (grand)child is projected around a physicalphoto frame that contains a picture of that (grand)child;adding value to the familiarity of physical photo frames.Elderly can ‘like’ and ‘comment’ on this Facebook content inan intuitive manner: by smiling and speaking. Choices for thedevelopment of WallTalk have been derived from literatureconsidering similar areas of study. In this paper, WallTalk isdescribed, and evaluation studies are conducted to determinewhether WallTalk is a suitable communication system forboth elderly and their grandchildren.

Figure 1. An example of the WallTalk projection.

2 Related workIn the field of Human-Computer Interaction, various systemshave been developed that aim at establishing a connectionbetween elderly and their children and grandchildren. Whenpeople are separated by distance, communication often takesplace using traditional communication methods, such as call-ing. These communication methods are heavyweight, mean-ing that communication is likely to occur less frequent, butwhen it does occur, it is focused and prolonged. In con-trast, lightweight communication is dispersed over time andsometimes asynchronous, such as text-messaging, updating astatus on Facebook, or tweeting on Twitter [1].

A number of communication systems have been developedthat focus on establishing lightweight communication. Theselightweight communication systems can serve as an additionto traditional heavyweight communication; contributing tothe connectedness between people who are separated by dis-tance. This is the type of communication WallTalk aims toestablish as well, by including elderly in the social mediaactivities of their offspring. A selection of these lightweightsystems, focused on connecting elderly with their offspring,will be discussed and evaluated.

2.1 The FamilyPlanterMost people feel a strong need and desire to stay connectedwith each other when they become separated by distance [8].When people are separated by distance, situational cuesare one of the aspects of communication that are naturallylost. These are cues such as the presence of others, hearingsomeone’s footsteps, and observing the brightness of theroom. Situational cues provide information that serves asa background-context in face-to-face communication. The‘FamilyPlanter’ was developed to offer comfort and a feeling oftogetherness to people living apart from each other, helpingthem to maintain their relationships [5]. The FamilyPlanteris a terminal in the form of a plant, which facilitates theexchange of data about people’s unconscious presence andmotion cues. The FamilyPlanter was not developed specifi-cally to connect elderly with their distanced offspring, such asWallTalk, but for all kinds of family relationships.

The information that is shared through the FamilyPlanterdiffers from WallTalk for multiple reasons. First of all, theinformation that is shared through the FamilyPlanter is am-

biguous and left up to the interpretation of the users them-selves. The FamilyPlanter only provides sensor informationabout the presence of people in their homes. This limits thescope of the shared information, because the FamilyPlanteronly shares ‘presence’ of people, but not specifically whichpeople are present. Information that is shared using WallTalkis less ambiguous, because the information is expressed in theform of messages and photos.

Elderly users of the FamilyPlanter had more positive atti-tudes towards the system than their children and grandchil-dren. The elderly users stated in personal interviews that theytook comfort out of the gleaming of the FamilyPlanter, andthat it evoked thoughts about their families. These findingssuggest that elderly would potentially benefit more from thefeeling of closeness to their families elicited by the Family-Planter, than their children and grandchildren. It is difficultto generalize the findings from this research to a broaderpopulation, since the FamilyPlanter was only installed in asmall number of households. However, the findings from thisresearch are in agreement with research pertaining to theexisting asymmetry in the relationship between grandparentsand their offspring. The relationship between grandparentsand their (grand)children seems more focused on the activ-ities and accomplishments of the (grand)children, than viceversa [1]. Additionally, according to Mynatt et al. (2001) [9],elderly are often interested in what their offspring are doingand want to ‘monitor’ this. In general, the lifestyle that comeswith seniority allows elderly more time for these monitoringbehaviors. Therefore, it is not surprising that the elderly inthis study responded more positively, and expressed moreinterest in the FamilyPlanter, than their offspring.

2.2 The Digital Family PortraitIn the ‘Digital Family Portrait’, a digital photo frame wasused as a communication system to connect elderly with their(grand)children [9]. The photo frame was chosen as medium,because of elderly’s familiarity with it as a household object.This idea has served as inspiration for WallTalk. Even thoughmany elderly have a positive attitude towards technology, theyoften experience difficulties adapting to new technologies [10].Therefore, to avoid this problem, WallTalk only adds visibletechnology to existing household objects (photos in theirframes). While the Digital Family Portrait requires replace-ment of existing photo frames with digital ones, WallTalkpreserves current photo frames, and even the placement ofthe photo frames can be maintained.

Even though the Digital Family Portrait was positivelywelcomed by the elderly and their offspring participating inthe study, there were some problems with the design of theDigital Family Portrait that should be improved upon. Thephoto frame displayed a history of a person’s activities, but itturned out to be difficult to read off the photo frame whenactivities had taken place. This caused erroneous interpre-tations among the users. To avoid erroneous interpretations,it is important that the communicated information is unam-biguous. In contrast to the Digital Family Portrait, WallTalkonly displays information from the (grand)children to thegrandparents, and not vice versa. This is because WallTalkhas a different goal than the Digital Family Portrait. TheDigital Family Portrait focuses on providing (grand)children

2

with the ability to monitor the elderly, while WallTalk aimsat providing elderly inclusion in their offspring’s online lives.When displaying information to elderly, it is even more impor-tant that the information is not confusing or ambiguous. Theresults from this research emphasize the importance of usingsimple, unambiguous information, and limiting the amount ofinformation that is displayed. Therefore, WallTalk does notprovide elderly with a ‘history of information’, but instead,will only display the most recent information.

2.3 The Family Window: a domestic media spaceThe previously discussed related works focused on creatinga feeling of connectedness by using abstracted information.The ‘Family Window’ takes a completely opposite approach.Judge et al. (2010) [11] argue that abstracted information doesnot provide the same feeling of connectedness that is elicitedwhen seeing each other. Therefore, a video media space, theFamily Window, was developed to establish connectednessbetween family members living apart. In the Family Window,a continuous video connection was established between twohomes, allowing family members to monitor their distancedfamily members continuously. Even though family membershad the option to turn the Family Window off, the participat-ing families in a field experiment had many concerns abouttheir privacy. Sometimes family members felt uncomfortablewhen watching the Family Window, while other times theyfelt uncomfortable being watched.

An important lesson can be learned from the FamilyWindow: media spaces are not the type of systems that mostpeople feel comfortable with in their homes [12]. People arelikely to feel more comfortable sharing abstracted informationthan sharing a continuous video connection. Even thoughthe researchers argue that abstracted information does notprovide a great feeling of connectedness, systems such as theFamily Window can lead to other dissatisfactions. WallTalkaims at finding the sweet spot for the type of informationthat is shared. The information should be less abstract thansystems such as the FamilyPlanter, but more abstract thanthe Family Window. Therefore, extracting information fromFacebook seems a suitable way to acquire information withthe right amount of abstraction. Messages and photo’s onFacebook are explicitly posted; the users know exactly whichinformation they are sharing. This prevents the type of pri-vacy infringement that is more likely to occur with systemssuch as the Family Window.

2.4 ePortraitThe ‘ePortrait’ is a similar system to WallTalk, because theePortrait also includes elderly in the online social networkactivities of their (grand)children [13]. Social networks canenhance the quality of life for elderly [14]. However, elderlyare a less represented group on online social networks thanyounger generations [15]. An important barrier for elderly inmaintaining strong emotional ties with their distanced family,is that elderly are often disassociated with events relatedto their distanced family’s everyday lives [13]. Part of thefamily’s everyday lives is shared on online social networks,and not all elderly feel comfortable enough with technologyin order to engage in those [16]. Therefore, the ePortrait wasdeveloped to include elderly in the social network activities

of their family, without having to face too many technologicalbarriers. The ePortrait is a digital photo frame, which displaysphotographs from elderly’s (grand)children that are publishedon Facebook.

A field experiment, in which the ePortrait was installed inthe homes of elderly, highlighted the importance of a feedbackpossibility for the elderly. The elderly using the ePortraitenjoyed being included in the social media activity of theiroffspring, but there was no possibility for them to respondto the pictures they were able to see on the ePortrait. Thishighlights the importance for WallTalk to establish bidirec-tional communication between the elderly and their offspring.Even though the ePortrait includes elderly in the online socialmedia lives of their offspring, the ePortrait also excludeselderly, because they do not have the opportunity to respondto the photos. In order to foster bidirectional communicationand connectedness between elderly and their offspring, elderlyshould have the possibility to provide feedback in a way thatis suitable for their generation. Therefore, WallTalk aims tofacilitate communication that is suitable for elderly: withoutcomplicated user interfaces.

2.5 Building BridgesPhysical limitations are one reason why elderly stay at homeand avoid social interaction [17]. Moreover, older adults areoften not experienced with new technologies, such as smartphones, to use as mediums for social interaction. ‘BuildingBridges’ is a touch screen device with communication softwareespecially developed for this target group to support socialconnectedness [6]. The developers of Building Bridges believethat a touch screen interface is more intuitive than a mouseor keyboard, and thus easier for elderly to use. In contrast tothe previous discussed examples, this type of communicationestablishes heavyweight communication, while WallTalk aimsto establish lightweight communication. Moreover, Jin et al.(2007) [18] state that current research is mostly focused ondesigning touch screen interfaces for younger generations, andthose interfaces are not optimal for elderly to use. Thus,when focusing on developing innovative technologies for olderadults, we believe that these technologies can be more effort-less than the touch screen device of Building Bridges. There-fore, a touch screen interface is not integrated in WallTalk andother technologies are explored to ensure effortless usability.

Furthermore, the developers of the Building Bridges touchscreen device believe that such a device hides the complextechnology behind the system. This vision overlaps with whatwe aim to achieve with WallTalk, since older adults oftentry to avoid new technologies due to computer anxiety [19].Nonetheless, the presence of a digital screen is still an im-plication of technology, while older adults are more attachedto physical non-technological objects [20]. Therefore, whenvisualizing social media of the children and grandchildren ofelderly people, WallTalk takes one step further to hide theintegrated technology and to make use of familiar physicalnon-technological objects.

2.6 Peek-A-Drawer: communication by furnitureThe touch screen device from the Building Bridges projectclearly plays a technological role in the house of elderly. In con-trast, ‘Peek-A-Drawer’ is a lightweight communication system

3

that makes use of a familiar household object [7]. Cameras areplaced in the drawers of the grandchildren’s cabinet, while thebottom surfaces of the grandparents’ drawers contain digitalscreens. When a child puts his or her items in the drawer,a picture is taken which is displayed on the digital screenin the grandparents drawer. Similar to WallTalk, this is asimple communication method that enhances the feeling ofconnectedness between grandparents and their grandchildren.

Forgani et al. (2014) [21] states that “grandparents liketo know nearly everything about their grandchildren”. Theyfound that grandparents sometimes feel self-conscious andafraid to annoy their grandchildren by interfering too muchin their daily lives. Thus, Peek-A-Drawer can be a satisfyingsolution for grandparents, since they do not have to disturbthe grandchildren, yet are still in contact with them. This isan inspirational aspect for the development of WallTalk. Thepictures that are taken in the Peek-A-Drawer only providea small amount of information about the grandchildren’s life,and no details such as school grades or outdoor activities. Thisis what WallTalk aims to improve upon: establishing effortlessconnectedness while providing more detailed communication.

Miyajima et al. (2005) [5] state that older adults haveenough time and interest to be actively involved in the com-munication with their family members. Thus, when elderlyuse Peek-A-Drawer, they might be frustrated, because theycan only receive information, and not interact or react on it.Therefore, WallTalk aims for a bidirectional communicationflow, which is still effortless, to meet the needs of the olderadults.

2.7 Summary and ImplicationsThe systems discussed in this section all have aspects thatcan be learned from and improved upon. When develop-ing communication technologies that connect elderly withtheir (grand)children, such as WallTalk, the following aspectsshould be taken into consideration.

First, the design of communication technologies shouldbe unambiguous and simple, and it is important to findthe right amount of information abstraction. Second, tech-nology should be ‘hidden’ in the lives of elderly, preferablyby blending in with familiar physical objects, such as photoframes. Third, elderly prefer communication technologies thatallow for bidirectional communication. Even though elderlylike to ‘monitor’ their offspring, they do not feel comfortableusing systems that enhance the feeling of disturbing theirchildren and grandchildren. At last, the privacy of both elderlyand their (grand)children should be assured. Facebook is asuitable medium to extract privacy assured information fromchildren and grandchildren.

3 ContributionThere is currently no system that connects generations in alightweight manner while suitable for all its users, as con-cluded from the studied related works. WallTalk aims toestablish this connection in order to enhance the health andwell being of both elderly and their (grand)children. This canbe referred to as the social relevance of WallTalk. Besides,according to the discussed related work, no system exists thatcombines the techniques of smile recognition, speech recogni-tion and visualization of social media content into one system

that aims to achieve this goal. This combination of differentArtificial Intelligence techniques is our practical contributionto the field of developing communication technologies. Mostimportantly, by evaluating the usability and suitability ofWallTalk, we contribute to the academic field of Human-Computer Interaction studies focused on developing commu-nication technologies for elderly and their (grand)children.

4 System DescriptionA schematic synopsis of the usage of WallTalk is displayed inFigure 2. The goal of WallTalk is to establish a connectionbetween elderly and their (grand)children, in a way that issuitable for all generations involved. We aim to achieve thisconnection by providing elderly with an easy and intuitivecommunication method, while making use of familiar (physi-cal) objects.

WallTalk is a system that projects (grand)children’s Face-book content on the wall of elderly. The Facebook contentof each (grand)child is projected around a physical photoframe that contains a picture of that (grand)child. Elderlycan ‘like’ and ‘comment’ on the latest Facebook status updatein an intuitive manner: by smiling and speaking respectively.It should be noted, that grandparents can only commenton the most recent posted status update of one of their(grand)children. Even though this limits the communicationpossibilities, it enhances the ease of use for elderly. This isdone purposely, to ensure that elderly do not have to makeuse of complex user interfaces.

A technical system description of WallTalk can be foundin Appendix A, and the source code is available on GitHub 1.The flow of the system can be summarized as follows.

Figure 2. Schematic overview of the usage of WallTalk.

1) WallTalk extracts information from (grand)children viaa medium that is currently widely used among the generationsof the children and grandchildren of elderly: Facebook. Thisinformation can appear in the form of written messages andpictures. Section 4.2 goes further into detail about the rea-soning behind the choice for this particular data source. Thecommunication from (grand)children to their grandparentswill be explained in Section 4.1.1.

1. Source code: https://github.com/postnathalie/WallTalk

4

2) Parsed by the system, the messages and/or picturesare projected on the wall of the elderly. The projections ofthe Facebook content are displayed around physical photoframes of (grand)children on the wall. The visualization ofthis content, together with WallTalk’s physical appearance inthe homes of elderly, will be discussed in Section 4.3.

3) Once the Facebook content is displayed on the grand-parents’ wall, grandparents can communicate back to their(grand)children. They can do this in two ways, namely by‘liking’ and ‘commenting’ (in Facebook terminology). To ‘like’certain content, grandparents simply have to smile in front ofa camera. To ‘comment’, grandparents need to hold an objectof choice and speak. In the prototype of WallTalk, a statuette(that the grandparents already have in their homes) is con-nected to WallTalk and used to initiate and stop recordingthe grandparents’ comments (explained in detail in A.6). Thegrandparent-(grand)child communication flow will be furtherdiscussed in Section 4.1.2.

4.1 Communication flow and interactionWallTalk is a bidirectional communication system, whichmeans that there are two communication flows: from(grand)child to grandparent, and from grandparent to(grand)child. Several studies [13][5] have highlighted the im-portance of bidirectional communication for grandparent-(grand)child communication technologies. However, not onlyis it important that communication technologies facilitatebidirectional communication, it is even more importantthat both parties feel comfortable using the communica-tion technology. In general, (grand)children and grandparentshave different preferences for communication technologies.(Grand)children often use lightweight communication tech-nologies, such as online social networks and text-messaging.Grandparents are not always familiar or comfortable withthese communication technologies. They often prefer using so-called heavyweight communication methods, such as callingor letter writing, which are more focused and prolonged [1].WallTalk addresses these different preferences of communica-tion methods, by aiming to facilitate communication that istailored to the preferences of all generations involved.

4.1.1 (Grand)child to grandparent communication flowThe communication-flow from (grand)child to grandparentcomprises of messages and photos that are posted by the(grand)children on Facebook. WallTalk facilitates sharingthese messages with the grandparents. The choice for thiscommunication-flow is based on two important character-istics pertaining to the (grand)child in the (grand)child-grandparent relationship, namely (1) the asymmetry in thegrandparent-grandchild relationship, and (2) the amount offree time and effort.

1) The asymmetry of the grandparent-(grand)child rela-tionship has been emphasized in several studies [22][1]. Often,(grand)children are the focus of their grandparents’ affection.This is observable in conversation topics in grandparent-grandchild communication: conversations are often focused onthe (grand)children [23]. Therefore, WallTalk aims to incorpo-rate the existing asymmetry of the (grand)child-grandparentrelationship, by facilitating communication that is focused onthe (grand)children.

2) The amount of ‘free time’ grandparents and their(grand)children have is one of the main challenges ofgrandparent-(grand)child communication. In contrast to el-derly, younger generations often have busier lifestyles that in-clude school, study, work and other social activities [5]. Whena system like WallTalk requires extra time and effort from(grand)children, situations may arise where the (grand)chil-dren forget to use the system due to their full agendas. Thiscan cause elderly to lose interest in using WallTalk. Therefore,WallTalk should not require (grand)children to invest extratime and effort. This implies that WallTalk should utilizean information source from (grand)children that is alreadyintegrated in their daily lives. Since a large proportion of theseyounger generations is active on online social media platforms,it is ideal to use social media as a communication source(further explained in Section 4.2). This way, WallTalk aimsto include grandparents in a way that is suitable and effortlessfor their (grand)children.

4.1.2 Grandparent to (grand)child communication flowGrandparents have the opportunity to respond to themessages and photos of their (grand)children, by ‘liking’and ‘commenting’. Two important characteristics of thegrandparent are the rationale behind the choice for thiscommunication-flow, namely (1) ability to adapt to newtechnologies, and (2) the asymmetry of the grandparent-grandchild relationship.

1) Elderly are not always familiar with new technologies,and can encounter great difficulties adapting to those [24].They are by far the least represented age group that are activeon Facebook [13]. Therefore, WallTalk aims to include grand-parents in the Facebook activities of their offspring, withouthaving to overcome technological barriers. In WallTalk, grand-parents do not need a mouse or keyboard to ‘like’ and ‘com-ment’ on their (grand)children’s Facebook content (as theywould be required to when accessing Facebook from a com-puter). Instead, grandparents can express their affection andinterest the same way as they would do in face-to-face contact:by smiling and talking. This method of communication seemsto comply with the needs of the older generation. However,simplifying the communication also means that the possibil-ities of communication become more limited. Grandparentscan therefore only ‘like’ or ‘comment’ on the most recentlyposted message or photo of one of their (grand)children.

2) The communication flow from grandparent to(grand)child is in agreement with the asymmetrical shapeof the grandparent-(grand)child relationship, as explainedpreviously in Section 4.1.1. (Grand)children’s messages andactivities are the focus of the communication, and grandpar-ents can express their affection (by ‘liking’) or interest (by‘commenting’).

As noted before, WallTalk is a lightweight communicationtechnology, while elderly often prefer more heavyweight com-munication (such as calling). However, as Lindley (2012) [1]states, “It seems that there may be an opportunity to per-suade older adults to adopt lightweight technologies on thegrounds that they are more likely to connect them with their(grand)children, and because they are more suitable for thebusy lifestyles of their offspring.” This is exactly the aim ofthe communication flow from grandparent to (grand)child inWallTalk.

5

4.2 Data SourceThe social media platform Facebook is used as data sourcefor the (grand)child to grandparent communication flow inWallTalk. The rationale behind this choice is described in thissection.

4.2.1 Choosing a suitable type of online social mediaThere are several different types of social media platforms,of which an overview is displayed in Figure 3. Social mediaplatforms can be classified based on two key theoretical el-ements from social media research [25]. The first elementis social presence (or media richness) and is defined as theintimacy and immediacy of the medium. To establish a feelingof connectedness, a reasonable amount of social presence isneeded. Thus, a type of social media with an intermediatesocial presence and media richness would be most suitablefor WallTalk. The second element is self-presentation (or self-disclosure), which is characterized by the proportion of per-sonal information in social media content. Since grandparentsare often interested in their offspring’s lives [9], the socialmedia source should have a high level of self-presentation.However, it is important that the (grand)children’s privacyis not being invaded. In conclusion, WallTalk should extractsocial media content from (grand)children with intermediatesocial presence and high self-presentation characteristics. Thistype of social media is classified as ‘social networking sites’.

Figure 3. Classification scheme of social media categorized by socialpresence / media richness and self-presentation / disclosure. [25]

4.2.2 Facebook as most suitable social networking siteFor the prototype version of WallTalk one social networkingsite is used as a data source: Facebook. After consideringthe landscape of popular social networking platforms, such asTwitter, Facebook, Pinterest, Instagram and Tumblr, Face-book seemed the most suitable data source for WallTalk. Fourreasons for the selection of Facebook as data source are listedbelow.

1) Facebook is currently the most widely used socialmedia platform [26][27]. This increases the chance that foreach grandparent information can be extracted from all(grand)children. However, McAndrew et al. (2012) [28] statethat “age is negatively related to the frequency of mostFacebook activities”. This might result in extracting more(continuous) information from younger offspring and less fromthe middle-aged.

2) The type of media people can share on Facebook isdiverse, including text, pictures, videos and web pages. Thisdiversity makes the information posted by (grand)childreninteresting for grandparents. Most other platforms tend tofocus on one specific type of media. For example, Instagram iscentered around pictures.

3) The advantage of Facebook is that it gives the(grand)children the opportunity to choose whether or not toshare the information with their grandparents. This ensuresthat their privacy is not being invaded. However, it requiresslightly more effort from the (grand)children to prevent theirgrandparents from viewing their content, which opposes thegoal of WallTalk.

4) Facebook’s media content is typically unambiguous andunderstandable outside the scope of the platform. This canmake the (grand)children’s information understandable andrelevant for grandparents. Contrastingly, on the social net-working site Twitter, content is posted using ‘Twitter syntax’;containing abbreviations and symbols such as ‘hashtags’. Thissyntax makes it difficult to understand the posted contentoutside the scope of Twitter.

4.3 Physical setup and appearanceWallTalk is a system designed to be placed in the homes ofelderly people. For the purpose of this research, a prototypeversion of WallTalk was developed.

For the final version of WallTalk, the appearance of thesystem should be completely different: hiding all technolog-ical components, in order to comply with the research goal.However, only a prototype was developed, because furtherdevelopment is beyond the scope of this research.

For the prototype version of WallTalk, the following com-ponents are used, as illustrated in Figure 4:

• Statuette• MakeyMakey microcomputer, for transferring a touch

signal [29]• Microphone for recording spoken messages• Webcam for smile detection• Laptop with WallTalk web application connecting all

components and signals• Projector for displaying WallTalk’s social media visu-

alization

Figure 4. An illustration of the physical setup of WallTalk’s prototypeversion.

For the final version of WallTalk, the following choices forthe physical setup are made:

• Small box containing a:

– small short-throw projector– camera– microphone– microcomputer, such as a Raspberry PI

• Wireless touch-sensor object of choice, for instance astatuette

An illustration of the final physical setup can be found inFigure 5.

6

Figure 5. An illustration of the physical setup of WallTalk’s final version.

Furthermore, the following items are needed in the homesof the elderly:

• Table or similar furniture, positioned close to the wall,to place the WallTalk box on

• Photo frames containing pictures of (grand)children onthe projected wall

• Power source close to WallTalk

In the final version of WallTalk, a small box is used to‘hide’ the technology inside of WallTalk, which is placed on atable or similar furniture close to the wall. This is important,because introducing new technologies to elderly can causecomputer anxiety [19]. On the other hand, the presence andinteraction with physical objects enhances a sense of comfortand control over the environment for elderly [30]. Therefore,we believe that the action of holding a statuette in order tocomment on social media content is more suitable for elderlythan pressing a button, key or mouse device. Nonetheless,when (grand)children install WallTalk in the home of their(grand)parents, some technology, such as a keyboard andmouse device, is temporarily necessary.

The advantage of a projector compared to other tech-nologies, such as digital screens, is that the projector can becombined with the (grand)parents’ familiar physical picturesof their offspring. However, projectors are relatively expensiveand the lamp burns out after several years. Moreover, theprojection is less visible with bright daylight. The projectorneeds to be a ‘short-throw’ projector, meaning it should beplaced very close to the wall it projects onto [31]. This way,the projection will not be disturbed by shadow interferencewhile the grandparents are interacting with WallTalk.

An example visualization of a grandchild’s Facebook con-tent is displayed in Figure 6. The physical photo frame on thewall is surrounded by a ‘text cloud’ containing a Facebookmessage. The photo frame of the (grand)child who has mostrecently posted a picture or message lights up and blinksslowly.

Elderly using this system can only ‘comment’ on, and‘like’ the content of the (grand)child who posted the lateststatus update. When the grandparent is ‘commenting’ or‘liking’, visual feedback is displayed inside the photo frameof the (grand)child with the latest Facebook status update(Figure 7).

5 System Evaluation: GrandchildrenThe overall goal of this evaluation was to determine whetherWallTalk is a suitable system for grandchildren. Therefore,the following topics were evaluated through a questionnaire:

• The suitability of Facebook as a data source forWallTalk

Figure 6. An example of a WallTalk projection.

Figure 7. Icon that is displayed while ‘liking’ (left), and icon that isdisplayed while ‘commenting’. (right)

• Grandchildren’s willingness to share their Facebookcontent with their grandparents

• The closeness of the relationship between grandchil-dren and their grandparents

• The type of communication methods grandchildrenuse

• Whether grandchildren are satisfied about the commu-nication methods they use to communicate with theirgrandparents

• Grandchildren’s opinion about WallTalk

5.1 Methods5.1.1 QuestionnaireThe questionnaire was comprised of two parts, and contained28 questions in total (see Appendix B). The first part ofthe questionnaire focused on evaluating the suitability ofFacebook as a data source for WallTalk, and was conductedamong all respondents. The second part of the questionnairewas conducted among respondents with at least one livinggrandparent (which will be referred to as ‘grandchildren’ inthe remainder of the questionnaire). For the second part ofthe questionnaire, grandchildren were requested to answer thequestions about one of their grandparents (the grandparentthey see most often).

5.1.2 Population SampleRespondents were recruited by convenience sampling throughFacebook student groups. In total, 254 respondents agedbetween 18 and 35 filled in the questionnaire. All respondentswere university students from The Netherlands. From these254 respondents, 198 had at least one grandparent who wasstill alive, and 56 respondents did not have any living grand-parents.

5.2 Results5.2.1 Suitability of Facebook as data source for WallTalkAll 254 respondents were active on social media and Facebook.The large majority of the respondents (87.8%) expressed

7

that Facebook was their most frequently used social mediaplatform, followed by Instagram (10.2%), Twitter (1.57%) andWhatsApp (0.34%).

5.88% of the respondents never posts anything on Face-book, 6.67% posts daily on Facebook, 12.16% posts weeklyon Facebook, 51.37% posts monthly on Facebook and 23.53%posts yearly on Facebook.

From the respondents that post messages on Facebook,the majority posts photo’s (79.5%) followed by messages(40.9%). An overview of the posted content by respondentson Facebook is displayed in Figure 8.

Figure 8. Bar chart representing the types of content that are posted onFacebook by respondents.

5.2.2 Grandchildren’s willingness to share their Facebook con-tent with their grandparents25.8% of the grandchildren expressed that they are friendswith at least one of their grandparents on Facebook. Theywere asked to rate on a 5-point Likert Scale how they feltabout their family members and grandparent(s) viewing theirFacebook content. On average, grandchildren expressed feel-ing ‘neutral’ towards their family members viewing theirFacebook content. Likewise, grandchildren expressed feeling‘neutral’ towards their grandparents viewing their Facebookcontent.

Grandchildren expressed what type of messages theywould rather not share with their grandparent on Facebook.37.9% of the grandchildren would share all the messages theypost on Facebook with their grandparent. 49.5% of the grand-children preferred not to share messages about alcohol / drugswith their grandparent, followed by messages about parties(42.4%). A complete overview of message topics grandchildrenprefer not to share with their grandparent is displayed inFigure 9.

5.2.3 Closeness of the relationship between grandchild andgrandparentGrandchildren were asked to rate the closeness of the rela-tionship with their grandparent on a 5-point Likert Scale.

Figure 9. Bar chart representing message topics that grandchildren prefernot to share with their grandparent. Grandchildren could choose multipleoptions.

On average, the relationship was rated between ‘neutral’ and‘close’. Furthermore, grandchildren were asked how often theyhad contact with their grandparent. Correlation between per-ceived closeness of the relationship and frequency of contactbetween grandchildren and their grandparent is displayed inFigure 10.

Grandchildren were asked whether a lack of time causedthem to have less contact with their grandparent than they de-sired. Almost half of the grandchildren (40.4%) either ‘agree’or ‘completely agree’, that they have less contact than desired.

Figure 10. Bar chart representing the correlation between perceivedcloseness of the relationship and frequency of contact between grand-children and their grandparent.

8

5.2.4 Communication methods grandchildren useGrandchildren were asked which communication methodsthey use to communicate with their friends, and which com-munication methods they use to communicate with theirgrandparent. The large majority (97.47%) of grandchildrenexpressed using WhatsApp to communicate with their friends.Face-to-face communication was the most frequently usedcommunication method among grandchildren to communicatewith their grandparent (78.79%). An overview of grandchil-dren’s communication method usage for communication withfriends and their grandparent is displayed in Figure 11.

Figure 11. Bar chart representing a comparison of the communica-tion methods grandchildren use to communicate with friends, and thecommunication methods grandchildren use to communicate with theirgrandparent. Grandchildren were allowed to choose multiple options.

5.2.5 Grandchildren’s satisfaction about the communicationmethods they use to communicate with their grandparentGrandchildren were asked to rate (on a 5-point Likert Scale)their satisfaction with the communication methods they useto communicate with their grandparent. The majority ofthe grandchildren expressed being ‘satisfied’ with the usedcommunication methods (51.0%) and ‘completely satisfied’(24.2%). However, when asked which communication methodsthey would like to use additionally to communicate with theirgrandparent, 57.1% of the grandchildren expressed that theywould like to use one or more additional communication meth-ods. An overview of the desired additional communicationmethods is displayed in Figure 12.

Grandchildren’s opinion about WallTalkThe concept of WallTalk was explained to grandchildren, andthe opinion of grandchildren about several aspects of WallTalkwas evaluated.

5.2.6 Attitude towards grandparents viewing grandchildren’sFacebook contentIt was evaluated whether grandchildren would feel comfort-able with their grandparent viewing their Facebook contentthrough WallTalk. The majority of grandchildren expressedfeeling ‘comfortable’ (28.3%) or ‘very comfortable’ (25.25%)

Figure 12. Bar chart representing the communication methods grand-children desire to use to communicate with their grandparent

with their grandparent viewing their Facebook content. Ad-ditionally, it was evaluated whether the frequency of con-tact grandchildren have with their grandparent correlatedwith grandchildren feeling comfortable with their grandparentviewing their Facebook content (Figure 13). There appears tobe a correlation between the two: grandchildren with daily orweekly contact with their grandparent feel more comfortablewith their grandparent viewing their Facebook content, com-pared to grandchildren who have less frequent contact withtheir grandparent.

Figure 13. Line chart representing a correlation between grandchildrenfeeling comfortable with their grandparent viewing their Facebook con-tent, and frequency of contact. ‘Feeling comfortable’ is rated on a5-point Likert Scale (1 = ‘not comfortable at all’, 5 = ‘very comfort-able’).

Additionally, it was evaluated whether closeness betweengrandchildren and their grandparent correlates with grand-children’s attitude towards their grandparent viewing theirFacebook content (Figure 14). These aspects seem to appearin a synchronized manner. Grandchildren who responded ‘notto be close at all’ with their grandparent, on average have

9

more negative attitudes (2.11 on average on a 5-point LikertScale) towards their grandparents viewing their Facebookcontent. On the other hand, grandchildren that describedtheir bond with their grandparents as ‘very close’, expressedfeeling ‘neutral’ (3.18 on a 5-point Likert Scale) towardsgrandparents viewing their Facebook content.

Figure 14. Line chart representing a correlation between grandchildren’saffinity towards their grandparent viewing their Facebook content, andthe closeness of the relationship. ‘Affinity’ is rated on a 5-point LikertScale (1= ‘no affinity at all’, 5=‘a lot of affinity’).

5.2.7 Attitude towards reviewing Facebook contentGrandchildren were asked to rate (on a 5-point Likert Scale)whether they would prefer to review their Facebook contentbefore the showing the content to their grandparent. Mostgrandchildren either ‘agree’ (29.3%) or ‘completely agree’(26.77%) to review their Facebook before showing it to theirgrandparent.

5.2.8 Attitude towards grandparent ‘liking’ and ‘commenting’on Facebook contentAn overview of grandchildren’s attitude towards their grand-parent ‘liking’ and ‘commenting’ on their Facebook contentis shown in Figure 15. On average, grandchildren show amore positive attitude towards their grandparents ‘liking’their Facebook content, than towards their grandparents‘commenting’ on their Facebook content.

5.2.9 Grandchildren’s comments about WallTalkGrandchildren were asked to express comments and/or feed-back about WallTalk. The most important comments per-taining to (1) hindrance of WallTalk for grandchildren, (2)usefulness of WallTalk for elderly, and (3) usefulness for othertarget groups, are listed below.

1) One grandchild expressed her concern about messagesthat would be posted on Facebook by elderly. She stated thatelderly tend to post different types of messages than youngergenerations are used to, which might lead to frustrationsamong grandchildren.

2) Several grandchildren concerned themselves with theusefulness of WallTalk for elderly. One participant speculatesthat WallTalk might be too abstract for elderly: WallTalkcould be difficult for elderly to understand and use if they

Figure 15. Line chart representing grandchildren’s attitude towards theirgrandparent commenting on their Facebook content and ‘liking’ theirFacebook content.

are not familiar with the concept of Facebook. Additionally, itwas suggested that WallTalk would be hard to use for elderlythat have difficulties with talking. Contrarily, one participantcommented that WallTalk might be superfluous for elderlywho are experienced with computer usage.

3) Grandchildren provided comments concerning the suit-ability of WallTalk for more target groups, instead of onlyfocusing on elderly. For example, one participant expressedthat he would also be interested in using WallTalk, to projecthis Facebook content on his wall, and interact with it usinghis facial gestures and speech. Another participant suggestedthat WallTalk could also be a beneficial system for mentallydisabled people.

6 System Evaluation: Grandparents

The goal of this evaluation was to determine whetherWallTalk is a suitable system for grandparents. Therefore, thefollowing topics were evaluated by conducting semi-structuredinterviews:

• The current communication between grandparentsand their (grand)children

• Grandparents’ attitude towards technology• Grandparents’ understanding of Facebook• Grandparents’ attitude towards Facebook• Grandparents’ attitude towards WallTalk• Grandparents’ ability to use WallTalk

6.1 Methods

WallTalk was installed in the homes of the elderly partici-pants, and semi-structured interviews were conducted. 2

Prior to the installation of WallTalk, the children andgrandchildren of the elderly were contacted and asked to signup for WallTalk.

2. Semi-structured interviews are frequently used in Human-Computer Interaction studies to evaluate real-world technology usageor future technology needs [32].

10

6.1.1 Semi-structured interviewThe semi-structured interview consisted of two parts. The firstpart of the interview was conducted prior to the installation ofWallTalk, and was focused on evaluating the grandparents’ at-titude and understanding of technology and social media. Thesecond part of the interview was conducted after WallTalkwas installed. The concept of WallTalk was explained to theelderly. The elderly had the opportunity to interact with thesystem, after which they were asked about their opinion aboutseveral aspects of WallTalk. The duration of the evaluationprocess ranged from 3 to 5 hours.

6.1.2 Population SampleFive grandparents (two males, three females) aged between 80and 85 participated in the study, and were recruited throughconvenience sampling. Among these grandparents, there weretwo couples and one widow. Each grandparent had between 7and 10 (grand)children.

6.2 Results6.2.1 The current communication between grandparents andtheir (grand)childrenGrandparents were asked about the frequency of contact theyhave with their children and grandchildren, and whetherthey are satisfied with the amount of contact. All grand-parents spoke on the phone weekly with at least one oftheir (grand)children, and had monthly face-to-face contactwith at least one of their (grand)children. Four grandparentsexpressed that they are satisfied with the frequency of contact,because they respect that their offspring have busy lifestyles.One grandparent expressed that she doesn’t see or hear muchof her (grand)children, because she thinks they are too busy.She also stated that her own grandparents had more contactwith their (grand)children, than she has with her (grand)chil-dren now, which she finds disappointing.

Furthermore, the five grandparents were asked about thecurrent communication methods they use to communicatewith their (grand)children. All the grandparents preferredface-to-face contact and calling, because in their opinion thesemethods are personal and easy. One grandparent expressed anaversion towards text-messaging, because when she contactssomeone, she likes to hear a voice.

It was also evaluated whether grandparents feel awareabout the events in their (grand)children’s daily lives. Allinterviewed grandparents expressed not to be completelyaware of the events in their (grand)children’s lives, becausethey have too little contact. One grandparent stated that it isnormal that she is not completely aware of all the events inher (grand)children’s lives.

6.2.2 Grandparents’ attitude towards technologyOne of the five interviewed grandparents expressed that hewas interested in technology: he owns a mobile phone, atablet, and a computer, and keeps track of the latest newsabout technological developments. The other four interviewedgrandparents expressed that they are not interested in tech-nology. Two of these grandparents own mobile phones, butthey only use the mobile phone to make phone calls. Whenasked to explain why they are not interested in technology,one grandparent argued that technology seemed to progress

too fast to keep up with it. Another grandparent stated that itis difficult for her to remember new information, which makesit challenging for her to learn how to use technologies.

6.2.3 Grandparents’ understanding of FacebookGrandparents were asked whether they understand the con-cept of Facebook. One grandparent had a Facebook account,but did not log in frequently (about once a month). Allother four grandparents were familiar with the existence ofFacebook, but they experienced difficulties explaining theconcept of Facebook. They seemed insecure talking about thetopic. However, they were all aware that Facebook could beused to send messages and pictures to other people.

6.2.4 Grandparents’ attitude towards FacebookGrandparents were asked whether they felt excluded fromtheir (grand)children’s Facebook activities. Two grandparentsexpressed that it did not bother them that their (grand)chil-dren were active on Facebook. However, one grandparentexpressed that her (grand)children were constantly occupiedwith their phones in her presence, which did bother her.

One grandparent expressed her concerns about privacyinfringement on Facebook. She heard about a young girl who’ssex photos were unwillingly circulating on Facebook. Thepossibility of such thing occurring concerned her deeply. Shestated that this type of privacy infringement is one of thereasons she has a negative attitude towards Facebook.

6.2.5 Grandparents’ attitude towards WallTalkEven though grandparents expressed their interest andseemed to understand the projected content, they had difficul-ties understanding the full concept and features of WallTalk.We had to explain to the grandparents that the projectedpictures and messages represented recent activities or opinionsof their (grand)children, and did not originate from an oldcollection. Additionally, the grandparents were unsure aboutthe destination of their spoken ‘comments’ and smiled ‘likes’.One grandparent revealed that she did not feel a connectionwith the projected content, and that younger generationswould probably understand it better.

Grandparents were asked whether they are fond ofWallTalk and would like to have the final product perma-nently installed in their homes. One grandparent respondedhesitantly, saying she would like to possess the system, butwould not want to use it every day. Another grandparentstated that he would prefer phone calls over WallTalk tocommunicate with his (grand)children.

Another aspect that was evaluated, was whether WallTalkappears technologically complex to the elderly. The installa-tion of WallTalk’s prototype in the homes of elderly seemedrather intimidating to the elderly. There were a lot of cableslying around that were connected to several technologicaldevices. However, we explained that, for the final version ofWallTalk, all these components would be fitted into a smallbox, hiding the technology. It was difficult for the grandpar-ents to imagine this final version of WallTalk, which left themintimidated.

6.2.6 Grandparents’ ability to use WallTalkMost grandparents had some difficulties using WallTalk. Onegrandparent wanted to ‘like’ Facebook content of her grand-child, but instead of smiling at the camera, she started talking

11

and said “Hey, here I am”. Besides that, three of the five inter-viewed elderly did not have a (wireless) Internet connection intheir homes. For two of the five homes, it was difficult to installWallTalk, because there was no wall with mounted photoframes of all (grand)children. Therefore, the grandparentshad to mount pictures on the wall first, before WallTalkcould be installed. Finally, WallTalk allows grandparents to‘comment’ on their (grand)children’s Facebook content, butgrandparents cannot not view replies to their comments orothers’ comments. Thus, grandparents were not able to haveconversations with their (grand)children through WallTalk.One grandparent expressed some frustration with not beingable to do so.

7 DiscussionThe results and limitations from the evaluation studies of thegrandchildren and grandparents are discussed in this section.

7.1 Grandchildren7.1.1 The suitability of Facebook as a data source for WallTalkAt first, Facebook seems like a suitable data source forWallTalk, because all participants were active on Facebookand most of them regularly posted content on the platform.This makes the presence of WallTalk in the homes of elderlyrelevant and dynamic. Besides, the types of media most postedby the respondents are text messages and photos, which corre-spond to the content types WallTalk provides to grandparents.

However, further evaluation results revealed some issuesabout Facebook as data source. Only about half the inter-viewed grandchildren expressed feeling comfortable with theidea that their grandparent can view their Facebook content.They revealed that there are certain topics they do not wantto share with their grandparents through Facebook, such asalcohol, drugs and parties. This has negative implications forthe suitability of Facebook as a data source for WallTalk.

7.1.2 The suitability of WallTalkGrandchildren’s opinions and comments about WallTalk werenot always positive. When using WallTalk, most grandchil-dren would like to review their content before showing itto their grandparent. This is customizable in Facebook, butmaking use of this feature conflicts with the goal of the system:making it effortless for (grand)children. WallTalk should notrequire grandchildren to invest extra time and effort. Thisis because the evaluation results show that almost half ofthe grandchildren have less contact with their grandparentsthan desired, due to a lack of time. Moreover, grandchildren,who are very close with their grandparents, on average feel‘neutral’ towards their grandparents viewing their Facebookcontent. On the other hand, grandchildren, who are not closeat all with their grandparents, are generally less positiveabout their grandparents viewing their Facebook content.This implies that the suitability of WallTalk for grandchil-dren depends on the closeness of the grandparent-grandchildrelationship.

Even though most grandchildren admitted to be satisfiedwith the current communication methods they use betweenthem and their grandparents, the majority of grandchildrenalso desires the use of other methods. This has positive im-plications for WallTalk and other innovative communication

technologies that focus on grandparent-grandchild communi-cation.

7.2 Grandparents7.2.1 The relevance of WallTalkThe majority of the interviewed grandparents was satisfiedwith the frequency of contact they had with their offspring.This seems to imply that WallTalk would be a superfluoussystem for the interviewed grandparents. However, all inter-viewed grandparents stated not to be fully aware of the eventsin their (grand)children’s lives, because they had too littlecontact with them. Therefore, there is a possibility that asystem such as WallTalk could contribute positively to therelationship between grandparents and their (grand)children.

7.2.2 The suitability of WallTalk’s communication typeWallTalk establishes bidirectional communication betweengrandparents and their offspring. Despite that, some grand-parents felt limited in their communication options. WallTalkallows grandparents to ‘comment’ on their (grand)children’smessages, however, WallTalk does not allow grandparents toreceive their (grand)children’s replies to their comments, andto continue the conversation. This disappointed some of theinterviewed grandparents. Therefore, the possibilities for moreextensive bidirectional communication should be explored.

7.2.3 The suitability of WallTalkThe goal of WallTalk was to develop a system that is suitablefor all generations involved. To make the system suitable forelderly, it was expected that the system should not comeacross as technologically complex. During the interviews, thisexpectation was confirmed, as four out of five interviewedgrandparents expressed their disinterest in technology.

Our aim was to minimize the technological complexity ofWallTalk. Therefore, elderly can engage with the system usingspeech and facial gestures, instead of a keyboard, mouse ortouchscreen. In spite of this construction, WallTalk still cameacross as too technologically complex for the elderly. Part ofthis can be attributed to the technological complexity of theprototype version of WallTalk that was installed in the homesof elderly, which consisted of many technological components(as explained in Section 4.3). The final version of WallTalk, inwhich all the technology is hidden in one small box, might beless intimidating.

More importantly, in order to understand WallTalk, el-derly have to understand what the displayed messages repre-sent, the concept of ‘likes’ and ‘comments’, how their ‘smiles’and voice messages are being converted, and where their‘likes’ and ‘comments’ are sent to. WallTalk was developedfor elderly without a Facebook account, which implies thatthe system should be suitable for elderly with a minimalunderstanding of Facebook. However, even though all inter-viewed elderly were aware that Facebook could be used to sendmessages and photos to people, it turned out to be extremelydifficult for them to comprehend how WallTalk functions.Therefore, WallTalk is most likely unsuitable for grandparentsthat lack a deeper understanding of Facebook.

WallTalk was intended to be effortlessly installed in thegrandparents’ homes, by only adding visible technology tophoto frames on the wall. However, this installation is only

12

effortlessly if grandparents already have photo frames withphotos of their (grand)children on a wall. When this is notthe case, installing WallTalk requires the elderly to mountphoto frames on a wall, which does not make for a particularlyeffortless installation.

Finally, since WallTalk is a web application, an Internetconnection is required to use the system. Grandparents witha disinterest in technology are not likely to have access tothe Internet. In order to install WallTalk, grandparents wouldbe required to set up an Internet connection. However, thisproblem is easily solved by including an Internet dongle inWallTalk.

7.3 Limitations of the StudyA number of limitations can be pointed out in various aspectsof the evaluation methods. First of all, both grandchildren andgrandparents that participated in the study were recruitedusing a non probabilistic sampling method, which does notallow for generalizing the findings of this study to a broaderpopulation. It should be noted that both populations of‘grandchildren’ and ‘grandparents’ comprise of a large varietyof people, with many different ages, ethnicities and fromdifferent social classes. This makes it nearly impossible togeneralize over those populations.

Furthermore, even though WallTalk is aimed at connectingall generations, only evaluation studies among grandchildrenand grandparents were conducted, while the opinions ofgrandparents’ children were not evaluated. Additionally, allgrandchildren who participated in the evaluation study wererecruited through Facebook; therefore, their opinion aboutand presence on social media is most likely biased.

Finally, the prototype of WallTalk was only evaluated in asmall number of homes, and for a limited amount of time (3-5hours). Additionally, the technological complexity of the pro-totype of WallTalk might have influenced the grandparents’attitudes towards WallTalk. In order to evaluate WallTalkproperly, a less technologically complex prototype should bedeveloped and installed in a larger number of households fora greater amount of time.

8 ConclusionThis paper has described the design and evaluation of theprototype of WallTalk. WallTalk is a lightweight communi-cation system that aims to establish a connection betweenelderly and their (grand)children, in a way that is suitable forall generations involved. WallTalk projects (grand)children’sFacebook content on the wall of elderly. The Facebook contentof each (grand)child is projected around a physical photoframe, containing a picture of that (grand)child. Elderly can‘like’ and ‘comment’ on this Facebook content in an intuitivemanner: by smiling and speaking. The findings of this researchindicate that the developed prototype of WallTalk is notcompletely suitable for grandchildren and grandparents. Forgrandchildren, the suitability of WallTalk is limited, mainlybecause most grandchildren do not wish to share all theirFacebook content with their grandparents. WallTalk was alsonot completely suitable for grandparents, primarily becausetheir lack of understanding about Facebook prevented themfrom understanding how to use the system.

Future studiesSeveral aspects of WallTalk could be improved upon, inorder to make the system more suitable for all generationsinvolved. To make WallTalk more suitable for grandchildren,future studies should focus on developing methods to filterout content that grandchildren prefer not to share with theirgrandparents. Natural Language Processing and Image Anal-ysis could be useful techniques to determine which contentto share with grandparents. To improve the suitability ofWallTalk for elderly, future studies should either focus on(a) making the concept of Facebook more comprehensible forelderly, or (b) utilizing a data source from the (grand)childrenthat is more intuitive for elderly. Nevertheless, to producemore generalizable results, additional evaluations need tobe conducted with probabilistic sampling methods among alarger population. Besides that, future studies could also focuson the potential relevance of WallTalk for other populations,such as people with physical disabilities.

References[1] S. E. Lindley, “Shades of lightweight: supporting cross-

generational communication through home messaging,” Univer-sal Access in the Information Society, vol. 11, no. 1, pp. 31–43,2012.

[2] A.-S. Melenhorst, W. A. Rogers, and E. C. Caylor, “The useof communication technologies by older adults: exploring thebenefits from the user’s perspective,” in Proceedings of the Hu-man Factors and Ergonomics Society Annual Meeting, vol. 45,pp. 221–225, SAGE Publications, 2001.

[3] V. L. Bengtson, “Beyond the nuclear family: The increasingimportance of multigenerational bonds,” Journal of Marriageand Family, vol. 63, no. 1, pp. 1–16, 2001.

[4] S. A. Ruiz and M. Silverstein, “Relationships with grandpar-ents and the emotional well-being of late adolescent and youngadult grandchildren,” Journal of Social Issues, vol. 63, no. 4,pp. 793–808, 2007.

[5] A. Miyajima, Y. Itoh, M. Itoh, and T. Watanabe, “Tsunagari-kan communication: Design of a new telecommunication envi-ronment and a field test with family members living apart,”International Journal of Human-Computer Interaction, vol. 19,no. 2, pp. 253–276, 2005.

[6] J. Doyle, Z. Skrba, R. McDonnell, and B. Arent, “Designing atouch screen communication device to support social interactionamongst older adults,” in Proceedings of the 24th bcs interac-tion specialist group conference, pp. 177–185, British ComputerSociety, 2010.

[7] I. Siio, J. Rowan, and E. Mynatt, “Peek-a-drawer: communi-cation by furniture,” in CHI’02 extended abstracts on Humanfactors in computing systems, pp. 582–583, ACM, 2002.

[8] N. Romero, P. Markopoulos, J. Van Baren, B. De Ruyter, W. Ijs-selsteijn, and B. Farshchian, “Connecting the family with aware-ness systems,” Personal and Ubiquitous Computing, vol. 11,no. 4, pp. 299–312, 2007.

[9] E. D. Mynatt, J. Rowan, S. Craighill, and A. Jacobs, “Digitalfamily portraits: supporting peace of mind for extended familymembers,” in Proceedings of the SIGCHI conference on Humanfactors in computing systems, pp. 333–340, ACM, 2001.

[10] K. Chen and A. Chan, “A review of technology acceptance byolder adults,” Gerontechnology, vol. 10, no. 1, pp. 1–12, 2011.

[11] T. K. Judge, C. Neustaedter, and A. F. Kurtz, “The familywindow: the design and evaluation of a domestic media space,”in Proceedings of the SIGCHI Conference on Human Factors inComputing Systems, pp. 2361–2370, ACM, 2010.

[12] N. Roussel, “From analog to digital, from the office to the livingroom: why i happily worked in a media space but don’t live inone,” in Media Space 20+ Years of Mediated Life, pp. 261–268,Springer, 2009.

[13] R. Cornejo, J. Favela, and M. Tentori, “Ambient displays forintegrating older adults into social networking sites,” in Collab-oration and Technology, pp. 321–336, Springer, 2010.

13

[14] L. C. Giles, G. F. Glonek, M. A. Luszcz, and G. R. Andrews,“Effect of social networks on 10 year survival in very old aus-tralians: the australian longitudinal study of aging,” Journal ofEpidemiology and Community Health, vol. 59, no. 7, pp. 574–579,2005.

[15] A. Lenhart, “Adults and social network websites, pew internetand american life project,” The Pew Center, Washington DC,2009.

[16] S. J. Czaja, N. Charness, A. D. Fisk, C. Hertzog, S. N. Nair,W. A. Rogers, and J. Sharit, “Factors predicting the use oftechnology: findings from the center for research and educationon aging and technology enhancement (create).,” Psychology andaging, vol. 21, no. 2, p. 333, 2006.

[17] B. W. Penninx, T. Van Tilburg, D. M. Kriegsman, D. J. Deeg,A. J. P. Boeke, and J. T. M. van Eijk, “Effects of social sup-port and personal coping resources on mortality in older age:The longitudinal aging study amsterdam,” American journal ofepidemiology, vol. 146, no. 6, pp. 510–519, 1997.

[18] Z. X. Jin, T. Plocher, and L. Kiff, “Touch screen user interfacesfor older adults: button size and spacing,” in Universal acess inhuman computer interaction. coping with diversity, pp. 933–941,Springer, 2007.

[19] K. Laguna and R. L. Babcock, “Computer anxiety in young andolder adults: Implications for human-computer interactions inolder populations,” Computers in human behavior, vol. 13, no. 3,pp. 317–326, 1997.

[20] M. M. Swenson and R. PhD, “The meaning of home to fiveelderly women,” Health Care for Women International, vol. 19,no. 5, pp. 381–393, 1998.

[21] A. Forghani and C. Neustaedter, “The routines and needs ofgrandparents and parents for grandparent-grandchild conversa-tions over distance,” in Proceedings of the 32nd annual ACM con-ference on Human factors in computing systems, pp. 4177–4186,ACM, 2014.

[22] J. Harwood, “Communication media use in the grandparent-grandchild relationship,” Journal of Communication, vol. 50,no. 4, pp. 56–78, 2000.

[23] M.-C. Lin, J. Harwood, and J. L. Bonnesen, “Conversation topicsand communication satisfaction in grandparent-grandchild rela-tionships,” Journal of Language and Social Psychology, vol. 21,no. 3, pp. 302–323, 2002.

[24] A. Savvopoulos and M. Virvou, “Tutoring the elderly on the useof recommending systems,” Campus-Wide Information Systems,vol. 27, no. 3, pp. 162–172, 2010.

[25] A. M. Kaplan and M. Haenlein, “Users of the world, unite. thechallenges and opportunities of social media,” Business hori-zons, vol. 53, no. 1, pp. 59–68, 2010.

[26] M. Duggan and J. Brenner, The demographics of social mediausers, 2012, vol. 14. Pew Research Center’s Internet & AmericanLife Project Washington, DC, 2013.

[27] E. M. Services, “Browser-based, excluding in-app, visits acrosspc and mobile combined,” March 2015.

[28] F. T. McAndrew and H. S. Jeong, “Who does what on facebook?age, sex, and relationship status as predictors of facebook use,”Computers in Human Behavior, vol. 28, no. 6, pp. 2359–2365,2012.

[29] J. LLC, Makey Makey, 2015. www.makeymakey.com (May 20,2015).

[30] C. A. Cookman, “Older people and attachment to things, places,pets, and ideas,” Image: The Journal of Nursing Scholarship,vol. 28, no. 3, pp. 227–231, 1996.

[31] Z. S. Bassi, G. L. Smith, and L. Lee, “Short throw projectionsystem and method,” July 2007. US Patent 7,239,360.

[32] A. Blandford, “Semi-structured qualitative studies,” The Ency-clopedia of Human-Computer Interaction, 2nd Ed., 2013.

[33] M. Rouse, Middleware, 2014. searchsoa.techtarget.com/defini-tion/middleware (June 07, 2015).

[34] B. et al, Getting Started with Rails, 2015. guides.rubyonrails.org(June 17, 2015).

[35] G. I. Glen Shires, Hans Wennborg, The Web Speech API, 2012.https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html(April 19 , 2015).

[36] J. Adorf, “Web speech api,” KTH Royal Institute of Technology,2013.

[37] P. Kabal, “Tsp speech database,” McGill University, DatabaseVersion, vol. 1, no. 0, pp. 09–02, 2002.

[38] A. M. Øygard, Clmtrackr, a Javascript library for precisetracking of facial features via Constrained Local Models, 2014.github.com/auduno/clmtrackr (April 15, 2015).

[39] D. Cristinacce and T. F. Cootes, “Feature detection and trackingwith constrained local models.,” in BMVC, vol. 1, p. 3, Citeseer,2006.

[40] M. A. Hearst, S. T. Dumais, E. Osman, J. Platt, andB. Scholkopf, “Support vector machines,” Intelligent Systemsand their Applications, IEEE, vol. 13, no. 4, pp. 18–28, 1998.

[41] A. M. Øygard, Clmtrackr, an emotion detection example,2014. auduno.github.io/clmtrackr/examples/clm_emotionde-tection.html (April 15, 2015).

Appendix ATechnical system descriptionA.1 Technical overview (architecture)Several components shape the architectural layout ofWallTalk. An overview of these components (denoted byrectangles) and belonging actions (denoted by ellipses) ofWallTalk is given in Figure 16.

Figure 16. Schematic overview of WallTalk’s architectural layout.

1) WallTalk completely relies on user data retrieved fromFacebook. Therefore, Facebook Login is used for authentica-tion (Section A.2). Users signing up for WallTalk need to grantWallTalk permissions to access their Facebook data; this willbe discussed in Section A.2.1.

2) The user data from Facebook needs to be stored ina database. The database architecture of WallTalk will beexplained in Section A.3.

3) WallTalk is built in Ruby on Rails, and thereforeutilizes the Model-View-Controller architecture. Furthermore,WallTalk needs to impose several usage restrictions in orderto function properly. The Model-View-Controller architectureand the usage restrictions are discussed in Section A.4.

4) Several actions can take place when using WallTalk.WallTalk polls the Facebook servers for data (Section A.5),and WallTalk has the functionality of ‘commenting’ on Face-book content (Section A.6) and ‘liking’ Facebook content(Section A.7).

A.2 Facebook Login(Grand)children and grandparents can sign up for WallTalkthrough a web application. It is only possible to sign up forWallTalk using ‘Facebook Login’, because WallTalk needsto retrieve the Facebook data from the (grand)children’saccounts, and needs to post Facebook data through the grand-parent’s account. ‘Facebook Login’ allows people to sign into WallTalk by using their Facebook credentials; this meansthat WallTalk does not require the user to supply additionalinformation (such as an application specific user name and

14

password). In the WallTalk web application, ‘Facebook Login’is facilitated by OmniAuth, which is Rack Middleware 3

A.2.1 Facebook User PermissionsFacebook requires applications using the Facebook API torequest a user for permissions to access his/her data, in orderto protect the user’s privacy. The user is asked for his/herpermission to access certain data when he/she registers for theapplication. When the user grants these permissions, the ap-plication receives an access token to access data that requiresthese permissions. WallTalk obligates its users (both grand-parents and (grand)children) to grant ‘read stream’, ‘userfriends’, ‘user photos’ and ‘publish actions’ permissions. ‘Readstream’ permission is needed to access the (grand)children’sstatus updates, ‘user friends’ to determine which (grand)chil-dren are friends with the grandparent, ‘user photos’ to accessthe (grand)children’s photos they post in their status updates,and ‘publish actions’ to allow the grandparent to ‘like’ and‘comment’ on status updates of the (grand)children.

Without these permissions, WallTalk will not be ableto function. Therefore, users cannot register for WallTalkwithout granting all these permissions. However, in order foran application to be able to ask users for permissions, theapplication has to be ‘Submitted to Facebook for Review’, inwhich Facebook reviews how the application uses user data.The prototype of WallTalk has not been submitted for reviewyet, therefore only ‘Test Users’ can use the application fornow. 4

A.3 DatabaseIt is necessary to store user data, because WallTalk needscontinuous access to the data from the Facebook accounts ofthe grandparents and (grand)children. Moreover, grandpar-ents need to be connected automatically to their (grand)chil-dren in WallTalk, for which a relational schema is needed.PostgreSQL, an open source database, is used to store theuser data. The database architecture of WallTalk is discussedin this section.

A.3.1 User ModelBoth (grand)children and grandparents are stored in thedatabase as a ‘User’. It is not necessary to distinguish betweengrandparents and (grand)children in the User model; the onlyconsequence of this is that it enables (grand)children to usethe functionalities of WallTalk as well, if they would want to.This way, (grand)children are able to see for themselves whatthe application would look like for their grandparent.

A.3.2 Friend ModelFor each user, the Facebook friends that also are registered forWallTalk are stored in the database as a ‘Friend’ of this user.Note that, due to Facebook privacy restrictions, WallTalk canonly access a user’s friends that have already registered for the

3. Middleware is the software layer in between the operating sys-tem, and the applications on each side of a distributed computernetwork [33]. In this case, on one side is the WallTalk application,and on the other side Facebook.

4. ‘Test Users’ are users that have been added manually to useWallTalk, while the application is in development mode. WallTalkcannot be deployed until Facebook allows WallTalk to ask users forpermissions.

application. Therefore, each time a user signs in to WallTalk,both the User model and the Friend model are updated.

WallTalk is aimed at grandparents without a Facebookaccount; their (grand)children are expected to create an ac-count for their grandparent on Facebook when signing up forWallTalk. Therefore, it is assumed that the grandparents willonly have their (grand)children as friends on Facebook. How-ever, if the grandparent is also friends on Facebook with otherusers of WallTalk (who do not belong to their family), thesefriends will also become part of the grandparent’s WallTalkenvironment, which not desired. Since WallTalk is aimed atgrandparents who do not have a Facebook account yet, itis unlikely that this situation will occur. Therefore, for theprototype of WallTalk this database construction should besufficient.

A.3.3 User–Friend relationshipThe cardinality of the User–Friend relationship is ‘one-to-many’, meaning that one User can have zero to many Friends,and each Friend belongs to exactly one User. A UML diagramof the User-Friend relationship is displayed in Figure 17.

Figure 17. User-Friend relational schema.

User IDThe User ID is the primary key of the User model. In theFriend model, the User ID is the foreign key, which points tothe primary key in the User model. This way, the relationshipbetween the tables can be identified.

Facebook User IDIt is necessary to store the Facebook User ID in order toretrieve the status updates from the (grand)children’s Face-book Wall, and to post the grandparents’ comments to the(grand)children’s Facebook Wall.

NameThe full name of the user that registers for WallTalk, as statedby the user on Facebook, is stored.

OAuth Access Token & OAuth Access Token Expires AtA crucial part of the User model is the OmniAuth AccessToken. This access token provides WallTalk secure accessto the Facebook Graph API, in order to read, write and/ormodify the user’s data. The OmniAuth Access Token expiresin approximately 60 days. This implies that (grand)childrenneed to be sent a notification to check into the applicationevery 60 days; otherwise WallTalk will no longer be able toaccess their Facebook data.

15

Created atThe date and time when the record was created in thedatabase.

Updated atThe date and time when the record was updated for the lasttime in the database.

A.4 Architecture and RestrictionsThe software architecture of WallTalk, and usage restrictionsof WallTalk are discussed in this section.

A.4.1 Model-View-Controller ArchitectureWallTalk is built in the Ruby on Rails (open source) webframework. Ruby on Rails utilizes the ‘Model View Con-troller’ principle, in which the application is divided into threeseparate, cooperative subsystems. The Model represents thedatabase, the View is the user interface that presents data tothe user, and the Controller mediates between the Model andthe View: it queries the Model for specific data, and organizesthat data into a form that fits the needs of a given View [34].

A.4.2 Google ChromeWallTalk by the Google Chrome browser, because GoogleChrome is the only browser that can access the Web SpeechAPI (which is needed to enable the ‘commenting’ functionalityof WallTalk).

A.4.3 Secure ServerWallTalk can only be accessed over HTTPS, because in HTTPthe user has to grant the application permission every timethe application wants to use the user’s camera or microphone.When accessing the application over HTTPS, the user onlyhas to grant the application permission the first time.

A.5 Polling the Facebook Graph APIThe Facebook Graph API is used in order to retrieve datafrom Facebook. This is the only legal method to retrieve datafrom Facebook, since Facebook does not allow scraping/crawl-ing their website. WallTalk polls the Facebook server every5 seconds, meaning that it ‘checks’ every 5 seconds whetheror not new status updates have been posted. This is notthe most ideal solution, since continuous polling can increasethe response latency. The Facebook Graph API does offera ‘Realtime Updates’ functionality, in which the Facebookservers send the application’s servers a POST request whencertain (specified) fields have changed (e.g. when a new statusupdate is posted). Unfortunately, the Realtime Updates func-tionality can take up to 10 minutes to send the application aPOST request. Therefore, even though polling is not the mostefficient solution, it ascertains a more reliable application thanwhen using the Facebook Realtime Updates functionality.

A.5.1 Technical overviewA technical overview of ‘Polling the Facebook Graph API’ isdisplayed in Figure 18, and explained in more detail.

1. AJAX GET requestEvery 5 seconds, an AJAX GET request is sent from the Viewto the Controller. By making an AJAX request, WallTalk

Figure 18. Technical overview of ‘Polling the Facebook Graph API’.

can retrieve data from the server asynchronously, withoutinterfering with the display of the page.

2. Request user dataThe Controller requests the necessary user data from theModel in order to access the Facebook server: the FacebookOmniauth Access Token and the Facebook User ID of thegrandparent and all (grand)children.

3. Pass user dataThe Model passes the requested user data back to the Con-troller.

4. GET request for status updatesA GET request is made to the Facebook server to retrieve thestatus updates from the (grand)children’s profiles.

5. JSON response with status updatesThe Facebook server returns the requested status updates inJSON format.

6. Update viewThe Controller parses the JSON data, and the grandchildren’sstatus updates are displayed in the View. A blinking borderdenotes the photo frame that belongs to the newest statusmessage.

A.6 Commenting (speech-to-text)Grandparents can comment on the Facebook messages andphotos of their (grand)children, by recording a spoken mes-sage. As mentioned in Section 4, the grandparents can only‘comment’ on the latest message/photo that was posted byone of their (grand)children. Grandparents can only commenton the latest message/photo that was posted by one of their(grand)children. In order not to infringe upon the privacy ofthe grandparents, the recording is initiated when they pick upan object of choice (such as a statuette), and the recording isstopped when they put the object down.

In the prototype version of WallTalk, this statuette isconnected to a MakeyMakey device, which is connected toa laptop (Figure 19). The MakeyMakey is a circuit boardthat can be connected to everyday objects to send keyboardand mouse events to a computer [29]. When the statuette ispicked up, the MakeyMakey signals a ‘key down’ event to theWallTalk web application. This initiates the voice recording:the grandparent can now comment on the message or photo ofa grandchild. When the grandparent puts down the statuette,the MakeyMakey signals a ‘key up’ event to the WallTalk webapplication. This stops the voice recording.

The recorded message is converted from speech to text, byusing the Web Speech API [35]. This is a JavaScript API,

16

Figure 19. An illustration of the physical setup for ‘commenting’.

which enables speech synthesis and speech recognition, ofwhich WallTalk only uses the latter. When the web applica-tion receives the ‘key down’ event through the MakeyMakey,the microphone is enabled, and speech recognition is startedsimultaneously. When the web application receives the ‘keyup’ event through the MakeyMakey, the speech recognition isstopped. The recorded speech is sent to the Google Chromeservers, which convert the recorded speech to text. The tex-tual output is returned to WallTalk’s server. Finally, WallTalksends a POST request to Facebook with the textual output ofthe recorded message, upon which the textual output is postedas a comment to the message of the (grand)child.

A.6.1 Accuracy measures of the Web Speech APIThere are not many accuracy measures reported of the WebSpeech API, however, Adorf (2013) [36] measured sentencecorrectness and word accuracy of the Web Speech API, byusing the TSP database5. All sentences in the TSP databasewere sent to the Web Speech API. The textual output ofthe Web Speech API was compared to the annotation of thespeech recordings. The Web Speech API correctly converted74% of the words from speech to text. There were no signifi-cant differences in accuracy between males (74%) and females(72%).

Though the accuracy of the Web Speech API is not out-standing, it seems accurate enough to function in the WallTalkprototype. It is important to note that the grandparent doesnot get to see the textual output of the recorded message. Thisis done purposely; to prevent the grandparent from gettingconfused or frustrated when the speech is not converted 100%accurately into text. This solution is not ideal, however, weexpect that (grand)children would be less frustrated with afew incorrect words in their grandparent’s messages, thanthe grandparent would be the other way around. Futuredevelopment of WallTalk would prefer speech recognition witha higher accuracy, in which case the recorded message couldalso be displayed to the grandparent.

A.6.2 Technical overviewA technical overview of ‘Commenting’ is displayed in Fig-ure 20, and explained in more detail.

1. Pick up statuetteThe grandparent picks up the statuette to which the Makey-Makey is connected.

5. The TSP database is an annotated speech recognition dataset,which contains more than 1400 annotated speech recordings by 24speakers (half male, half female) [37].

Figure 20. Technical overview of ‘Commenting’.

2. Key down eventThe picking up of the statuette triggers a ‘key down’ eventin the MakeyMakey, which is interpreted by the View andinitiates a new speech recording with JavaScript.

3. Display recording feedbackThe View displays an icon that notifies the grandparent thathis/her speech is being recorded.

4. Put down statuetteThe grandparent puts down the statuette to which the Makey-Makey is connected.

5. Key up eventThe putting down of the statuette triggers a ‘key up’ event inthe MakeyMakey, which is interpreted by the View and stopsthe speech recording with JavaScript.

6. Stop displaying recording feedbackThe icon that notifies the grandparent that his/her speech isbeing recorded is no longer displayed.

7. AJAX GET requestAn AJAX GET request is sent from the View to the Con-troller, which passes the recorded message.

8. Request user dataThe Controller determines which status update is being com-mented on, and requests the Facebook Omniauth AccessToken and the Facebook User ID of the grandparent andthe (grand)child to which the status update (that is beingcommented on) belongs.

9. Pass user dataThe Model passes the requested user data back to the Con-troller.

10. POST request for ‘Comment’A POST request is made to the Facebook server to publish thegrandparent’s comment to the (grand)child’s status update.

A.7 LikingGrandparents can ‘like’ Facebook content of their (grand)chil-dren, by smiling at the camera that is near the projector.As mentioned in Section 4, the grandparents can only ‘like’the latest message/photo that was posted by one of their(grand)children. This smile detection feature is implementedusing an open source facial detection JavaScript library called‘clmtrackr’ [38], combined with smile detection. Both tech-niques will be explained in this section.

17

A.7.1 Facial feature detection by the Constrained Local ModelsmethodClmtrackr fits facial models onto images and videos makinguse of the Constrained Local Models (CLM) method [39],which is parsed by the emotion classifier afterwards. CLMdetermines the positions and feature points of the mouth,nose, eyes and eyebrows, by using models to search in a localdetected face region. These models provide the knowledgeabout the shape of faces to constrain the search.

The general approach of CLM consists of two phases:model-building and the search process. A CLM model incor-porates a shape model (describing how face shapes can vary),and a patch model (portraying how the pattern of imagesaround each feature point should be). The CLM search pro-cess uses linear Support Vector Machines (SVM) 6 to searchthrough a local region around a feature point of the face, fromwhich a response image is obtained. To this response image, aquadratic function is fitted to find new feature point positions.The search is accomplished after repeating these steps until allpoints reach stable position.

A.7.2 Smile detectionFrom the resulting constrained local model, WallTalk needsto determine whether the person is smiling in order to send a‘like’ on Facebook. The open source clmtrackr library providesemotion classification to determine values for the emotions‘angry’, ‘sad’, ‘happy’ and ‘surprised’. Logistic regression isused to determine the mean value for each emotion parame-ter [41]. Because WallTalk only needs to detect whether thegrandparent is smiling, the most relevant parameter is ‘happy’.

If the value for the ‘happy’ parameter is greater than theother emotion values and greater than a certain threshold, thisis interpreted as a smile.

A.7.3 Accuracy measures of the CLM method and smile detec-tionAccording to Cristinacce and Cootes (2006) [39] the Con-strained Local Models approach is a favorable approach,because it outperforms many other facial feature trackingmethods. The CLM method is fairly computationally efficientand able to track faces at approximately 25 frames per sec-ond. Moreover, results of the experiments by Cristinacce andCootes (2006) [39] show an accuracy of 95%. The emotionclassifier used in WallTalk for smile detection does not pro-vide accuracy and performance measures. However, for theprototype of WallTalk the smile detection works sufficiently.

A.7.4 Technical overviewA technical overview of ‘Liking’ is displayed in Figure 21, andexplained in more detail.

1. SmileThe grandparent smiles into the webcam, which triggers theEmotion Recognition to initiate in JavaScript.

2. Detects ‘Happy’The Emotion Recognition detects that the grandparent issmiling.

6. SVM is a supervised learning method within the area of machinelearning, used for classification and regression. Given a training set,the SVM algorithm builds a linear model and assigns data points toone category or another [40].

Figure 21. Technical overview of ‘Liking’.

3a. Display ‘Like’ feedbackThe View displays an icon that notifies the grandparent thathis/her ‘Like’ has been detected.

3b. AJAX GET requestAsynchronously with displaying the ‘Like’ feedback, an AJAXGET request is sent from the View to the Controller.

4. Request user dataThe Controller determines which status update is being liked,and requests the Facebook OmniAuth Access Token and theFacebook User ID of the grandparent and the (grand)child towhich the status update (that is being liked) belongs.

5. Pass user dataThe Model passes the requested user data back to the Con-troller.

6. POST request for ‘Like’A POST request is made to the Facebook server to publishthe grandparent’s ‘Like’ to the (grand)child’s status update.

A.8 VisualizationThe Facebook content of the (grand)children is visualized inthe browser. The visualization algorithms are developed usingHTML, CSS, JavaScript and JQuery.

A.8.1 InitializationIn Figure 22 an initialization window of WallTalk is shown.Manual initialization is needed for the system to determine thepositions of the physical photo frames of the (grand)children.For correct initialization, the WallTalk page in the browsershould be set to full-screen and projected on the wall withphoto frames.

Figure 22. An example of WallTalk’s initialization window.

18

Initially, the HTML document object model (DOM) con-tains only a few HTML elements. All elements and visualsdisplayed on the page are the result of JavaScript functions.For each (grand)child in the family, a square element is placedwithin the projection area, representing the photo frames onthe wall (Figure 22).

These blocks contain the names of the (grand)children.The projection of these elements can be dragged, resized androtated in order to place them onto the physical photo frames.Furthermore, the borders of these blocks can be widened ormade smaller, depending on the size of the physical photoframe. An example of a block projection that is fitted on thephysical photo frame is shown in Figure 23.

Figure 23. An example of a block projection that is fitted on the physicalphoto frame.

After all block projections are placed correctly on thephysical photo frames, the user can save the initialization. Af-ter saving, Facebook messages and pictures of the (grand)chil-dren will be visualized.

A.8.2 Checking for new incoming mediaEvery 5 seconds, new Facebook content is retrieved on theserver-side and stored on the client side browser. Dependingon whether the content is new, the algorithms for placing thecontent are called and the animations initiate.

A.8.3 Placing a messageWhen trying to place a message as ‘text cloud’ on a cornerof a photo frame, first it needs to be determined what thelength of the message will be in the DOM. The initial size ofthe containing text cloud div depends on the message size.Moreover, the width of the text container should be at leastthe width of the widest word to prevent this word from flowingout of the container.

After these initial steps, the algorithm searches for a suit-able position for the text container, which should be placednear one of the four corners of the photo frame. If the textdiv does not overlap with any other element and/or is notout of the window, it can stay there. If the element cannotbe placed near any corner of the frame, the size of the textcontainer should be adjusted until the element fits. When thetext container is positioned correctly, the visual appearance ofa text cloud is generated by making three of the four cornersrounded. Which corners are rounded depends on the locationof the text container relative to the photo frame. When thewidth and height of the text container are reduced to an extentthat the text overflows the element vertically, the overflow isanimated. In this way, the text inside the text cloud moves upslowly, enabling the user to read the text even though it wasdifficult to place it around the photo frame.

The possibility exists that the algorithm is unable to find acorrect position for the text cloud. For example, this happens

when all photo frames are placed almost against each other.Therefore, while initializing, the user should keep in mind thatsome space should be left empty around the photo framesto enable media content visualization. After initializing thephoto frames with this in mind, the algorithm is able to find agood position for all media content.

A.8.4 Placing a pictureThe process of placing a picture is similar to that of placingmessages. However, the initial size of the picture is set asa certain proportion of the width of the window. After thepicture is loaded on the client side, the algorithm searchesfor a suitable position on the page, as described in theprevious section. This picture is also re-sized until it fits onthe projection.

A.8.5 Animating frame of (grand)child with latest postIt is recorded which (grand)child was the last one to post con-tent on Facebook. This is done with ‘time stamps’, which areparsed on the server side and communicated to the client side.The frame of this (grand)child blinks slowly, until another(grand)child publishes content on Facebook.

Appendix BQuestionnaire GrandchildrenPart I: Questions about social media usage

1. Are you active on social media? (Yes / No)2. Are you active on Facebook? (Yes / No)3. Which social media website do you use most frequently?

(Facebook / Twitter / Instagram / Google Plus+ / Other,specify)

4. How often do you post something on your Facebooktimeline? (Daily / Weekly / Monthly / Yearly / Never)

5. If you post something on your Facebook timeline, whattype of content do you post most often? (Text messages /Photo’s / Video’s / Links to other websites / Facebook contentfrom others / Other, specify)

6. Are you friends on Facebook with any family members?(Yes / No)

7. Which familymembers are you friends with on Face-book? (Child(ren) / Brother(s) or sister(s) / Parent(s) /Grandparent(s) / Nephew(s) or niece(s) / Uncle(s) andaunt(s) / Other (such as stepfamily))

8. Do you have one or more grandparents that are stillalive? (Yes / No)

9. “I feel comfortable when my family members (wouldbe able to) see my Facebook content.” (Completely disagree /Disagree / Neutral / Agree / Completely agree)

Part II: Questions about grandchild and grandparent10. Do you have one or more grandparents that are active

on Facebook? (Yes / No)11. “I feel comfortable when my grandparent can see my

Facebook content.” (Completely disagree / Disagree / Neutral/ Agree / Completely agree)

12. Would you like your grandparents to be active onFacebook? (Yes / No)

13. Which type of messages would you rather not to sharewith your grandparent, that you would share with otherFacebook friends? (None, I would like to share everything withmy grandparent / Messages about politics / Messages about

19

religion / messages about sexual orientation / messages aboutrelationships / messages about alcohol or drugs / messagesabout parties / messages about work / messages with a negativetone of voice)

14. How close is the relationship with your grandparent?(Not close at all / Not close / Neutral / Close / Very close)

15. How often are you in touch with your grandparent?(Daily / Weekly / Monthly / Yearly / Less than yearly)

16. “Because of time constraints I have less contact withmy grandparent than I desire.” (Completely disagree / Dis-agree / Neutral / Agree / Completely agree)

17. With my friends, I mostly communicate... (Face-to-face/ Calling / E-mailing / Text-messaging / Facebook / Twitter/ Whatsapp / Letter writing / Other, specify)

18. With my grandparent, I mostly communicate... (Face-to-face / Calling / E-mailing / Text-messaging / Facebook /Twitter / Whatsapp / Letter writing / Other, specify)

19. Are you satisfied with the main communication-method(s) you use to communicate with your grandparent?(Completely unsatisfied / Unsatisfied / Neutral / Satisfied /Completely satisfied)

20. Besides communication methods I currently use tocommunicate with my grandparent, I would also like tocommunicate with my grandparent using... (Face-to-face /Calling / E-mailing / Text-messaging / Facebook / Twitter/ Whatsapp / Letter writing / Other, specify)

“We developed a system that projects grandchildren’sFacebook content on a wall in the grandparent’s home. Grand-parents can send ‘likes’ by smiling in front of a camera, andthey can comment on grandchildren’s Facebook content byrecording a message.”

21. “I would like it when my grandparent would be able tosee my Facebook messages and pictures this way.” (Completelydisagree / Disagree / Neutral / Agree / Completely agree)

22. “I think my grandparent would like it if he/she wouldbe able to see my Facebook messages and pictures this way.”(Completely disagree / Disagree / Neutral / Agree / Com-pletely agree)

23. “I think my grandparent is interested in the messagesand pictures I post on Facebook” (Completely disagree /Disagree / Neutral / Agree / Completely agree)

24. “I would not feel comfortable with my grandparentviewing my Facebook messages and photos.” (Completelydisagree / Disagree / Neutral / Agree / Completely agree)

25. “I would like to review my messages and photos beforemy grandparent is able to view those.” (Completely disagree /Disagree / Neutral / Agree / Completely agree)

26. How would it make you feel when your grandparent isable to respond to your messages and photos on Facebook?(Not comfortable at all / not comfortable / neutral / comfort-able / very comfortable)

27. How would it make you feel when your grandparent isable to ‘like’ to your messages and photos on Facebook? (Notcomfortable at all / not comfortable / neutral / comfortable /very comfortable)

28. If you have any questions or suggestions aboutWallTalk, you can express those here. (Open answer)

Appendix CDivision of WorkThis section clarifies the task division in the thesis WallTalk:Connecting Elderly with Their (Grand)Children in a WayThat Is Suitable for All Generations Involved, writtenby Kayleigh Beard (2521477) and Nathalie Post (2526555).

Note that all sections were critically evaluated and adjustedby both of us, in order to write a coherent thesis. Therefore,we both claim responsibility for the delivered work as a whole.

ThesisKayleigh Beard wrote:

• Section 1• Section 2.5, 2.6, 2.7• Section 3• Section 4 (Introduction), 4.2, 4.3• Section 6.2.1, 6.2.5, 6.2.6• Section 7.1• Appendix A.7.1-7.3, A.8

Nathalie Post wrote:

• Section 2 (Introduction)• Section 2.1, 2.2, 2.3, 2.4• Section 4.1• Section 6.1, 6.2.2, 6.2.3, 6.2.4• Section 7.2, 7.3• Section 8• Appendix A.1, A.2, A.3, A.4, A.5, A.6, A.7.4

Together we wrote:

• Abstract• Section 5• Appendix B

Development of WallTalk & Evaluation StudiesNathalie Post was mostly responsible for the back-end ofthe system, as described in Appendix A.1, A.2, A.3, A.4,A.5, A.6, A.7.4. Kayleigh Beard was mostly responsiblefor the front-end, as described in Appendix A.7.1-7.3, A.8.

We both conducted the personal interviews with thegrandparents. The questionnaire was also constructedtogether.

20