““Theory of Mind” tasks, Theory of Mind” tasks, Human-Computer Human-Computer

Interaction,Interaction,Persistent Conversation & Persistent Conversation &

FlickrFlickr

John C. ThomasJohn C. ThomasIBM T. J. Watson Research CenterIBM T. J. Watson Research Center

January 7, 2008January 7, 2008HICSS, Kona HawaiiHICSS, Kona Hawaii

Persistent Conversation WorkshopPersistent Conversation Workshop

““Theory of Mind” Theory of Mind”

• Of interest to comparative Of interest to comparative psychologists, anthropologists, psychologists, anthropologists, developmental psychologist, AIdevelopmental psychologist, AI

• Can an entity “imagine” what it is like Can an entity “imagine” what it is like for another intelligent entity? (Do they for another intelligent entity? (Do they have a theory of the mind of others?)have a theory of the mind of others?)

Typical Theory of Mind TaskTypical Theory of Mind Task

• Tom watches Susan hide a LAN cable Tom watches Susan hide a LAN cable under the table. Susan leaves.under the table. Susan leaves.

• Later, Tom sees John come in and Later, Tom sees John come in and move the LAN cable behind the drapes move the LAN cable behind the drapes at the back of the room.at the back of the room.

• Now, Susan returns. We ask Tom Now, Susan returns. We ask Tom where he thinks Susan will look for the where he thinks Susan will look for the cable. cable.

General interest is a “capacity” General interest is a “capacity” question: who has it? Who does question: who has it? Who does not?not?

• Apparently, human children do not “have it” till they are 4 or Apparently, human children do not “have it” till they are 4 or 5 years old.5 years old.

• Apparently, Apparently, only only humans have this ability although earlier humans have this ability although earlier tests suggested perhaps dolphins and apes have it.tests suggested perhaps dolphins and apes have it.

• Smart adult humans with Asperger’s Syndrome apparently do Smart adult humans with Asperger’s Syndrome apparently do not have this ability.not have this ability.

• But even “normal” adult humans have limited capacity…I But even “normal” adult humans have limited capacity…I think that Tom believes that Susan wishes that Wendy would think that Tom believes that Susan wishes that Wendy would have forgotten about Tom’s requests for John to inform Susan have forgotten about Tom’s requests for John to inform Susan about Wendy’s desire for longer talks. Huh?about Wendy’s desire for longer talks. Huh?

• In contrast, the ability to follow linear causal sequences In contrast, the ability to follow linear causal sequences seems basically unlimited. seems basically unlimited.

A different set of questions: A different set of questions:

• When When or or under what conditions under what conditions are are people more likely to exhibit this people more likely to exhibit this ability?ability?

• What does this tell us, What does this tell us, in general, in general, about HCI? About conversation? about HCI? About conversation?

• How can we use this to analyze a How can we use this to analyze a particular system like Flickr or a particular system like Flickr or a particular conversation within Flickr? particular conversation within Flickr?

ToM and conversationToM and conversation

• First degree ToM probably necessary for “true” First degree ToM probably necessary for “true” conversation between A and B.conversation between A and B.

• Second degree needed for C to listen to & Second degree needed for C to listen to & understand a simple story about A & B’s understand a simple story about A & B’s conversation about the physical world. conversation about the physical world.

• Third degree needed for D to understand a story Third degree needed for D to understand a story about how A tried to use ambiguity to trick B into about how A tried to use ambiguity to trick B into thinking something untrue about C. thinking something untrue about C.

• Fourth degree needed for E to write a story that Fourth degree needed for E to write a story that intentionally misleads the audience D into intentionally misleads the audience D into believing believing that A was trying to use ambiguity to that A was trying to use ambiguity to trick B into thinking something untrue about C. trick B into thinking something untrue about C.

““Theory of Mind” and HCITheory of Mind” and HCI

• Designing good systems for a Designing good systems for a single user single user requires requires the designers to imagine the user’s tasks, the designers to imagine the user’s tasks, capabilities, context and motivations --- capabilities, context and motivations --- all all different from theirsdifferent from theirs

• One common failing is understanding that what is One common failing is understanding that what is obvious to the project team who have worked on obvious to the project team who have worked on something for months is something for months is not not obvious to someone obvious to someone just beginning to use the systemjust beginning to use the system

• In debates about immigration policy, many native In debates about immigration policy, many native speakers of English seem completely oblivious to speakers of English seem completely oblivious to how difficult acquiring it can be for non-native how difficult acquiring it can be for non-native adultsadults

Social Computing and ToMSocial Computing and ToM

• The designer’s task is now more difficult The designer’s task is now more difficult because they must also understand and because they must also understand and support the users and support the users and their their “theories of “theories of mind” --- issues such as these arise:mind” --- issues such as these arise:– TrustTrust

– ReputationReputation

– Plausible deniabilityPlausible deniability

– Social translucence Social translucence

Flickr profileFlickr profile

• ““This is me.”This is me.”

• ““This is how I want (these) people to This is how I want (these) people to see me.”see me.”

• ““This is how I want (these) people to This is how I want (these) people to imagine that I want to be seen.”imagine that I want to be seen.”

Flickr PictureFlickr Picture

• This is how it is.This is how it is.

• This is how I see it.This is how I see it.

• This is how I want other people to This is how I want other people to see it.see it.

• This is how I want people to see This is how I want people to see me me (e.g., professionalism, choice of (e.g., professionalism, choice of subject)subject)

Flickr conversations about…Flickr conversations about…

• A picture A picture

• A person’s A person’s set set of pictures.of pictures.

• The pictures of all participants around a The pictures of all participants around a topic.topic.

• The definition of what is in and what is out The definition of what is in and what is out of this topic.of this topic.

• How the topic could have/should have How the topic could have/should have been better defined.been better defined.

An interesting example An interesting example becausebecause• Despite the person A setting up the topic or Despite the person A setting up the topic or

contest trying to be “clear” rules contest trying to be “clear” rules a prioria priori

• Almost always debate about what should and Almost always debate about what should and should not be in or out. should not be in or out.

• Furthermore, almost never this reaction:Furthermore, almost never this reaction:

– ““Oh, now I see what Oh, now I see what your your referents are.”referents are.”

– Rather: A: “Well I meant this!” B: “Well, I meant Rather: A: “Well I meant this!” B: “Well, I meant this!” C: “I agree with A because…” D: “I agree this!” C: “I agree with A because…” D: “I agree with B because…”with B because…”

Odd because, there is probably Odd because, there is probably no way no way in a in a finite set of words to specify what is “in” and finite set of words to specify what is “in” and “out”“out”

• Example topics: Example topics:

– Something in which the Something in which the subject subject is blackis black

– A photograph with error or mistake as a A photograph with error or mistake as a themetheme

• What about something “obvious” and What about something “obvious” and “objective” like a picture of a tree?“objective” like a picture of a tree?

Is this a tree?Is this a tree?

How about this?How about this?

Or these? Or these?

Discussion fragments about “The aesthetics of Discussion fragments about “The aesthetics of failure”failure”

• I just quickly scanned the pool and i d say more than 1/2 the pics don t I just quickly scanned the pool and i d say more than 1/2 the pics don t belong. Please people make an effort ..belong. Please people make an effort ..

• You mean - some people are even failing to adhere to the guidelines? That's You mean - some people are even failing to adhere to the guidelines? That's real dedication! real dedication!

• There was a discussion a while ago on this topic, and I think the ultimate There was a discussion a while ago on this topic, and I think the ultimate observation was that while failing to moderate the group would be in line with observation was that while failing to moderate the group would be in line with the meta-ideology of the group, it would also produce a meaningless pile of the meta-ideology of the group, it would also produce a meaningless pile of photos without relevant context, negating the idea of failure beyond "we don't photos without relevant context, negating the idea of failure beyond "we don't keep this clean." keep this clean."

• If "failing to moderate" is "in line", then mustn't there be a posted photo of If "failing to moderate" is "in line", then mustn't there be a posted photo of that particular failing?that particular failing?

Is this group for a discussion on failure or is it based in photography?Is this group for a discussion on failure or is it based in photography?

A photo of a pond, by itself, is not a photo of failure (and there are ponds A photo of a pond, by itself, is not a photo of failure (and there are ponds here)... a photo of an accidental pond in the living room of a fishing tycoon here)... a photo of an accidental pond in the living room of a fishing tycoon because the roof caved in during a storm... flanked by fish tanks...because the roof caved in during a storm... flanked by fish tanks...

...possible candidate...? ...possible candidate...? • what guidelines??? You've failed to post them.... what guidelines??? You've failed to post them....

Further discussion points…Further discussion points…

• May I, as a philosopher specialising in the field of aesthetics, point out that May I, as a philosopher specialising in the field of aesthetics, point out that the Wikipedia definition of aesthetics is somewhat limited and the last the Wikipedia definition of aesthetics is somewhat limited and the last sentence complete crap.sentence complete crap.

How can there be any aesthetics if it is all "a matter of personal taste"? How can there be any aesthetics if it is all "a matter of personal taste"?

• I just joinedI just joined this group. I looked around, but I cannot seem to find the rules this group. I looked around, but I cannot seem to find the rules for posting. I guess you could say I for posting. I guess you could say I failedfailed to find them. (is that part of the to find them. (is that part of the plan?) Little help? plan?) Little help?

• still unclear - are we posting photos of failures or failed photographs?still unclear - are we posting photos of failures or failed photographs?I hope the former as it's much more interesting I hope the former as it's much more interesting

• @GustavoG -- Thank you for the promotion to moderator. I've just re-read @GustavoG -- Thank you for the promotion to moderator. I've just re-read this discussion thread, and it seems that we got off topic. Could you, or any this discussion thread, and it seems that we got off topic. Could you, or any of the other admins, please post a set of guidelines? It seems pointless to of the other admins, please post a set of guidelines? It seems pointless to talk about what the guidelines should be, with this many people -- somebody talk about what the guidelines should be, with this many people -- somebody needs to take the forefront. I think, though, that if we had a set of guidelines needs to take the forefront. I think, though, that if we had a set of guidelines to debate, refining them with this many points of view would make for an to debate, refining them with this many points of view would make for an interesting discussion. interesting discussion.

More comments on “aesthetics of failure”More comments on “aesthetics of failure”

• d_boon_147 says: d_boon_147 says: My last photo was, My last photo was, (http://www.flickr.com/photos/dboon147/1833723848/) as far as I can tell, (http://www.flickr.com/photos/dboon147/1833723848/) as far as I can tell, not accepted to the pool. I've re-entered it into the pool, and I was not accepted to the pool. I've re-entered it into the pool, and I was wondering whether I might post a justification here, see if the admin might wondering whether I might post a justification here, see if the admin might change their mind.change their mind.

If you're going to start talking about failure and whether a photo fits within If you're going to start talking about failure and whether a photo fits within the guidelines (lest we forget "Brakdoewn, slippgae, entrpy, accidetn, gl!the guidelines (lest we forget "Brakdoewn, slippgae, entrpy, accidetn, gl!tczhes"), then you should know that I've thought a lot about this stuff. It's tczhes"), then you should know that I've thought a lot about this stuff. It's not an offhand thing to put a photo in this pool, not thinking about whether not an offhand thing to put a photo in this pool, not thinking about whether it fits. On the one hand, the photograph is the result of an unintended it fits. On the one hand, the photograph is the result of an unintended technical and real-life mistake - the subject moving too fast, and with too technical and real-life mistake - the subject moving too fast, and with too weird a situation of light, for the camera to process - hence fractures, weird a situation of light, for the camera to process - hence fractures, glitching, the failure to produce the intended picture. On the one hand you glitching, the failure to produce the intended picture. On the one hand you can relate this to certain processes in electronic music (Autechre/Oval's can relate this to certain processes in electronic music (Autechre/Oval's use of tech-scrambling, noise musicians' use of battered equipment/circuit-use of tech-scrambling, noise musicians' use of battered equipment/circuit-bending), and on the other you can relate it to certain ideas in art and bending), and on the other you can relate it to certain ideas in art and philosophy (aleatory techniques, Maurice Blanchot's work on the 'art of the philosophy (aleatory techniques, Maurice Blanchot's work on the 'art of the disaster'). I've worked with intended distortion in the past (my Nightlights disaster'). I've worked with intended distortion in the past (my Nightlights set), using it to open up these ideas, by ruptures in the technological, so set), using it to open up these ideas, by ruptures in the technological, so surely it's even more in line with pool guidelines if the result is unintended.surely it's even more in line with pool guidelines if the result is unintended.

Hoping you'll actually include it. Hoping you'll actually include it. Posted 2 months ago. ( permalink ) Posted 2 months ago. ( permalink )

What ultimately is “in” or “out” is a the What ultimately is “in” or “out” is a the result of a social process…disguised as result of a social process…disguised as a logical onea logical one

• About Fortunate MistakeAbout Fortunate Mistake• A group for posting and/or discussing photos A group for posting and/or discussing photos

which were a "fortunate mistake", or "fortunate which were a "fortunate mistake", or "fortunate accident" -- something where you specifically accident" -- something where you specifically didn\'t capture what (or how) you intended, but didn\'t capture what (or how) you intended, but where you like the results.where you like the results.

(with a (with a light sprinkling of the usual rules -- light sprinkling of the usual rules -- left vague for now, and hopefully left vague for now, and hopefully indefinitely.)indefinitely.)

Discussion about fortunate mistakesDiscussion about fortunate mistakes

• Would this include Photoshop mistakes?Would this include Photoshop mistakes?• LarryBLarryB       says:   says: • I ask this because at least once I've made an error adjusting I ask this because at least once I've made an error adjusting

brightness or contrast or other basic adjustments (NOT filters, etc) brightness or contrast or other basic adjustments (NOT filters, etc) and gotten a nice surprise. For example:and gotten a nice surprise. For example: Where I accidentally jogged the contrast slider all the way to the Where I accidentally jogged the contrast slider all the way to the right. right. Originally posted at 5:47PM, 14 February 2006 EST ( Originally posted at 5:47PM, 14 February 2006 EST ( permalinkpermalink ) ) LarryB edited this topic 23 months ago. LarryB edited this topic 23 months ago.

• lindes     says: lindes     says: • Sure... I'm thinking it should start out as a photograph, but Sure... I'm thinking it should start out as a photograph, but

other than that, probably pretty much anything (that's a other than that, probably pretty much anything (that's a "mistake" that you like the results of) goes. "mistake" that you like the results of) goes. Posted 23 months ago. ( permalink ) Posted 23 months ago. ( permalink )

Another example discussion about a group re. Another example discussion about a group re. mistakesmistakes

• Three Hundred Thwarts (and all) -- The Three Hundred Thwarts (and all) -- The ContestContest

• therese flanagan     says: therese flanagan     says:

• There will be guidelines, okay, rules, but There will be guidelines, okay, rules, but they'll come later. Once we hit 295 they'll come later. Once we hit 295 photos I'll post the contest rulesphotos I'll post the contest rules and we'll and we'll let the game begin! let the game begin! Posted at 7:57PM, 19 September 2005 EST Posted at 7:57PM, 19 September 2005 EST ( permalink ) ( permalink )

Conclusion:Conclusion:

• ““Theory of Mind” is not something people either “have” or Theory of Mind” is not something people either “have” or “don’t have”…“don’t have”…

• People have People have some some ability to “see” how the world looks ability to “see” how the world looks through the eyes of othersthrough the eyes of others

• Easiest to do in the spatial domain (hidden object)Easiest to do in the spatial domain (hidden object)

• Harder to do in the temporal domain (learning English; Harder to do in the temporal domain (learning English; understanding how a system seems to users)understanding how a system seems to users)

• Harder yet to do in the conceptual domain (how do Harder yet to do in the conceptual domain (how do definitions in words relate to sets of photos?)definitions in words relate to sets of photos?)

• There may be tools or training or representations to help…There may be tools or training or representations to help…

• Being “good” at ToM may be a critical skill to allow people Being “good” at ToM may be a critical skill to allow people to pull scams, outsmart the competition, “win” at war or to pull scams, outsmart the competition, “win” at war or love, be an effective orator or writer…yet, when and how is love, be an effective orator or writer…yet, when and how is it taught?it taught?