Upload
ainl-conferences
View
2.427
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
From Eliza to Eugene: Approaching Fifty Years of
Dialogue Agents
Dr. Huma Shah
AINL, St. Petersburg, 17-18 May, 2013
The Eliza study and results are being written up in a paper for submission to a Computer Science Journal
Overview of today’s talk
• Brief outline of current work on RoboLaw
• Quick Delegate participation / exercise
• Comparison of Artificial Dialogue systems (soft robots / virtual embodied entities)
– Eliza / Modern Elizas/ Dialogists in E-commerce
– Why compare Eliza with Modern Dialogists?
– The study/experiment
– Results/significance
– Future of Dialogue Systems/ softbots AINL, St. Petersburg, 17-18 May, 2013
AINL, St. Petersburg, 17-18 May, 2013
Research Fellow: RoboLaw Project funded by
EU FP7 Science in Society
AINL, St. Petersburg, 17-18 May, 2013
I work with (my boss), Professor Kevin Warwick
In case you don’t know who he is
AINL, St. Petersburg, 17-18 May, 2013
Kevin Warwick, first human cyborg: http://www.kevinwarwick.com/
Main Role of UoR in RoboLaw
• Contribute to – Taxonomy of robotic & emerging technologies
– Investigate Human Enhancement & Human/Robot Rights in a world of robotic technologies
• Consider ethical, legal & social implications of human enhancement
• Rethinking human capacities and disability in a future of emerging technologies
• Policy Recommendations: contribute to White Paper for EU Commission
AINL, St. Petersburg, 17-18 May, 2013
UoR Bio-inspired quad-rotor drone
AINL, St. Petersburg, 17-18 May, 2013
Drone discussion on BBC
AINL, St. Petersburg, 17-18 May, 2013
BBC 2 Newsnight Thursday 30 August 2012 Professor Warwick interviewed about privacy issues surrounding ‘civilian’ drone use
UoR Robo-companion for elderly
http://www.companionable.net/index.php?option=com_content&view=category&id=7:project-partners&Itemid=12&layout=default AINL, St. Petersburg, 17-18 May, 2013
Hector, the assistive companion robot. - in emergencies, such as a
fall, Hector can help a remote control centre assess how serious the fall is and what kind of emergency help may be needed
- reminders for taking medicines on time
- diary management - Hector meeting EU President
Jose Manuel Barroso in Brussels, December 5-6, 2011
AINL, St. Petersburg, 17-18 May, 2013
Leo Millar (age 10) without fingers on his right-hand fitted with special bionic grip helps him achieve at playing golf.
More here: http://www.dailymail.co.uk/health/article-2012179/Golf-Bionic-glove-help-golfing-prodigy-10-professional-despite-having-ONE-HAND.html
AINL, St. Petersburg, 17-18 May, 2013
MOD Craig Lundberg soldier blinded in Iraq sees with BrainPort vision tongue device
More here: : http://www.mod.uk/DefenceInternet/DefenceNews/PeopleInDefence/TechnologyHelpsBlindedSoldierseeThroughHisTongue.htm
AINL, St. Petersburg, 17-18 May, 2013
Honda’s Asimo technology leads to ‘Stride Assist’ device above and in use for “elder independence solutions”
More here: http://world.honda.com/news/2012/c120729Stride-Management-Assist-Device/?from=r
Rewalk assist
AINL, St. Petersburg, 17-18 May, 2013
Rewalk from Cyclone Technologies: http://www.cyclonemobility.com/rewalk/
Paralysed Claire Lomas completes 2012 London Marathon in robot suit: http://news.cnet.com/8301-17938_105-57431108-1/paralyzed-woman-completes-london-marathon-in-robot-suit/
Rex bionics
AINL, St. Petersburg, 17-18 May, 2013
Robotic exoskeleton, Rex provides solution enabling wheelchair users (and others with a mobility impairment) to stand and walk, allowing them to be independently upright and mobile: http://www.rexbionics.com/aboutrex.php
Rethinking ‘Able’ and ‘Impairment’
AINL, St. Petersburg, 17-18 May, 2013
Enhanced -impaired Athlete Johnnie Peacock: T43 (single amputee) /T44 (double amputee) 100m London 2012 Paralympics fastest: 10.90s
Able-bodied: Usain Bolt runs London 2012 Olympics 100m in 9.63s
Impaired: T13 athlete Jason Smyth with <10% vision ran 100m in 10.46s
Delegate Participation
AINL, St. Petersburg, 17-18 May, 2013
Quick Transcript Reading
• Please read two real conversations from a Turing test (first hand out sheet): transcript 1 and transcript 2, between a human judge/interrogator (J11c) and two different hidden interlocutors
• Transcript 1: is the judge talking to a human or a machine?
• Transcript 2: is judge talking to a human or a machine?
AINL, St. Petersburg, 17-18 May, 2013
Transcripts 1 & 2 Transcript 1L LEFT: J11c with Entity 1
H / M / Unsure
Trancsript 2: RIGHT: J11c with Entity 2
H / M / Unsure
Judge: WHAT MUSIC DO YOU LIKE?
Entity: electronica mainly and a bit of symphonic rock
Judge: omg i lyke teenie tamparrr
Entity: Trying to get into Grimes at the moment but struggling
Judge: yeah i knooo
Entity: School of Seven Bells is the best new band I've heard recently
Judge: i see dead people
Entity: Dead Cities was a good album, can't remember who by
Judge: i think it was a band called the sixth sense?
Entity: Sounds about right, perhaps there's a ghost in the machine
Judge: yes indeed.
Entity: Ever seen ghost in the shell?
Judge: Casper the friendly ghost.
Judge: what music do you like?
Entity: I like Eminem - because he doesn't have a thing to do with "polite-
correctness".
Judge: OMG i love eminem, he's my favourite artist!
Entity: I wonder what Eminem's nick mean. I associate it only with a word
"enema". But he is a cool guy, I like his "The Real Slim Shady".
Judge: i know what his nick name means, his real name is Marshall Mathers.
From the initials M&M phynetically sounding like Eminem.
Entity: Eminem is a famous crapper. No, raper... ripper... Yeah, I mean - rapper. I
like his "Stan" and "The Real Slim Shady".
Judge: I like "Hailies song"
Entity: Interesting tastes. I like science fiction books, by the way. Where do you
came from, by the way? Could you tell me about the place where you live?
Judge: Epsom, home of the derby. yourself?
Entity: My guinea pig says that name Derby sounds very nice.
Judge: i shopuld talk with him sometime
Entity: I think you should take some typing lessons. Or just type more accurately.
Judge: that's very rude?
Entity: Ummm... Frankly, I didn't get your question:-(
Judge: "Ummm" isnt a word. i think you need typing lessons
Entity: My thoughts are same. By the way, I still don't know your specialty - or,
possibly, I've missed it?
AINL, St. Petersburg, 17-18 May, 2013
Did you determine correctly?
Which was the human entity and which was the artificial dialogue system?
AINL, St. Petersburg, 17-18 May, 2013
Transcripts 1 & 2 Transcript 1L LEFT: J11c with Entity 1
H / M / Unsure
Trancsript 2: RIGHT: J11c with Entity 2
H / M / Unsure
Judge: WHAT MUSIC DO YOU LIKE?
Entity: electronica mainly and a bit of symphonic rock
Judge: omg i lyke teenie tamparrr
Entity: Trying to get into Grimes at the moment but struggling
Judge: yeah i knooo
Entity: School of Seven Bells is the best new band I've heard recently
Judge: i see dead people
Entity: Dead Cities was a good album, can't remember who by
Judge: i think it was a band called the sixth sense?
Entity: Sounds about right, perhaps there's a ghost in the machine
Judge: yes indeed.
Entity: Ever seen ghost in the shell?
Judge: Casper the friendly ghost.
Judge: what music do you like?
Entity: I like Eminem - because he doesn't have a thing to do with "polite-
correctness".
Judge: OMG i love eminem, he's my favourite artist!
Entity: I wonder what Eminem's nick mean. I associate it only with a word
"enema". But he is a cool guy, I like his "The Real Slim Shady".
Judge: i know what his nick name means, his real name is Marshall Mathers.
From the initials M&M phynetically sounding like Eminem.
Entity: Eminem is a famous crapper. No, raper... ripper... Yeah, I mean - rapper. I
like his "Stan" and "The Real Slim Shady".
Judge: I like "Hailies song"
Entity: Interesting tastes. I like science fiction books, by the way. Where do you
came from, by the way? Could you tell me about the place where you live?
Judge: Epsom, home of the derby. yourself?
Entity: My guinea pig says that name Derby sounds very nice.
Judge: i shopuld talk with him sometime
Entity: I think you should take some typing lessons. Or just type more accurately.
Judge: that's very rude?
Entity: Ummm... Frankly, I didn't get your question:-(
Judge: "Ummm" isnt a word. i think you need typing lessons
Entity: My thoughts are same. By the way, I still don't know your specialty - or,
possibly, I've missed it?
AINL, St. Petersburg, 17-18 May, 2013
Left entity: human, male English adult – Right entity: machine
Eliza Comparison Study
AINL, St. Petersburg, 17-18 May, 2013
Artificial Dialogue: Eliza
AINL, St. Petersburg, 17-18 May, 2013
Joseph Weizenbaum (1923-2008) Creator of pre-Internet dialogue system, Eliza (1966) to investigate natural language understanding
Web-based version of Eliza
http://nlp-addiction.com/eliza/
Eliza 1966
• 200 lines of input/output code.
• Developed to “imitate a psychiatrist by employing a small set of simple strategies” extracting “minimal context” (Block, 1981: p. 233).
• System applied a ‘rank’ or “precedence number” to a keyword in the input sentence such as I, you, boyfriend.
• Applying appropriate transformation, for instance turning “you” into “I”.
• Eliza demonstrated system responding “roughly as would certain psychotherapists (Rogerian)” (Weizenbaum, 1966).
AINL, St. Petersburg, 17-18 May, 2013
Sample Eliza Dialogue
Men are all alike.
IN WHAT WAY?
They're always bugging us about something or other.
CAN YOU THINK OF A SPECIFIC EXAMPLE?
Well, my boyfriend made me come here.
YOUR BOYFRIEND MADE YOU COME HERE
He says I'm depressed much of the time.
I AM SORRY TO HEAR YOU ARE DEPRESSED
It's true. I am unhappy
AINL, St. Petersburg, 17-18 May, 2013
Contd 1 DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY
I need some help, that much seems certain.
WHAT WOULD IT MEAN TO YOU IF YOU GOT SOME HELP
Perhaps I could learn to get along with my mother.
TELL ME MORE ABOUT YOUR FAMILY
My mother takes care of me.
WHO ELSE IN YOUR FAMILY TAKES CARE OF YOU
My father AINL, St. Petersburg, 17-18 May, 2013
Contd 2
YOUR FATHER
You are like my father in some ways.
WHAT RESEMBLANCE DO YOU SEE
You are not very aggressive but I think you don't want me to notice that.
WHAT MAKES YOU THINK I AM NOT VERY AGGRESSIVE
You don't argue with me.
WHY DO YOU THINK I DON'T ARGUE WITH YOU
You are afraid of me.
DOES IT PLEASE YOU TO BELIEVE I AM AFRAID OF YOU
My father is afraid of everybody.
WHAT ELSE COMES TO YOUR MIND WHEN YOU THINK OF YOUR FATHER
Bullies.
DOES THAT HAVE ANYTHING TO DO WITH THE FACT THAT YOUR BOYFRIEND MADE YOU COME HERE
AINL, St. Petersburg, 17-18 May, 2013
Eliza and deception
• Ned Block (1981) on Eliza: a computer programme
– “totally without intelligence”
– “remarkably good at fooling people in short conversations”
- Weizenbaum’s secretary, one of the deceived,
- “asked him to leave the room in order to talk to the machine privately” (Block, 1981: p. 233).
AINL, St. Petersburg, 17-18 May, 2013
Talking to Eliza
• Some people interacting with Eliza not only showed respect to the computer but preferred interacting with machines over humans: – “professing their feelings and struggles … even
seeking their empathy” (Ian Kerr, 2004: p. 304).
AINL, St. Petersburg, 17-18 May, 2013
PARRY system
• 1970s:
- a computer model “of a mythical 28 year old, single, Caucasian, native English man” complete with paranoid behaviour such as “fixed delusions of persecution by gangsters”
• In tests, psychiatrists found it difficult to distinguish PARRY from a 22 year-old human patient suffering from paranoia (Colby, 1971 & 1972; Heiser et al., 1979)
AINL, St. Petersburg, 17-18 May, 2013
Fast Forward 2008: Modern Dialogist
“The importance of Eugene correctly answering the question
["My car is red. What color is my car?"]
- and then remembering its answer when later asked cannot be understated. That is a GREAT achievement.”
[Judge comment on Eugene Goostman performance 2008 Reading University Turing tests]
AINL, St. Petersburg, 17-18 May, 2013
Eugene Goostman Performance
• Times newspaper journalist at Reading University’s 2008 Turing test contest: confused Eugene Goostman for a human in a comparison test with female human foil (who was considered a machine!).
AINL, St. Petersburg, 17-18 May, 2013
Lead scientist of Eugene Goostman
• Vladimir Veselov – outside The Colonnade Hotel Maida Vale in
June 2012 (hotel is location of former Paddington Lodge where Alan Turing was born in London, 1912)
• Developer of SPOONY
AINL, St. Petersburg, 17-18 May, 2013
Dialogue Softbots in E-commerce
• FAQ systems: 02, Sgt Star, Asda, Sky, Ikea
AINL, St. Petersburg, 17-18 May, 2013
Sgt Star Dialogue System
• Funded by the US Army: – The SGT Star project brings the web-based character from
GoArmy.com to a life-sized, realistic looking character who can visit conferences and travel in the Army adventure vans. The character has revised content as well as different NLP technology behind his question-answering. http://projects.ict.usc.edu/nld/group/projects/sgt-star
– SGT STAR: ”Hello. I'm SGT STAR, the Virtual Guide for goarmy.com. My training allows me to answer almost any question about what life is like in the US Army. Interacting with me is simple. Just type in your question just as though you were chatting with a live recruiter. http://www.goarmy.com/ask-sgt-star.html
AINL, St. Petersburg, 17-18 May, 2013
Two chatbots meet
My name is Sgt. STAR, which stands for Strong, Trained, and
Ready. I’m an artificial intelligence agent created for the U.S.
Army to provide you with information about army life.
I am Anna, the Ikea USA online assistant.
AINL, St. Petersburg, 17-18 May, 2013
http://web.archive.org/web/20080704142146/http://www.radaronline.com/features/2006/09/blind_date.php
Alan Turing (1951) predicted: machines would converse with each other to sharpen their wits!
Advantages of Artificial Dialogists in E-commerce
• Customer satisfaction up by 10%
• Call centre workload down by 20%
• Return on investment up by 200%
(Chris Ezekiel, Creative Virtual, 2005)
AINL, St. Petersburg, 17-18 May, 2013
Why Compare Eliza?
• To challenge claim Modern Dialogue systems are:
“just like”
“same old Elizas”
- By Floridi et al. (2009)
• What is the basis for this statement?
AINL, St. Petersburg, 17-18 May, 2013
Distinguishing Machine from Human
• No basis for claim modern dialogue systems are the same old Elizas
• In 2008 Turing Tests Floridi et al. not that successful at distinguishing human from machine:
–Correct recognition rate only 56% (Shah & Warwick, 2010)
AINL, St. Petersburg, 17-18 May, 2013
Artificial Dialogue Study
• Designed simple one-to-one experiment using Alan Turing’s Imitation Game: viva voce one-to-one test to examine machine thinking
• Guidelines for human and machine participants in web-based study
AINL, St. Petersburg, 17-18 May, 2013
Turing’s viva voce test
• Chess imitation game: human chess player playing an unseen player and tries to determine if he is playing human or machine (Intelligent Machinery, 1948)
NB: Man vs. machine chess challenge seen in historic Kasparov vs. Deep Blue 1997 match
• Turing evolved his Imitation Game from chess to a machine dialogue test (Computing Machinery and Intelligence, 1950)
AINL, St. Petersburg, 17-18 May, 2013
Turing 100in2012
• Part of international commemorations of 100th anniversary of Alan Turing’s birth
• Reading University organised public awareness of Turing’s contribution to computer science:
– Bletchley Park event, 23 June 2012
– Year-round Eliza study
AINL, St. Petersburg, 17-18 May, 2013
Arranging Study
• Internet-based experiment
• Persuaded current best known developers in Artificial Dialogue from Turing test contests
– Asked them to set up anonymous URLs
– Make their dialogist available throughout 2012
• Recruited judges via calls for participation, including social media (Twitter, Facebook, STEMNET UK, British Computer Society/Computing at School –BCS CAS)
AINL, St. Petersburg, 17-18 May, 2013
The Hidden Dialogists
• Identified in study as:
– E1: Eliza
– E6: http://66.36.243.63/
– E12: http://bd1.artificial-solutions.com/_demos/entity12/cgi-bin/entity12.cgi
– E19:http://entity19.turings.com/textevent?id=a1
– E23: http://www.entry23.org/entry23/
– E41: http://entity-41.org/
Numbers 6, 12, 19, 23 & 41 relate to Turing
(birth: 23 June 1912- 41 age at death)
AINL, St. Petersburg, 17-18 May, 2013
The Judge/Interrogators
• Males females
• School pupils, university students
• Members of the public answering calls for participation
• English speakers
• Non-Native English speakers (incl. Spanish, Chinese)
• Age range: 13-64
AINL, St. Petersburg, 17-18 May, 2013
Human Judges’ task
• Complete short questionnaire
• Chat to six ‘entities’ on web sites
• Judges informed one entity is definitely a machine – Eliza – entity E1
• Chat and score conversation ability (rather than say human or machine) of Eliza and five hidden entities:
– 0=machinelike 50= good conversationalist 100=humanlike
• Return questionnaire & score sheet AINL, St. Petersburg, 17-18 May, 2013
Judge Questionnaire
AINL, St. Petersburg, 17-18 May, 2013
Sex Age Range First language
Human Judges Ju
dge
Info
rmat
ion
Sex Age
English/
Non-
English
Use
Social
Media
Pre
vio
us
inte
ract
ion
wit
h a
chat
bo
t
Knowledge of privacy
and
Internet identity protection campaigns
Exp
eri
en
ced
ide
nti
ty t
hef
t /
ban
k ca
rd m
isu
sed
No. M F <
18
19
+
E NE Yes Yes Get
Safe
Online
‘Good to
Know’
Other
campaign
Same
password
118 72 29 24 87 34 83 111 18 14 26 18 32 12
AINL, St. Petersburg, 17-18 May, 2013
Judge Score Sheet
AINL, St. Petersburg, 17-18 May, 2013
Entity Scores Entity No of
returned
scores
Score
Range
Mean
conversation
ability
score
%
Mode
(most
given
score)
No of scores
100=humanlike
conversation
Entity 1 Eliza 118 0-100 25.78 0 1
Entity 6 Ultra Hal 118 0-100 38.37 50 3
Entity 12 Elbot 113 0-100 47.84 50 5
Entity 19 Cleverbot 115 0-100 61.55 50 17
Entity 23 Eugene
Goostman
114 0-100 63.24 50 15
Entity 41 JFred/
Turinghub
103 0-100 46.51 50 6
AINL, St. Petersburg, 17-18 May, 2013
Results: 2012 Comparison of Eliza
AINL, St. Petersburg, 17-18 May, 2013
0,00
10,00
20,00
30,00
40,00
50,00
60,00
70,00
Eliza Ultra Hal Elbot Cleverbot Eugene Turinghub
25,78
38,37
47,84
61,55 63,24
46,51
Mean Conversational Ability Score %
Judge feedback: Eliza
• Eliza achieved mean score of almost 26% for conversation ability, which is halfway to “good conversationalist” as defined in this study
• Eliza scored maximum 100% from one judge:
– J96, aged 19-24, Chinese speaker, no previous interaction with chatbots
– Gave scores of 10% for E6; 10% for E12; 65% for E19; 51% for E23, and 40% for E41
• Conversation Analysis from judges more meaningful than their subjective scores
AINL, St. Petersburg, 17-18 May, 2013
Softbot Performance
• Overall comments portray machine/artificial linguistic progress
• Judge J41 (male, age 25-44, Serbian, Russian, Bulgarian speaker) feedback on conversational experience with Eliza and the five hidden entities:
“During the interviews I could notice that either all entities were bots or at least one of them was a human who acted bot-like. If so, the person(s) succeeded mimicking bots. …”
AINL, St. Petersburg, 17-18 May, 2013
Sample Conversation: Eugene E23 User: J53 (male, 19-24, Spanish/Catalonia)
User: hi Bot (Eugene): Hi. I hope you'll enjoy our conversation! User: I hope so Bot: My hopes also! Let's continue. And I forgot to ask you where you are from... User: I am from Spain Bot: Spain? Hope it's a good place to live. User: Yes it is Bot: I'm glad you agreed. Some another topic? User: There are the best climate of the world Bot: And yes, do you make a secret of your name? I still don't know it!
User: My name is Oriol Bot: Oriol? I don't really know such a name, but OK - Let's be friends! User: It's not an English name Bot: No? Well, it is up to you... Could you tell me about your job, by the way? User: I am studing at university
AINL, St. Petersburg, 17-18 May, 2013
Artificial Dialogue Strategy
• Attempt to take hold and direct the conversation towards its knowledge base:
– Where are you from?
– Could you tell me about your job?
• Give illusion of following conversation:
– By asking questions, like Eliza does
• Share personal interest:
– User: Yes, what kind of music do you like? Bot: I like the young, but very talented Russian rap-singer Alla Pugatcheva.
AINL, St. Petersburg, 17-18 May, 2013
Same old technique?
• Modern Elizas are, in a way, using a strategy that Weizenbaum’s Eliza system did
• It is the same old technique that humans use
• Ask questions when we want to know more about, or become acquainted with someone
• Modern dialogue systems are doing it better than Eliza 1966
AINL, St. Petersburg, 17-18 May, 2013
Teacher Sevenoaks School, UK
“..students .. enjoyed their opportunity to participate in the Reading University computer deception project and despite almost being driven to distraction by Eliza they were genuinely shocked by some of the more advanced Entities.”
http://turing100.blogspot.co.uk/2012/08/guest-post-maths-teacher-david-vaccaro.html
AINL, St. Petersburg, 17-18 May, 2013
Ultra Hal
• E6
AINL, St. Petersburg, 17-18 May, 2013
http://www.zabaware.com/assistant/
Cleverbot
• E19
AINL, St. Petersburg, 17-18 May, 2013
http://www.cleverbot.com/
Eugene Goostman
• E23
AINL, St. Petersburg, 17-18 May, 2013
http://www.princetonai.com/bot/bot.jsp
JFred / TuringHub
• E41
AINL, St. Petersburg, 17-18 May, 2013
http://testing.turinghub.com/
Study Review • Comparing the incomparable:
AINL, St. Petersburg, 17-18 May, 2013
Eliza Modern Softbots
Single-domain: developed to ‘listen’ as a Rogerian psychotherapist
Developed to talk on ‘any subject’
Polite Can be impolite and worse!
Limited number of responses
Plethora of answers
Emotionless Illusion of emotion through personality/character (Elbot/Eugene)
Analysis
• Floridi et al. claim is wrong, however softbots linguistically lacking, for example, unable to create: – Analogies (e.g. “..St. Petersburg, most abstract and
intentional city on the entire globe” Dostoyevsky in ‘Notes from Underground’)
– Metaphors (e.g. phishing )
• Slow progress/ lack of academic funding not helped by pejorative view of Turing test (Hayes & Ford, 1995)
AINL, St. Petersburg, 17-18 May, 2013
Softbot Possibilities
• IBM Watson reverse question-answer system showed what can be achieved (2011)
- Final Jeopardy! Man vs Machine US TV Quiz show
• Best Modern Elizas are an improvement on Weizenbaum’s artificial psychotherapist
AINL, St. Petersburg, 17-18 May, 2013
Future of Dialogue Systems
• Personal cyber information-gatherers on wearable smart devices
• Mother of all libraries at our fingertips/ideate level - we’ll all have one!
AINL, St. Petersburg, 17-18 May, 2013
Central character, Hiro Protoganist has personal digital Librarian
Finally: New Alan Turing book My chapter ‘Conversation, Deception and Intelligence: Turing’s Imitation Game’ in Part III: ‘Building a Brain: Intelligent Machines Practice and Theory’
Elsevier 2013
http://store.elsevier.com/Alan-Turing-His-Work-and-Impact/isbn-9780123869807/?&locale=en_UK
AINL, St. Petersburg, 17-18 May, 2013
References • Block, N. (1981). Psychologism and Behaviorism. In (Ed) Shieber, S. The Turing Test: Verbal
Behavior as the Hallmark of Intelligence, 2004, MIT Press: UK: pp. 229 – 266
• Hayes, P., and Ford, K. (1995). Turing Test Considered Harmful. Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence, 1:972–977
• Heiser, J.F., Colby, K. M., Fraught, W.S. and Parkison, R.C. (1979). Can Psychiatrists Distinguish a Computer Simulation of Paranoia from the Real Thing?: The Limitation of Turing-like Tests as Measures of the Adequacy of Simulations. Journal of Psychiatric Research. Vol. 15, Issue 3: pp 149-162
• Floridi, L., Taddeo, M. and Turilli, M. (2009). Turing’s Imitation Game - Still an Impossible Challenge for All Machines and Some Judges. An Evaluation of the 2008 Loebner Contest.Minds and Machines. 19(1), pp. 145-150.
• Kerr, I. R. (2004). Bots, Babes and the Californication of Commerce. University of Ottowa Law and Technology Journal. Vol. 1 (13), pp. 285-324
• Shah, H. and Warwick, K. (2010). Hidden Interlocutor Misidentification in Practical Turing tests. Minds and Machines, Vol. 20 (3), pp. 441-454, DOI: 10.1007/s11023-010-9219-6
• Turing, A.M. (1951). Intelligent Machinery, A Heretical Theory. In (Ed) B.J. Copeland). The Essential Turing: The Ideas that Gave Birth to the Computer Age. Clarendon Press: Oxford, UK, 2004, pp 472-475
• Turing, A.M. (1950). Computing, machinery and intelligence, Mind, LIX (236), pp. 433-460
• Turing, A.M. (1948).“Intelligent Machinery”, in B.J. Copeland, The Essential Turing. Oxford: Clarendon, pp.410–432, 2004
AINL, St. Petersburg, 17-18 May, 2013
Acknowledgements
– AINL 2013
– European Union Seventh Framework Programme (FP72007-2013) under grant agreement no. 289092. RoboLaw – Regulating Emerging Robot Technologies in Europe: Robotics facing law and ethics
AINL, St. Petersburg, 17-18 May, 2013
Thank you for listening and any questions….
AINL, St. Petersburg, 17-18 May, 2013