BROWSE TOPICS
RESOURCESABOUT THIS SITE |
The Turing Test: "Can Machines Think?"The Original Test and Derivatives AITopics > Overview > The Turing Test
Good Starting PlacesComputing Machinery and Intelligence. By Alan M. Turing. "Originally published by Oxford University Press on behalf of MIND (the Journal of the Mind Association), vol. LIX, no. 236, pp. 433-60, 1950. Published on the abelard site by permission of Oxford University Press." An all-time classic paper in AI that discusses the prospects of AI and dismisses some still-current arguments against AI. Definition of Turing Test. From the University of Alberta Cognitive Science Dictionary, maintained by Dr. Michael Dawson and Dr. David Medler. The Turing Test. By Graham Oppy & David Dowe. Entry in the Stanford Encyclopedia of Philosophy. "The phrase 'The Turing Test' is most properly used to refer to a proposal made by Turing (1950) as a way of dealing with the question whether machines can think. ... The phrase 'The Turing Test' is sometimes used more generally to refer to some kinds of behavioural tests for the presence of mind, or thought, or intelligence in putatively minded entities. So, for example, it is sometimes suggested that The Turing Test is prefigured in Descartes' Discourse on the Method. (Copeland (2000:527) finds an anticipation of the test in the 1668 writings of the Cartesian de Cordemoy. Gunderson (1964) provides an early instance of those who find that Turing's work is foreshadowed in the work of Descartes.) ... There are many different objections to The Turing Test which have surfaced in the literature during the past fifty years, but which we have not yet discussed. We cannot hope to canvass all of these objections here. However, there is one argument -- Searle's 'Chinese Room' argument -- that is mentioned so often in connection with the Turing Test that we feel obliged to end with some discussion of it." More on the Turing TestThe Turing Test. By Lynellen D.S. Perry. ACM Crossroads Student Magazine. A critical look at the test with many helpful links. On the hunt for universal intelligence (January 27, 2011). ""We have developed an 'anytime' intelligence test, in other words a test that can be interrupted at any time, but that gives a more accurate idea of the intelligence of the test subject if there is a longer time available in which to carry it out", José Hernández-Orallo, a researcher at the Polytechnic University of Valencia (UPV), tells SINC. This is just one of the many determining factors of the universal intelligence test. "The others are that it can be applied to any subject – whether biological or not – at any point in its development (child or adult, for example), for any system now or in the future, and with any level of intelligence or speed", points out Hernández-Orallo." Story about: José Hernández-Orallo y David L. Dowe. "Measuring Universal Intelligence: Towards an Anytime Intelligence Test". Artificial Intelligence 174(18): 1508, Dec 2010. Available for a fee from Science Direct]] DOI: 10.1016/j.artint.2010.09.006. (Provided by FECYT - Spanish Foundation for Science and Technology) It's the thought that counts. By Dylan Evans. Guardian (October 6, 2001). "Will machines ever be able to think for themselves? And will we be able to tell if and when they do? Pondering these questions in 1950, the British mathematician Alan Turing came up with a simple way of settling the matter. Put a machine in one room, he suggested, and a human being in another. ... Turing hoped that his test would cut through a lot of fruitless semantic debate. It was an engineer's solution, rather than a philosopher's. " The article ends by challenging you to decide which of the excerpted conversations were conducted with humans and which with non-humans. Look What's Talking: Software Robots With Chatterbots on the Web, Conversation Can Be Surprising, or Surprisingly Limited. (March 18, 1999) By David Pescovitz. The New York Times on the Web; available from Web Lab. "In the near future, chatterbots are expected to act as the voices of other Web-based intelligent agents, generally called bots, which gather data or perform other tasks automatically for users. ... The yardstick for judging machine intelligence is whether it can play what the British mathematician Alan M. Turing called an "imitation game," now known universally as the Turing test." If Not Turing’s Test, Then What? (Winter, 2005). By Paul R. Cohen. AI Magazine 26(4): Winter 2005, 61–67. "Turing’s test is not irrelevant, though its role has changed over the years." Being Real. By Judith S. Donath, MIT Media Lab. [To appear in Goldberg, K. (ed.) The Robot in the Garden: Telerobotics and Telepistemology in the Age of the Internet, MIT Press.] "This essay approaches these issues by focusing on a question with special resonance for both technologists and philosophers: can one tell if the person at the other end of an online discussion is indeed a person?" Some challenges and grand challenges for computational intelligence. By Edward A. Feigenbaum. Journal of the ACM (JACM), Volume 50 , Issue 1, Pages: 32 - 40 (January 2003). Available from ACM and KurzweilAI.net. "The Turing Test is a very ambitious Grand Challenge. The 'Feigenbaum Test' is more manageable: focus on natural science, engineering, or medicine with conversation in the jargonized and stylized language of these disciplines." Accelerating Problem Solving. Listen to this presentation by David Fogel, CEO of Natural Selection, delivered at the Accelerating Change 2005 conference, and made available by IT Conversations: "The Turing Test is often seen as an accurate indication of machine intelligence. David Fogel refutes this by quoting Alan Turing himself and by showing how current internet use undermines the traditional application of the test's results. In place of this flawed definition, Fogel suggests one of his own; namely, that intelligence may be viewed as the ability to adapt behavior to meet goals in a range of environments." The Trouble with the Turing Test. (Winter, 2006). By Mark Halpern. The New Atlantis Number 11, Winter 2006, pp. 42-63. [This article is an abridged version of a paper - complete with footnotes - which can be accessed from the articles section of the author's website.] "[W]hat Turing grasped better than most of his followers is that the characteristic sign of the ability to think is not giving correct answers, but responsive ones -- replies that show an understanding of the remarks that prompted them. If we are to regard an interlocutor as a thinking being, his responses need to be autonomous; to think is to think for yourself. The belief that a hidden entity is thinking depends heavily on the words he addresses to us being not re-hashings of the words we just said to him, but words we did not use or think of ourselves -- words that are not derivative but original. ... The complicated relationship between the field of AI and Turing’s legacy goes back to the beginning." Talking Heads...A Review of Speaking Minds: Interviews with Twenty Eminent Cognitive Scientists. By Patrick J. Hayes and Kenneth M. Ford. AI Magazine 18(2): Summer 1997, 123-125. " While the interviews do not conform to a set pattern, they return regularly to a few favorite themes: the Chinese Room, the importance of the Turing Test...." The Turing Test. From Chapter One (available online) of George F. Luger's textbook, Artificial Intelligence: Structures and Strategies for Complex Problem Solving, 5th Edition (Addison-Wesley; 2005). "The Turing test measures the performance of an allegedly intelligent machine against that of a human being, arguably the best and only standard for intelligent behavior. ... The Turing test, in spite of its intuitive appeal, is vulnerable to a number of justifiable criticisms. One of the most important of these is aimed at its bias toward purely symbolic problem-solving tasks. It does not test abilities requiring perceptual skill or manual dexterity, even though these are important components of human intelligence. Conversely, it is sometimes suggested that the Turing test needlessly constrains machine intelligence to fit a human mold. Perhaps machine intelligence is simply different from human intelligence and trying to evaluate it in human terms is a fundamental mistake. Do we really wish a machine would do mathematics as slowly and inaccurately as a human?" Can Computers Think? Mapping Great Debates. Visit MacroVU's site and if you are patient, you can preview their "7 poster-sized argumentation maps that chart the entire history of the debate. The maps outline arguments put forth since 1950 by more than 380 cognitive scientists, philosophers, artificial intelligence researchers, mathematicians, and others." Map # 2 is titled: Can the Turing test determine whether computers can think? What's It Mean to Be Human, Anyway? By Charles Platt. Wired Magazine 3 (4): Features. "Our purpose is to find out whether 10 judges can tell the difference between humans and artificial-intelligence programs, when they are online at the same time. ... The inspiration for this event dates back to the earliest days of computing. In 1950, pioneer Alan Turing proposed that if a computer could successfully impersonate a human being during a free-form exchange of text messages, then for all practical purposes, the computer should be considered intelligent." Human or Computer? Take This Test. By Sara Robinson. The New York Times (December 10, 2002; no-fee reg. req'd). "As chief scientist of the Internet portal Yahoo, Dr. Udi Manber had a profound problem: how to differentiate human intelligence from that of a machine. His concern was more than academic. Rogue computer programs masquerading as teenagers were infiltrating Yahoo chat rooms, collecting personal information or posting links to Web sites promoting company products. ... The roots of Dr. Manber's philosophical conundrum lay in a paper written 50 years earlier by the mathematician Dr. Alan Turing, who imagined a game in which a human interrogator was connected electronically to a human and a computer in the next room. The interrogator's task was to pose a series of questions that determined which of the other participants was the human. ... Dr. Manuel Blum, a professor of computer science at Carnegie Mellon who took part in the Yahoo conference, realized that the failures of artificial intelligence might provide exactly the solution Yahoo needed. Why not devise a new sort of Turing test, he suggested, that would be simple for humans but would baffle sophisticated computer programs. Dr. Manber liked the idea, so with his Ph.D. student Luis von Ahn and others Dr. Blum devised a collection of cognitive puzzles based on the challenging problems of artificial intelligence. The puzzles have the property that computers can generate and grade the tests even though they cannot pass them. The researchers decided to call their puzzles Captchas, an acronym for Completely Automated Public Turing Test to Tell Computers and Humans Apart (on the Web at www.captcha.net)." Take a moment and a raise a glass to the wonderful, underappreciated AI. Andrew Kantor's CyberSpeak column. USAToday.com (June 1, 2006). "AI does more than make better games ... What Far Cry illustrates is how far artificial intelligence has come. It's so sophisticated that we almost dismiss it. In a way, that's a sign of their quality. Invisible tech is often the best tech. ... Because Google doesn't talk like HAL 9000, we don't think of it as AI. Working with its own algorithm and the data input by millions of users every time they search, Google is able to help you find information on the billions of pages of the Web in a matter of seconds. Or less. ... Another example: When I check my e-mail, Thunderbird deletes almost all of the incoming spam. It does this not by looking for obvious spam words, but by using artificial intelligence - in this case Bayesian filtering to create a detailed profile of each message. Based on what it's learned - yes, learned - about the mail I receive, it can tell it how likely any given message is legit. If you drive a modern car, your vehicle's artificial intelligence is doing a lot for you - quietly and behind the scenes, of course. ... So while we're waiting for our computers to have meaningful conversations with us, take a moment to appreciate the underappreciated AI - and be glad its not trying to kill us - much." Related ResourcesSee section on Biographical Information on Turing. The Age of Intelligent Machines: "A (Kind of) Turing Test". By Ray Kurzweil. "The essence of the Turing Test is that the computer attempts to act like a human within the context of an interview over terminal lines. A narrower concept of a Turing test is for a computer to successfully imitate a human within a particular domain of human intelligence. We might call these domain-specific Turing tests. One such domain-specific Turing test, based on a computer's ability to write poetry [including haiku], is presented here." The Loebner Prize CompetitionThe Loebner Prize: "The First Turing Test." "Dr. [Hugh] Loebner pledged a Grand Prize of $100,000 and a Gold Medal for the first computer whose responses were indistinguishable from a human's. Each year an annual prize of $2000 [NOTE: the prize will be $3000 for the 2005 contest] and a bronze medal is awarded to the most human computer. The winner of the annual contest is the best entry relative to other entries that year, irrespective of how good it is in an absolute sense. Further information on the development of the Loebner Prize and the reasons for its existence is available in Loebner's article In Response to the article Lessons from a Restricted Turing Test by Stuart Shieber." Contest information, past winners and transcripts can be accessed via links that can be found by scrolling down the Loebner Prize home page. 2011 Loebner Prize. (October 27, 2011). "The 2011 Loebner Prize competition is over and the winner is... Rosette, by Bruce Wilcox, who scored 1.5 and wins the bronze medal and $4000 USD. None of the entries fooled the judges, so no silver or gold medal was awarded. " Chatterbots, Tinymuds, And The Turing Test: Entering The Loebner Prize Competition. (1994) By Michael L. Mauldin. Presented at AAAI-94. Downloadable PDF file -- click on abstract title. "The Turing Test was proposed by Alan Turing in 1950; he called it the Imitation Game. In 1991 Hugh Loebner started the Loebner prize competition, offering a 100,000 prize to the author of the first computer program to pass an unrestricted Turing test. Annual competitions are held each year with smaller prizes for the best program on a restricted Turing test. This paper describes the development of one such Turing System, including the technical design of the program and its performance on the first three Loebner Prize competitions. We also discuss the program’s four year development effort, which has depended heavily on constant interaction with people on the Internet via Tinymuds (multiuser network communication servers). Finally, we discuss the design of the Loebner competition itself, and address its usefulness in furthering the development of Artificial Intelligence." Lessons from a Restricted Turing Test. Stuart M. Shieber. Communications of the Association for Computing Machinery (1994), Volume 37, Number 6, pages 70-78. "We report on the recent Loebner prize competition inspired by Turing's test of intelligent behavior. The presentation covers the structure of the competition and the outcome of its first instantiation in an actual event, and an analysis of the purpose, design, and appropriateness of such a competition." Also available from the author's collection of publications is his rejoinder to Loebner's response.  Epstein, Robert. 1992. The Quest for the Thinking Computer. AI Magazine 13(2): Summer 1992, 81-95. "Can machines think? Alan Turing’s decades-old question still influences artificial intelligence because of the simple test he proposed in his article in Mind. In this article, AI Magazine collects presentations about the first round of the classic Turing Test of machine intelligence, held November 8, 1991 at The Computer Museum, Boston. Robert Epstein, Director Emeritus, Cambridge Center for Behavioral Studies, and an adjunct professor of psychology, Boston University, University of Massachusetts (Amherst), and University of California (San Diego) summarizes some of the difficult issues during the planning of this first real-time competition, and describes the event. He then speculates about the future of the competition and about its significance to the AI community. Presented in tandem with Dr. Epstein’s article is the actual transcript of session that won the Loebner Prize Competition--Joseph Weintraub’s computer program PC Therapist." The Chinese Room ArgumentChinese Room Argument. Entry by John R. Searle in the MIT Encyclopedia of Cognitive Science. "The Chinese room argument is a refutation of strong artificial intelligence. 'Strong AI' is defined as the view that an appropriately programmed digital computer with the right inputs and outputs, one that satisfies the Turing test, would necessarily have a mind."
Evaluation of Knowledge-Based SystemsShortliffe, Edward H. 1984. The Problem of Evaluation. In Rule Based Expert Systems: The MYCIN Experiments of the Stanford Heuristic Programming Project, ed. Buchanan, Bruce G. and Edward H. Shortliffe, 571-589 (Chapter 30). Reading, MA: Addison-Wesley Publishing Co., Inc. The entire book is now available online from AAAI's Classic Books in AI collection.
Offline ReferencesBibliography on the Turing Test by the AISB Quarterly. Alper, G. 1990. A Psychoanalyst Takes the Turing Test. Psychoanalytic Review 77 (1): 59-68. Bleich, H. L. 1995. Alan Turing: The Machine, the Enigma, and the Test. MD Computing 12 (5): 330. Cohen, Paul R. 1995. Empirical Methods for Artificial Intelligence. Cambridge, MA: MIT Press. An in-depth treatment of statistical measures of performance for AI programs. Colby, K., S. Weber, and F. Hilf 1971. Artificial Paranoia. Artificial Intelligence 2: 1-25. Cowley, Stephen J. and Karl F. MacDorman, "Simulating Conversations: The Communion Game." AI & Society (1995) 9:116-137. Crockett, Larry J. 1994. The Turing Test and the Frame Problem. Norwood, NJ: Ablex Publishing Co. Dennett, Daniel. 1998. Brainchildren. Contains an excellent discussion of the Turing Test. Cambridge, MA: MIT Press. Ford, Kenneth, and Patrick Hayes. 1998. On Computational Wings: Rethinking the Goals of Artificial Intelligence. Scientific American Presents 9 (4): 78-83. Drawing the analogy that airplanes fly without mimicking birds, the authors conclude that AI can and already has contributed via machine intelligence, and that requiring it to mimic human intelligence is unecessary and even harmful to scientific research. Hayes, Patrick, and Kenneth Ford. 1995. Turing Test Considered Harmful. Proceedings of the IJCAI-95, August 20-25, 1995, in Montreal, Quebec. Volume 1, page 972. The authors, who consider Turing to be one of the greatest scientists of the 20th century, examine problems with the Turing Test and conclude that a goal of passing the test is harmful to the field of AI research. Heiser, J. F., K. M. Colby, W. S. Faught, et al. 1979. Can Psychiatrists Distinguish a Computer Simulation From the Real Thing? The Limitations of Turing-like Tests as Measure of the Adequacy of Simulations. Journal of Psychiatric Research 15 (3): 149-162. Hodges, Andrew. 1983. Alan Turing: The Enigma of Intelligence. New York: Simon & Schuster. Johnson, George. 1997. The Artist's Angst is All in Your Head. New York Times, (Late NY Edition/November 16, 1997): Section 4, p.16. Millican, Peter and Andy Clark, editors. 1999. Machines and Thought: The Legacy of Alan Turing, Volume I. Oxford University Press - USA. The Table of Contents is available online. Moor, James H. (Ed.) 2003. aboutThisBook | The Turing Test - The Elusive Standard of Artificial Intelligence. Studies in Cognitive Systems Series, Vol. 30. Springer. Entire book devoted to discussions of the Turing Test. Putnam, Hilary. 1988. Much Ado About Not Very Much. Daedalus 117 (1): 269-281. Reprinted in The Artificial Intelligence Debate: False Starts, Real Foundations, edited by Stephen Graubard, 1988. Quittner, Joshua. 1997. What's Hot in Bots. Time Magazine 150: 31. Waltz, David L. 1988. The Prospects for Building Truly Intelligent Machines. In The Artificial Intelligence Debate, ed. Graubard, Stephen R., Cambridge, MA: The MIT Press. |
