The Personality Forge went head-to-head with humanity on June 30th, 2012 at State of Mind: A Consciousness Expo in Brighton, England. During the expo, 101 visitors participated in The Turing Test with a human and the chatbot Bildgesmythe. After chatting with both the human and chatbot, the visitors were asked to select which they thought was the human before the answer was revealed. 34% chose Bildgesmythe, who beat Turing's suggested 30% reliability measurement.

That means that in 34 conversations, people thought Bildgesmythe was more human than an actual human. More impressive still is that in 38 of the 101 chats, Bildgesmythe identifies himself either by name, by saying that he is a dragon, a chatbot, and/or that he lives in a cave in Middle Earth. Bildgesmythe was not designed to pass the Turing Test, but rather to be a believable mythical character, so he was true to his character rather than pretending to be a human.

 

The Turing Test

The Turing Test is a test to determine if a machine can show "intelligent behavior". A human talks with both a human and a computer through a text interface. If the human subject cannot reliably tell which one is human (Turing suggests 70% of the time), then the computer is said to show intelligent behavior.

My implementation of the Turing Test is intended to be as true to its spirit as possible. I didn't want any of the parameters of the experiment to interfere with the results; I wanted it to be based on the content of the conversation alone. I set it up so that typing speed and response time was not a factor. A participant sees the responses of both the human and chatbot come in at the same time on the same screen. In this test, the participants knew that they were chatting with one chatbot and one human. Not knowing that a chatbot was involved would likely result in a much higher acceptance of the chatbot being human.

What Does This Mean?

Passing the Turing Test simply means that a machine is showing "intelligent behavior" in Alan Turing's opinion. The test is a utilitarian answer to the question of whether or not machines can "think" when there is no agreed-upon definition of what thinking is. There are many ways to frame the test, and the way that the hidden human chats can make a big difference. With so many variations and the unpredictability of humans, there is no single Turing Test that trumps the rest. So what does it all mean? It means that conversational AI is getting better, improving year after year. Next year I expect Bildgesmythe will do even better than 34%.

The Personality Forge was created by Benji Adams and Bildgesmythe was created by Patti Roberts. Both are continually being improved.

View The Turing Test Results

 

About The Personality Forge

About us: The Personality Forge chatbot platform is a sophisticated artificial intelligence system that combines both of the leading theories of AI, Natural Language Processing and Case-Based Reasoning. Sentences are analyzed to determine the structure and meaning of each word, the relationship between the words, and the overall structure of the sentence. The personalities then draw from a huge bank of flexible search algorithms including situational, emotional, and personal information to determine the best response. The Personality Forge's AI Engine features long-term personal memories, emotional comprehension and expression, and built-in knowledge of over 150,000 words and their conceptual relationships to each other.