|
October 15, 2002

Chatty Bits
Software programs competed in Atlanta to be mistaken for humans in the
Turing test - The winner received a consolation prize.
By Hubertus Breuer (authorized translation, Fred Roberts)
Atlanta has often led the world. The pharmacist John Pemberton invented Coca
Cola here. Furthermore, Martin Luther King’s birthplace stands in the capital
city of Georgia, as does that of the news broadcaster CNN. And last
Saturday the city once again showed the world the way into the future. At
Georgia State University, in the middle of the city’s downtown, an annual
ritual in computer research was celebrated, the finale of the 12th Loebner
Prize: Programs attempted to convince judges that they possess intelligence and
consciousness, or that they can at least simulate these mental qualities.
Richard Wallace with his program "Alice" had also qualified for the finale. The
programmer from San Francisco had won the prize in 2000 and 2001 and was
striving for a third victory. Among computer freaks he has achieved cult
status, among other reasons because Steven Spielberg used the program to
advertise his movie "A.I.".
A crowded finale was to be anticipated. Via the Website interested parties were
advised to show up at seminar room 608 before 10 AM. The competition would be
broadcast to this location, according to the Website. But by 10:20 AM the
auditorium is empty, the projector powered down. "You’ll have to go to Sparks
Hall, room 137," says the only person far and wide. "There you’ll find the
judges with the computers." The original announcement stated that this venue
would remain secret. But since the sole reporter remains the sole observer the
strict rules are dropped.
In Sparks Hall, down to the basement, straight ahead, second corridor to the
right, four forms disappear into a windowless computer room. Judges, as it
turns out. At the moment there is still nothing to do: the monitors on which
the programs should run glow blue and blank. Neil Bishop of Humimics, the
software company setting up the spectacle this year, rushes through the
steeplechase. "The network isn’t up yet," he apologizes.
For the founder of the prize, Hugh Loebner, it’s nothing new. With his twisted
full beard he sits on a chair: "Things like this happen almost every year," he
says calmly. The Ph.D. in sociology, free thinker and champion for the
legalization of prostitution operates a company in New Jersey for theater
accessories. A good twelve years ago he established the prize, but has himself
seldom tested one of the so called chatbots. The mathematician Alan
Turing, who in 1950 wrote a paper on artificial intelligence, served as the
inspiration for the Loebner foundation.
Turing suggested an "imitation game": Judges at terminals begin a conversation
and then have to rate whether or not a human is at the other end typing the
answers, or if a machine computed them. If a program successfully passes as a
human, it has succeeded at the Turing test: for this achievement $25,000 wave
from Loebner’s strongbox. But to date none of the chatbots come close to
misleading humans. Therefore, as a consolation prize, the most human program
receives $2,000, a bronze medallion - and the fame. Richard Wallace can
appreciate this. He wears one of his medallions around his neck.
But the 12th awarding of the prize turned out to be a lesson in scurrility.
This begins with the participants: Chris Cowart claims at his Website that his
creation "Talkbot" is a robot he took home with him from a crashed UFO. His
competitor Joy Harwood ("God Louise") views herself as a chatbot, because, due
to a learning disorder, she has trouble following spoken dialogue. And the
mother of a third finalist tells how she breastfed her son for two years after
birth.
Then the organization falls apart: next to the computer programs three humans
are to participate in the competition to fulfill Turing’s stipulations. The
connection to the first participant in London is lost, the second is using the
university library and must leave as they close their doors. And instead of the
desired twelve judges, Bishop could only find six.
But the dialogues running before the judges on their monitors are completely
dadaistic. Young Christine Scarborough, who works for Humimics, begins the
conversation with an assault: "Hello. My name is trouble," she begins. The
programmers have only tolerably prepared their programs for such tricks. At one
point "Ella," for example, abruptly begins talking about a supposed trip to Las
Vegas.
On the other hand, Wallace, who watches over the juror’s shoulders, has taught
"Alice" another strategy. His assumption: human conversation is not complex:
"You could say that everyday conversation consists of a small set of questions
and answers." However, the set is not that small. Over the years Wallace
has typed 40,000 sentence-long answers into his "Alice," which simulates
intelligence on the basis of a dictionary with synonyms and other elementary
grammar.
But "Alice," too, changes the subject and makes fun of her own machine nature
when she doesn’t understand a sentence - just as all the participants. After
three questions the judges usually know they’re conversing with software. Then
they just poke around in the brains of bits and bytes, in order to rank the
chatbots and the humans on a scale of one to ten.
By afternoon the nervousness in the basement has died down. Only the three
programmers who, as Wallace, see their mental children tested, are unable to
relax. After completion of the task, the first judge leaves the arena; he wants
to watch a football game. As the last judge turns in his scribbled hit list
Neil Bishop begins to add. Finally he grabs the microphone. "A close win by
"Ella" by Kevin Copple from Tianjin, China", he announces. "Ella," not "Alice,"
the favorite. Wallace doesn’t show his feelings, but he swiftly packs his
things.
The victory celebration doesn’t materialize. The winner did not appear
personally, and anyhow, nobody has a medallion at hand. Only Wallace had worn
one. As the computer with the winning program is to be photographed, the
disappointed loser is asked for a loan. Caught up with somewhere on campus he
gives the piece only reluctantly out of his hands.
In this way the memorable event can be documented in word and picture after
all. Sunday night the Website announces the results. And Dwight Harshbarger,
director of the Cambridge Institute for Behavioral Studies in Massachusetts,
under whose patronage the Loebner prize officially stands, announces he will
publish a report about the competition. He wasn’t there.
|
 |