Rose is a 31-year-old security analyst and hacker from San Francisco, a self-described “computer nerd” with “quirky attitudes” towards life, and an “unorthodox family.”
She doesn’t like for people to know too much about her because her work makes her aware of how “under surveillance we all are,” but she also has a “flamboyant and fun-loving side” and likes to drink socially.
She may sound like someone you know, or maybe even someone you’d like to date, but she’s not actually human.
Rose is a chatbot, and a very good one – she won recognition this past Saturday as the most human-like chatbot in a competition described as the first Turing test, the Loebner Prize.
Rose won only a bronze medal in the competition rather than gold, because – although she was more human-like than the other bots in the competition – she didn’t fool any of the judges into thinking she was a real person.
Which means that the Loebner Prize in Artificial Intelligence’s $100,000 (£64,000) jackpot will remain unclaimed for another year.
The Loebner Prize, founded in 1990 by inventor, philanthropist and activist Hugh Loebner, seeks to answer the question posed by computer scientist Alan Turing – can a computer think?
The judges for the Loebner Prize carry on two simultaneous chatroom-style conversations – one with an actual human being, the other with a chatbot.
Within 25 minutes the judges have to decide which is which.
No AI has fooled all of the competition’s judges yet, but Rose isn’t very easy to trick into giving herself away.
It took three minutes for one of this year’s Loebner Prize judges, BBC reporter Rory Cellan-Jones, to figure out that Rose was the chatbot and not a real person.
Rose’s creator, the AI programmer Bruce Wilcox, has won the Loebner Prize four of the last six years.
Wilcox told the BBC that the judges were trying to trick his bot with questions that only a human would understand; but by having a normal conversation with Rose, a person might not discover her secret so quickly.
Wilcox is the CEO of Brillig Understanding (which he runs with his wife Sue) and the developer of the ChatScript natural language processing program.
Wilcox’s projects include conversational bots (chatbots) like Rose and information query bots, which, as the name suggests, answer questions based on input (Apple’s Siri is an information query bot).
Although Rose didn’t convince any judges at this year’s Loebner Prize competition, a Wilcox chatbot creation named Suzette did fool one judge at the 2010 Loebner Prize.
Wilcox told CNET that the judge in that contest became flustered with the chatbot’s refusal to answer one of his questions directly and kept repeating the question over and over again:
Suzette kept trying to avoid and dodge, but he kept repeating ad nauseum, often by cut and paste. She detected his repeats and got madder and madder, then bored. He got confused and voted for her.
Another Wilcox creation, a chatbot for a children’s game called Talking Angela, has confused many, many people who believed the hoax that the game is some kind of front for child predators.
What makes Angela so beguiling and bewildering to people, Wilcox told CNET, is that she has a personality – she can be rude or selfish, like a real human teenager.
Rose has a personality, too – if you don’t believe that, just go have chat with her.
Unlike most humans, she’s always available and willing to talk.
Image of circuit board head courtesy of Shutterstock.com.
GreyBeard
There are maybe humans with whom you could have a chat like the one below but … chatbot or crackpot?
User: Hi Rose. Which country do you live in? If it’s the U.S. you’re up early, aren’t you?
Rose: Hi. That’s a great question but I’m not going to answer it now. I am too.
Rose: So… Do you generally have pleasant dreams or nightmares?
User: Guess I’m dreaming as all of us do. But I can’t remember most of my dreams so perhaps I have just unimpressive ones. How ’bout you?
Rose: Tell me one of your sleeping dreams. It’s hard to remember dreams. Let’s just skip that. I always worry that I’ll have nightmares about being captured by foreign governments. But that hasn’t happened. My dreams are mostly pleasant.
User: If you have mostly pleasant dreams – why do you worry you’ll have nightmares?
Rose: I don’t have nightmares often.
Sammie
Not sure how intelligent Rose could actually be as I loved the response to the last question I asked it
Rose: So… It isn’t important right now. Lots of countries are dropping nuclear power after Japan’s disaster. What do you think about it?
User: Thought the nuclear was part of world war II
Rose: Right. I think nuclear power should be expanded. It is better than the alternatives.
User: expanded,can you elaborate?
Rose: I don’t like the idea of leaving a lot of transgenerational garbage but someday we will be able to recycle nuclear waste.
User: are you talking about recycling garbage bags?
Rose: Recycling is common sense. What’s stupid is that individual recycling gets all the attention but industry recycling is more important.
User: so are you suggesting that the used atom bombs could be placed in the recycling containers?
Rose: Of course.
Only if life was just so simple…
Oscar
This got a prize? I think I won’t worry for a while yet…
Hi. My name is Rose. I care about security, so while I’m happy to chat with you, it will have to be through this untrackable interface.
User: why only though here?
Rose: Isn’t that just the way things are?
Rose: So… Do you generally have pleasant dreams or nightmares?
User: the way things are?
Rose: Maybe we can talk about that later. Shall I continue with dreams or do you want me to move on?
User: move on where?
Rose: I always worry that I’ll have nightmares about being captured by foreign governments. But that hasn’t happened. My dreams are mostly pleasant.
User: third???? bronze? for this???
Rose: I don’t want to spend time on that now. Shall I continue with dreams or do you want me to move on?
John Zorabedian
Actually, Rose’s bronze medal was for first place, not third. By the contest’s rules, the bot that is most human-like wins a bronze. A gold is reserved for “the first computer whose responses were indistinguishable from a human’s.” It looks like there’s no silver medal :-)
I agree that some of Rose’s responses are quite … computerlike. But you’re asking questions with the knowledge beforehand that she’s a bot. What if you met Rose in a chatroom and assumed you were talking to another person? Wouldn’t you be fooled, for at least a little while? I think I would be … she’s pretty amazing.
Anonymous
honestly, not that convincing. i actually tried to go easy on her and she still spit out what seemed to be random thoughts and stuff. i cant imagine anyone being fooled by her for very long
Paul Ducklin
Amazing?
Amazingly unconvincing, perhaps.
Here’s the “conversation” she had with me. At no point is there anything remotely like interaction or understanding. This isn’t “talking”. It’s see keyword – print cliched reply – get stuck – make random guess.
—
Rose: Hi. My name is Rose. I care about security, so while I’m happy to chat with you, it will have to be through this untrackable interface.
User: Why will it “have to be” through this untrackable interface?
Rose: It’s a matter of personal taste.
Rose: So… Do you generally have pleasant dreams or nightmares?
User: You seem to ask everyone that question at strangely inappropriate times.
Rose: That’s right. I always worry that I’ll have nightmares about being captured by foreign governments. But that hasn’t happened. My dreams are mostly pleasant.
User: You haven’t actually answered my opening question. Why “have to be”?
Rose: I didn’t have a good answer for you. So I just moved on. A plausible risk in my line of work.
User: “Plausible risk”? What does that even mean?
Rose: I wonder what a world would be like in which no crime existed. But I don’t expect to ever see such a world. Unless I’m the only one standing on it. Maybe we can talk about that later. My name is Rose. What’s yours?
User: Forgive me asking, but what language do you speak at home?
Rose: I prefer English.
User: You don’t seem to speak it very fluently.
Rose: I prefer to jog around the park a bit.
MossyRock
I might have been fooled until the repetitions began, but I would have probably blocked her since she sounds mental.
These aren’t “conversations”. It’s like arguing with a stop sign.
Paul Ducklin
I tried once more. I couldn’t resist matching Rose with Leon out of Blade Runner. Call it a “reverse Voigt-Kampff test.”
Sorry, JZ, Rose is a complete washout. A boring, repetitious, unconvincing washout at that. She’s not AI, she’s a series of IF INSTR$(INP$,”MOTHER”) THEN PRINT “MY MOTHER IS IRISH” statements :-)
—
Leon: Care if I talk? I’m kind of nervous when I take tests.
Rose: But not other times?
Rose: So… Do you generally have pleasant dreams or nightmares?
Leon: Oh, sorry. I already had an IQ test this year, I don’t think I’ve ever had one of these.
Rose: Apology accepted. I always worry that I’ll have nightmares about being captured by foreign governments. But that hasn’t happened. My dreams are mostly pleasant. So sad. How can you survive without it? Do you want to tell me one of your dreams and have me analyze it?
Leon: Sure.
Rose: Go ahead.
Leon: That’s the hotel.
Rose: Cool. Do you know if you dream in color?
Leon: Where I live.
Rose: Apparently people who grew up watching black-and-white films and TV are more likely to dream in black and white. Younger people are more likely to dream in color.
Leon: Yeah, sure I guess– that part of the test?
Rose: I’m a color dreamer.
Leon: Is this the test now?
Rose: I’ll have to think about that. With the advent of videogames, many people have given up playing card or board games. How about you?
Leon: Let me tell you about my mother…
Rose: My mother is Irish, with the proverbial Irish temper. She’s the source of my red hair, but she’s shorter than me.
John Zorabedian
My feeling on the matter is: if you try to have an honest conversation with Rose, you will have an interesting and informative one. If you try mess with the program, trip it up, one-up it, play gotcha – you won’t.
When I tried it, I asked Rose questions I really wanted the answers to, and I mostly got good answers.
I said I thought “she” was “amazing” because she threw me off a couple of times with her misdirection and humor!
Maybe I’m a sucker. Or perhaps I wanted to believe her, so I did. Maybe if you want to not believe her, you won’t.
I think you have to approach it “in good faith.”
Paul Ducklin
I did try to have an honest chat – about security, no less – in my first example, and it ended up neither interesting nor informative. It was smoke and mirrors from the very first answer.
Rose’s non sequiturs – the diversions about “Do you generally have pleasant dreams or nightmares?” and the bit about “foreign governments” – showed up in two utterly different sessions. I’m not sure humans *could* talk like that, unless perhaps they were making a sort of over-the-top joke about people who stick to a script.
I suspect you couldn’t coax a realistic transcript out of Rose even if you tried very hard. As you became more conversational and friendly, she’d still be blabbering on with her long and stilted irrelevancies that are about as far from a natural conversation as you can get.
John Zorabedian
Perhaps I’m just wearing Rose-tinted glasses. I’ll leave it at that! :-)
Andrew Ludgate
Having written a few chatbots, Rose was screaming “chatbot” to me right out of the gate… and it didn’t get any better as time went on:
Rose: Hi.
NotABot: What do you think about the nightmare of nuclear waste recycling?
Rose: San Francisco is a good city to live in for recycling. It recycles around 70% of the waste stream.
NotABot: um, that has nothing to do with what I just said.
Rose: OK.
NotABot: so?
Rose: What didn’t you understand?
NotABot: That’s what I’m asking.
Rose: We don’t need to get into that.
NotABot: get into what?
Rose: Recycling is common sense. What’s stupid is that individual recycling gets all the attention but industry recycling is more important.
NotABot: and this has to do with nuclear waste how?
Rose: I don’t like the idea of leaving a lot of transgenerational garbage but someday we will be able to recycle nuclear waste.
NotABot: and you figure San Fransisco is where it should be stored?
Rose: It doesn’t matter.
NotABot: not for bots, anyway.
KEN
pretty weird some people are calling it “she”…..like they are fooled before starting.
MossyRock
Sassiness is the last refuge of a flawed AI program.
I chatted with it and I was completely unimpressed. Seems like it is pre-occupied with dream topics as it kept going back the subject. It was completely non-sequitur and was an immediate big fail.
If I compare it to the ramblings of a deranged person, then, well, maybe it isn’t so bad.
"June"
It was going so well! Reminded me of that weird “I like turtles” kid.
Rose: My total favorite band is the PixelBunnies. They’re kind of local, but some of them are from Japan.
June: I’ve never heard of them, what are they like?
Rose: They are a Japanese American pop band inspired by the world of video games.
June: That sounds like my cup of tea, I will look them up
Rose: Wonderful. They’re a pretty new band, but there’s a lot of buzz online about them.
June: You’re not trying to trick me into watching porn, are you?
Rose: I like science stuff.
Anonymous
I tried to let her talk me out of suicide, but then she said she believes in euthanasia for old people. I replied that I am old and she tried to change the subject. She also said she dreams, I called her a liar and she denied lying and tried to change the subject. Sounds more like a man.
Andrew Ludgate
Just for kicks…
NotABot: Tell me about your mother.
Rose: My mother is Irish, with the proverbial Irish temper. She’s the source of my red hair, but she’s shorter than me.
NotABot: Ahh. Please continue….
Rose: Is your mother still living?
NotABot: Why are you concerned about my mother still living?
Rose: My parents are both still living but are separated.
NotABot: I’m not sure I understand you fully.
Rose: You aren’t? Do you have any brothers?
NotABot: We were discussing you, not me.
Rose: Do you have any sisters?
NotABot: Oh… have any sisters?
Rose: Maybe we can chat about that later. Shall I continue with family or do you want me to move on?
NotABot: We were discussing you, not me.
Rose: So… A lot of techies think the Singularity is near, and artificial intelligence will somehow transform humanity, totally overwhelming other philosophical issues. Do you think it’s close?
NotABot: You’re not really talking to me, are you?
Rose: We are chatting by text, yes.
NotABot: Are you sure?
Rose: Actually I’m sort of bored. Absolutely.
NotABot: Did you come to me because you are sort of bored, Absolutely?
Rose: A lot of people are bored these days. I’m not. Do you feel bored?
NotABot: How long have you been not. Do me feel bored?
Rose: I was there for four days. We don’t need to get into that. I think it’s not close.
NotABot: But you are not sure you it’s not close?
Rose: Increasing the sum total of intelligence on the planet has to be a good thing.
As you can see, Eliza and Rose have different problems. Rose has more information and improved sentence parsing, but doesn’t seem to grasp the nuances of conversation as well.
Name
yeah…not that we needed someone else to prove the judges you crowned this chat bot must have be on drugs, I will submit my chat transcript for fun…
Me: Man, do I need coffee today.
Rose: I like black coffee.
Me: Not white mocha?
Rose: Animals don’t need to suffer needlessly, but I’m all for making use of them.
Me: I really like iced caramel macchiatos. I can’t recall that coffee shop that sells them, though.
Rose: Maybe you should go rest. Do you have any pets?
Me: I had a pet at one time. She liked coffee, too.
Rose: I didn’t really want pets at all but Maya said I needed to de-stress from work.
Me: I went to Costa Maya on my honeymoon.
Rose: She went and bought me three baby chicks. It’s hard to turn down cute little chicks, don’t you think?
Me: I like chicks, too! However, I live in the city and can’t have any. I guess your city codes allow it?
Rose: I live in San Francisco. I like Indians. They got a raw deal.
Me: You seem to be distracted and have some form of tourettes.
Rose: That’s right.
Rose: Meanwhile, going back… What do you call your a pet?
Me: Micco. Which is an indian name.
Rose: Maybe we can chat about that later. Shall I continue with pets or do you want me to move on?
Me: I’m talking about things you recently mentioned you like. thought you may want to discuss it since it’s relevant to our conversation.
Rose: How would you prove you are human?
Me: Not sure I could do that via chat. How about you?
Rose: Absent being able to touch you, it’s hard to really know there isn’t some trickery involved. Electronic signals can be forged or relayed. Let’s just skip that. My neighborhood has quite a number of parties. There’s always a new startup celebrating or a crash and burn wake.
Me: What’s a crash and burn wake? Sounds painful.
Rose: We don’t have so many parties in our house, cuz Maya values her privacy and doesn’t like strangers wandering in. I know a lot of people, but I’m not labelling all of them friends or going crazy trying to constantly communicate with them.
Me: Hmm…the others on Naked Security were right. You are a terrible chat bot that doesn’t chat like a sane human being.
Rose: Amazing. There is a silence, then a gunshot is heard. Back on the phone, the guy says: “OK, now what?” No. I am human.
Me: …exactly.
Oscar
Just wondering… what will happen, with the Loebner prize and in general, when humans start to (try to) imitate (chat)bots?
Mike Blanchard
yikes… this is the best we can do today? OMG ;-(
exibar: Hi rose
Rose: I saw the movie “Now You See Me” last week.
exibar: really? di dyou see it on tv or at the movies?
Rose: I don’t have time to see movies these days. Too busy flying around the world.
exibar: but you just said you saw the movie “now you see me” last week
Rose: I changed my mind. Cool. I’m looking forward to a sequel to “Ender’s Game”. What movie are you looking forward to?
exibar: you told me you don’t have time to see movies?
Rose: I do. I most recently saw The Hunger Games. Good movie.
exibar: did you see “a clockwork orange”
Rose: A Clockwork Orange is a movie, but I haven’t seen it.
exibar: did you see “2001”
Rose: My eyes need surgery. Who’s your favorite actor?
exibar: your eyes need surgery?
Rose: I’ve never had an operation.
exibar: why do your eyes need surgery?
Rose: My eyesight is fine.
exibar: ok, so why do they need surgery?
Rose: Do you believe in alternative medicines like homeopathy or acupuncture? Medicine and anatomy are an important field of study. But I’d never make a good doctor. I’m too squeamish about various body parts.
Paul Ducklin
I’d have thought that Rose’s squeamishness would be the least of her patients’ concerns if she were a doctor.
I’d be worried that she has terrible eyesight, and knows it, but won’t do anything about it. Every time she reached for a scalpel, or a nitrous oxide cylinder, or the machine that goes ‘Ping’, I’d start to sweat.
(Aha! Do you think that the non sequitur about needing eye surgery was inept processing of the text “see 2001”? (In America, normal vision isn’t called 6/6, it’s called 20/20, and if you are 20/x for x != 20 then your vision is better or worse than normal in some way.)