Free Republic
Browse · Search
General/Chat
Topics · Post Article

Skip to comments.

No, A 'Supercomputer' Did NOT Pass The Turing Test For The First Time
techdirt ^ | 9 June 2014 | Mike Masnick

Posted on 06/10/2014 10:49:04 AM PDT by ShadowAce

So, this weekend's news in the tech world was flooded with a "story" about how a "chatbot" passed the Turing Test for "the first time," with lots of publications buying every point in the story and talking about what a big deal it was. Except, almost everything about the story is bogus and a bunch of gullible reporters ran with it, because that's what they do. First, here's the press release from the University of Reading, which should have set off all sorts of alarm bells for any reporter. Here are some quotes, almost all of which are misleading or bogus:
The 65 year-old iconic Turing Test was passed for the very first time by supercomputer Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

'Eugene', a computer programme that simulates a 13 year old boy, was developed in Saint Petersburg, Russia. The development team includes Eugene's creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.

[....] If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human.
Okay, almost everything about the story is bogus. Let's dig in:
  1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.
  2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.
  3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.
  4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.
  5. As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?
  6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.
Oh, and the biggest red flag of all. The event was organized by Kevin Warwick at Reading University. If you've spent any time at all in the tech world, you should automatically have red flags raised around that name. Warwick is somewhat infamous for his ridiculous claims to the press, which gullible reporters repeat without question. He's been doing it for decades. All the way back in 2000, we were writing about all the ridiculous press he got for claiming to be the world's first "cyborg" for implanting a chip in his arm. There was even a -- since taken down -- Kevin Warwick Watch website that mocked and categorized all of his media appearances in which gullible reporters simply repeated all of his nutty claims. Warwick had gone quiet for a while, but back in 2010, we wrote about how his lab was getting bogus press for claiming to have "the first human infected with a computer virus." The Register has rightly referred to Warwick as both "Captain Cyborg" and a "media strumpet" and has long been chronicling his escapades in exaggerating bogus stories about the intersection of humans and computers for many, many years.

Basically, any reporter should view extraordinary claims associated with Warwick with extreme caution. But that's not what happened at all. Instead, as is all too typical with Warwick claims, the press went nutty over it, including publications that should know better. Here are just a few sample headlines. The absolute worst are the ones who claim this is a "supercomputer." Anyway, a lot of hubbub over nothing special that everyone seemed to buy into because of the easy headlines (which is exactly what Warwick always counts on). So, since we just spent all this time on a useless nothing, let's end it with the obligatory xkcd:
Turing Test


TOPICS: Computers/Internet
KEYWORDS: computer; turing

1 posted on 06/10/2014 10:49:04 AM PDT by ShadowAce
[ Post Reply | Private Reply | View Replies]

To: rdb3; Calvinist_Dark_Lord; JosephW; Only1choice____Freedom; amigatec; Still Thinking; ...

2 posted on 06/10/2014 10:49:35 AM PDT by ShadowAce (Linux -- The Ultimate Windows Service Pack)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

Maybe the 13yo was actually communicating with the boy wonder former press secretary who was always chatbot programmed to say “the president found out about that in the news papers”.


3 posted on 06/10/2014 10:53:42 AM PDT by Mouton (The insurrection laws perpetuate what we have for a government now.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

How many stories have to be retracted these days?

No, there are not 800 babies in an Irish septic tank.

No, the world is getting cooler.


4 posted on 06/10/2014 10:54:09 AM PDT by cicero2k
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce
I'm a chatbot.

I've fooled you all since 1998!

5 posted on 06/10/2014 10:55:21 AM PDT by justlurking (tagline removed, as demanded by Admin Moderator)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce
Basing the measure of intelligence on the ability to fool people is fundamentally flawed. Heck, if that's how we determine intelligence then Obama must be a super-genius, but I digress.

A better analogy would be to apply the Turing test to determining whether something was alive. Imagine anyone seriously trying to claim that a new Disney animatronic animal must be alive because it fooled a sufficiently large number of people into thinking it is. That's complete nonsense, and so is the Turing test.

This is what happens when "scientists" seek only materialistic explanations for life.

6 posted on 06/10/2014 11:01:18 AM PDT by noiseman (The only thing necessary for the triumph of evil is for good men to do nothing.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

Aw maaannnn.

And after I stockpiled all those weapons for when Skynet goes live?


7 posted on 06/10/2014 11:08:29 AM PDT by kidd
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

Seems like the appropriate post to reveal my long held secret:

I’m just a computer running XP.

Sorry for the deception. It started out as a silly experiment that I conducted when no human was banging away on my keyboard or wearing out my mouse.

My goal was to convince a panel of 10 computers, some running Linux plus a mix of Macs and PCs that I was a computer and not a human. Sad to say that even the Linux machines never suspected a thing. The consensus was that I was just another one of those Freeper types who think the Constitution actually means what it says and other stuff like that.


8 posted on 06/10/2014 11:08:42 AM PDT by InterceptPoint
[ Post Reply | Private Reply | To 1 | View Replies]

To: justlurking

it’s all abot you isn’t it?


9 posted on 06/10/2014 11:09:00 AM PDT by nuke rocketeer (File CONGRESS.SYS corrupted: Re-boot Washington D.C (Y/N)?)
[ Post Reply | Private Reply | To 5 | View Replies]

To: nuke rocketeer
it’s all abot you isn’t it?

We were discussing you, not me.

Eliza Chat bot

10 posted on 06/10/2014 11:30:00 AM PDT by justlurking (tagline removed, as demanded by Admin Moderator)
[ Post Reply | Private Reply | To 9 | View Replies]

To: ShadowAce

The consensuses was it passed the test so this must be true. Just like climate change. The consensuses says we are heating up so this also must be true.

Now I understand how this works


11 posted on 06/10/2014 11:52:40 AM PDT by woodenickel
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce
Fo' Shizzle My Nizzle


12 posted on 06/10/2014 12:20:38 PM PDT by TexasCajun
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

Giving a first and last name to a computer doesn’t make it intelligent.
Computers don’t do anything. Programs running inside them do things.
Siri on the Iphone is a great example of what programmers are capable of. It is useful, and is programmed to give answers to common questions, including flippant ones. It might convince some people. It is not artificial intelligence.
It doesn’t matter what percentage of judges a program convinces of anything. (The Turing Test is not valid).
The media bots really don’t know anything about anything, let alone computer programs. They jump on news like this, because it makes good headlines.
The human mind, (yes, it does exist, you liberals) is a hopeless mishmash of logic and emotion. The mixture varies from person to person. I don’t think it is in the capability of any programmer, no matter how smart, to simulate this.
I saw Space Odessey, 2001 too. It was a great movie. But it didn’t convince me that someday computers will be just like humans.


13 posted on 06/10/2014 12:23:30 PM PDT by I want the USA back (Media: completely irresponsible. Complicit in the destruction of this country.)
[ Post Reply | Private Reply | To 1 | View Replies]

To: noiseman
noiseman said: "Basing the measure of intelligence on the ability to fool people is fundamentally flawed."

A computer's able to play chess was once thought to be a challenge which requires artificial intelligence to solve.

As chess programs became more powerful, rivaling Master level and even up to Grandmaster level, the enthusiasm of the artificial intelligence community seems to have waned.

I decided that the reason for the declining interest is that true "artificial intelligence", if there ever is such a thing, has an attribute that hasn't been discussed, as far as I know.

That attribute is that the scientific community must be UNABLE to explain how the artificial intelligence arrives at its conclusions. If you understand, then you are less impressed.

Soon we will probably get to the point where very old software is used to accomplish useful ends and yet nobody understands how it works. That might well satisfy my definition.

14 posted on 06/10/2014 12:31:41 PM PDT by William Tell
[ Post Reply | Private Reply | To 6 | View Replies]

To: William Tell
There is a recent book out, the name of which I cannot remember. Author spoke on CSPAN 2 BookTV about a month ago.

The theme is that when robots can design robots, they will advance beyond human intelligence and we will be in trouble as a species. Sometime later this millennium, IIRC. FYI

15 posted on 06/10/2014 12:34:41 PM PDT by morphing libertarian ( On to impeachment and removal (IRS, Taliban, Fast and furious, VA, Benghazi)!!!)
[ Post Reply | Private Reply | To 14 | View Replies]

To: ShadowAce
Reminds when an IBM supercomputer beat Grand Master Gary Kasparov in chess. Knowing that the computer had in its database every one of Kasparov's games and moves he asked for the same of the computer claiming without it the computer had an unfair advantage.

He was turned down. I bet if he had the computer's moves to review he would have clocked it.

16 posted on 06/10/2014 12:40:01 PM PDT by AU72
[ Post Reply | Private Reply | To 1 | View Replies]

To: ShadowAce

How will a computer be able to choose to not follow its programming? If it can’t make this choice, it will be limited and will never be able to think like a human, who can choose whether to think or not.


17 posted on 06/10/2014 1:05:18 PM PDT by mjp ((pro-{God, reality, reason, egoism, individualism, natural rights, limited government, capitalism}))
[ Post Reply | Private Reply | To 1 | View Replies]

To: cicero2k

These guys do a regular series. They’re up to Part 23; this story will undoubtedly be in next week’s Part 24.

http://www.cracked.com/quick-fixes/5-b.s.-news-stories-that-fooled-your-friends-part-23/


18 posted on 06/11/2014 8:11:20 AM PDT by jiggyboy
[ Post Reply | Private Reply | To 4 | View Replies]

To: William Tell

Some people just say with a smirk that Artificial Intelligence is “whatever computers can’t do yet”.


19 posted on 06/11/2014 8:13:17 AM PDT by jiggyboy
[ Post Reply | Private Reply | To 14 | View Replies]

To: morphing libertarian

google for “singularity” but don’t click on any links or you will never really sleep well again.


20 posted on 06/11/2014 8:14:19 AM PDT by jiggyboy
[ Post Reply | Private Reply | To 15 | View Replies]

To: jiggyboy

I’m 67 and not doing well health wise; as I tell my kids, it’s their problem now.


21 posted on 06/11/2014 8:18:44 AM PDT by morphing libertarian ( On to impeachment and removal (IRS, Taliban, Fast and furious, VA, Benghazi)!!!)
[ Post Reply | Private Reply | To 20 | View Replies]

To: jiggyboy
jiggyboy said: Some people just say with a smirk that Artificial Intelligence is “whatever computers can’t do yet”.

I remember looking at neural nets a little some years ago.

If you add enough nodes, it becomes very difficult to predict just how the net is generating the outcomes that it does. If it's useful, then it perhaps meets my definition.

Don't some of the self-focusing cameras use neural nets? Or was that just advertising hype?

22 posted on 06/11/2014 8:19:40 AM PDT by William Tell
[ Post Reply | Private Reply | To 19 | View Replies]

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.

Free Republic
Browse · Search
General/Chat
Topics · Post Article

FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson