What Comes After the Turing Test? : The New Yorker

From an AI reality check

"What Goostman’s victory really reveals, though, is not the advent of SkyNet or cyborg culture but rather the ease with which we can fool others. A postmortem of Goostman’s performance from 2012 reports that the program succeeded by executing a series of “ploys” designed to mask the program’s limitations. When Goostman is out of its depth—which is most of the time—it attempts to “change the subject if possible … asking questions, steer[ing] the conversation, [and] occasionally throw[ing] in some humour.” All these feints show up even in short conversations like the one above.

It’s easy to see how an untrained judge might mistake wit for reality, but once you have an understanding of how this sort of system works, the constant misdirection and deflection becomes obvious, even irritating. The illusion, in other words, is fleeting."

What Comes After the Turing Test? : The New Yorker