The New Testament of AI researchers. The Old Testament was the whole Knowledge Representation, Expert Systems, and Fifth Generation debacle. Those utterly failed to produce anything remotely like intelligence, and demonstrated how insufficient structured knowledge representation is for representing knowledge about anything like the real world.
So now we're told that the real secret is to disorganise knowledge. No longer will AI try to impose structure on to knowledge. In fact, we're now supposed deliberately to consider systems where there is no visible location of the knowledge (so much for the great Hofstadter-Dennettesque ideas of isolating parts of other people's minds and plugging them into ours).
Here's what we're told we should do. We should take some large system of simple elements and interconnect (always interconnect; plain "connect" sounds weak) them in various ways we don't understand. This is a good place to mumble something about Neural Nets or Genetic Algorithms (we don't really understand those, either).
Now, the logic goes, we have achieved a simulation: we have one "system" (the mind) which we don't understand, and we've managed to construct another system that we don't understand. Surely they'll share features, and we will have achieved a simulation of consciousness!
The skeptic will ask why e should think so. Emergent behavior is the trump card here. "You can understand how each unit separately works," we're told to tell our skeptic, "but large enough ensembles of these units will exhibit emergent behavior which you cannot predict!"
Gee, that's great. And apparently not being able to predict the behaviour doesn't exclude claiming that it can (and will) be a simulation of thought. It could be that all this is true. But this is a claim requiring proof, not some article of faith.
I'll leave it to others (supporters, presumably) to node specific examples of emergent behavior which is more like thought than it is like the phase transition of ferromagnetic iron from its "natural" state to its magnetised state. I'll just give a sample of why this idea is not enough (on its own) to convince.
Materials are composed of atoms, and atoms are composed of neutrons, protons and electrons. The behavior of atoms and of materials is definitely a good example of "emergent behavior". So it would be natural to expect that the behavior of materials would follow from that of atoms, which would follow from that of elementary particles.
It doesn't. Theoretical calculations of strengths of various crystals don't work out: the values are far too high (this is because of flaws in the crystal; but we've no idea how to model flaws!). Glasses are even worse. And those are the simple forms of material behavior (I exclude gases, because their "emergent behavior" is immensely boring; it gets interesting during the transition to the liquid phase, which we also don't understand).
"BUT!" I hear you cry, "atomic behavior is given by behavior of elementary particles, which do behave by the known laws of Quantum Mechanics!"
True. And it is also true that the only atom to have been modeled from its constituents is hydrogen. Even something as simple as H2 (hydrogen molecule) or He (atom of Helium) cannot yet be modeled. We cannot model a glass of water (or even just its contents).
How do we know our systems which exhibit emergent behavior will "think" rather than behave like a glass of water? Or do we think there's no difference between the two?