Image may be NSFW.
Clik here to view.
There is ancient wisdom in the phrase never believe your own press that computer scientists have laid aside in their quest to discover “artificial” “intelligence”.
In the Axios article “Artificial intelligence pioneer says we need to start over” Steve LeVine writes:
In 1986, Geoffrey Hinton co-authored a paper that, four decades later, is central to the explosion of artificial intelligence. But Hinton says his breakthrough method should be dispensed with, and a new path to AI found…
Speaking with Axios on the sidelines of an AI conference in Toronto on Wednesday, Hinton, a professor emeritus at the University of Toronto and a Google researcher, said he is now “deeply suspicious” of back-propagation, the workhorse method that underlies most of the advances we are seeing in the AI field today, including the capacity to sort through photos and talk to Siri. “My view is throw it all away and start again,” he said…
In back propagation, labels or “weights” are used to represent a photo or voice within a brain-like neural layer. The weights are then adjusted and readjusted, layer by layer, until the network can perform an intelligent function with the fewest possible errors.
But Hinton suggested that, to get to where neural networks are able to become intelligent on their own, what is known as “unsupervised learning,” “I suspect that means getting rid of back-propagation.”
If you have not read the series on how an abacus cannot be a brain, nor considered “artificial intelligence”, please do so first (Part I, Part II, Unsupervised learning digression).
Back propagation is only a technique, meaning there are others, to create weights in an “artificial neural network”. Not for the first time do I praise, with genuine enthusiasm, the marketers of computer science for creating wonderful names.
Here is the world’s simplest ANN:
y –> w*y –> z
Some value of y is input into the “network”, and it is then “hit” by a weight, to produce the outcome z. So that if y = 7 and the weight is 2, then (brace yourselves!) z = 14.
It does not matter where the weight w came from, whether from back propagation or from the Lord Himself. It is the weight.
Now this is an ANN. The only thing that separates it from the over-hyped versions marketed in “deep learning” and similar-sounding programs is the complexity, by which is meant the number of possible inputs, layers (the “w*y” is a “layer”), and outputs. Some ANNs can be a tangled mess, with lines connecting layers here and there and everywhere, with weights aplenty. But none differs in any essential sense from our simple network.
In short, an ANN is just like our wooden abacus. It is not alive. It is completely dumb. It is a machine which takes inputs, applies definite operations to them, and produces an output. Your sewing machine and typewriter do the same. And so does an abacus. This is not intelligence, though these are all artefacts.
The proof is complete, but it is doubtful it will be convincing to those who have for too long believed their own press. So let’s press the example.
Suppose we add more complexity to our ANN, as in the picture above. The topmost “hidden” node takes inputs from three input nodes, and produces, after weighting these three inputs, two inputs to output nodes, which in those output nodes are hit with other weights to produce z_1 (the topmost output node). The weights are not drawn, but they are there, as described.
Well, this is simply a mechanical process, once the weights are specified. Barring malfunction, it is entirely deterministic. It is a dry process. There is no mystery to it. Adding layers and complexity just makes it bigger and more expensive to run. It will never make it alive, or intelligent. A pipe organ is not more alive than a flute because it has more gears and levers.
It does not matter where the weights come from, via back propagation or something else. A weight is a weight. Changing from w = 2 to w = π does not make our simple ANN alive or intelligent because the second weight is more complex. Playing only the black keys on piano does not make the tune closer to an intelligence than playing only Jingle Bells.
Since the origin of the weights do not matter, it does not matter if they are recreated on some regular basis, perhaps as a function of how far the output nodes are from some eventual reality. That is, making the mechanism “dynamic” does not make it alive, or intelligent. If fact, as was explained in the abacus example, it makes no difference whatsoever. It is just makes it more complex.
Too, speeding up the calculations only makes the thing run faster; speed is not intelligence. And there is nothing that will “emerge” from the structure as complexity grows—not supported by any physical process, that is. (The topic of “emergence” needs its own article.)
There is no hope of creating intelligence from artificial neural networks, or anything that works in a similar fashion to them.
I have more on this topic in Uncertainty: The Soul of Modeling, Probability & Statistics.
I learned of the Axios article from Christos Argyropoulos.