On On Intelligence

I recently finished Jeff Hawkins' (creator of the PalmPilot) book On Intelligence. The book is intended to expose the mechanism of intelligence through the wise and oft ignored neurophysiology of the neocortex. Most intelligence research has ignored the structure of the cortex, an odd move considering our perception of such as the seat of the remarkable intelligent behavior in humans. Similarly unsatisfactory, the majority of neurophysiology (the study of the cellular structure of the neural system) has ignored intelligence, focusing mostly on mapping connections and occasionally trying to describe the mechanism for reflex and instinctual behavior.

I myself had intended to research the anatomy of the brain in hopes to gain some insight to the workings of intelligence. It is a great fortune, having eschewed neuroscience in favor of the advanced mathematics of machine learning, to be the beneficiary of Hawkings' 25 years of neurophysiological study. In the book he outlines, to some but not all detail, the so called Memory-Prediction Framework. The quick essential components are two:

1st, the structure of information in our world is hierarchical by nature, in a way that can be represented as a tree data structure. For instance, if we were to talk about music, we could talk about many components that make up music. At the bottom of the hierarchy we have tones that represent notes, which is the precise thing that enters our ears. As we move up the chain, we get chords, phrases, songs, albums and so forth. Every other sense can be described in the same way -- at the bottom you have raw data, and as the data moves up the chain it becomes more and more abstract. Hawkins posits that the structure of the brain mirrors this hierarchy, an argument he makes based on the structure of the brain itself.

2nd, the brain operates in all areas with a single common algorithm. An algorithm is like a list of instructions required to complete a task. This is an interesting and immediately reasonable position. The brain is composed of somewhere between 100-200 billion specialized brain cells (neurons), each with around 1000 points of connection. This gives about 50 trillion connections, an astronomical number. But that is nothing compared to the number of possible ways to connect those, which is (((1*10^14)-1)! / ((5*10^13)!*((5*10^13)-1)!). If you don't know your combinatorics, that number is insanely large, so large that calculating it would require a lot of very special computer code. The estimated number of atoms in the universe is around 1*10^83, 1 with 83 zeroes. 150! is about 5.7*10^262. In order to make this calculation we'd need to take (1*10^14)!. It's unfathomable, trust me. The point is, there is no possible way that DNA can contain information for specific connections. The result of that is that yes, the brain must have a common algorithm, one that works the same no matter how the connections come in or from where. It is certainly possible that some connections are made in different ways than others through the use of different signalling chemicals, but the number of different ways is limited and even still a connection might come from this neuron or that, there is no way to specify otherwise.

The result of these foundational aspects is a neural network, which have been around for a long time. Ironic, as Hawkins' doesn't have much good to say for neural networks in the start of the book. This might be for a lack of study in the field, as he mentions (paraphrasing) "No one really knows why neural networks became popular again in the 90's" which isn't true. I don't know an awful lot about neural networks, but I know that there is a consensus that they became popular again because a way to handle XOR was introduced by one of the authors of the paper years prior that showed they couldn't handle XOR (either Minsky or Pappert, I can't remember which). Though it is a neural network, it's a very special kind of neural network.

All told, I suggest anyone interested in intelligence as a mechanism read this book. The ideas might turn out to be wrong, but in the mean time they are interesting and apparently apt; there have been many times since finishing that I have seen in people behaviors that fit cleanly into the model.

There is always so much to be said, never enough time with which to say. So until time comes to afford saying again...