Additional Resources

Bibliographic Material

On Intelligence contains an annotated bibliography. It was hard to know what to put in that bibliography because there are few works that relate to the theory presented in the book. The sources that do relate to the theory are mostly technical. We listed items here that didn't make it into the book. As we come across more items of relevance we will post them here. If you have suggestions please let us know.

Hawkins, Jeff. "An Investigation of Adaptive Behavior Towards a Theory of Neocortical Function." ©July, 1986
This is a paper I wrote while a graduate student at U.C. Berkeley in 1986. I wrote it as an independent research project and it was my way of explaining what I wanted to study for my Ph.D. thesis. I understood the importance of prediction at the time, but very little of the specific mechanisms used in the brain to store memories and make predictions. It contains a few ideas not in the book but mostly it might be interesting to see where my understanding was in 1986. At that time, the university said I wouldn't be able to pursue this research since there were no professors doing similar work. That was one of the reasons I went back to industry. U.C. Berkeley today is a very different institution. They have an active interest in theoretical neuroscience and graduate students at Berkeley can visit and do research at RNI. http://neuroscience.berkeley.edu/ ·Requires Acrobat Reader.

Kohonen, T. "Self-Organization and Associative Memory" (2nd edition), Springer-Verlag, ©1984
This book by Kohonen is sort of a classic in the field of auto-associative memories and is where I first learned about them. It has a brief introduction to using associative memories to store sequences based on the work of Willwacher.

Willwacher, Gerd. "Storage of a Temporal Pattern Sequence in a Network." ©July, 1982.
This early work demonstrates associative sequence memory in simulation experiments Willwacher proposed biologically plausible synaptic learning rules. Unfortunately this paper is a translation from German and is hard to read. You can get the gist of it in Kohonen's book. ·Requires Acrobat Reader.

Simulations and Source Code

Hierarchical sequence memories and other concepts in the book can be simulated and modeled on traditional computers and eventually built in silicon. One of the primary goals in writing the book was to encourage students, academics, and entrepreneurs to take up this challenge. As we become aware of relevant work we will post links to them here.

Dileep George, http://www.stanford.edu/~dil/invariance/ - Dileep George is a graduate student at Stanford University doing his PhD research at The Redwood Neuroscience Institute. He is modeling hierarchical memories based on ideas in On Intelligence and extending those ideas with probabilistic formulations. Dileep is solving invariance problems in a visual recognition task using hierarchical representations, including the backward propagation of predictions. He has a working prototype, which impressively demonstrates the principles. The link above takes you his web site where he describes his work and where he has generously agreed to provide source code.