xltd.com weblog

This post is inspired by the book On Intelligence by Jeff Hawkins (2004). I will introduce some of his key points found in the book, then comment and expand upon them. Hawkins describes four basic differences between computer memory and human (neocortex) memory:

Storing Sequences
Think of story telling. The story unfolds in a chronological order. Thus human memory is implicitly tied to time. Virtually every thought, every action includes a temporal aspect. Envision you home, driving to work, or dining at a restaurant. Everything is a sequence. Typically we progress though memories as if replaying them in chronological order, but not necessarily. For example, spelling your name backwards or playing a song on a musical instrument backwards. The changing of the order from a conditioned chronological pattern is not natural. It is jarring in nature an requires higher levels of attention. It is the juxtaposition of this story telling that makes for creative an more memorable story telling (if done well) such as the movies Pulp Fiction, Slaughterhouse Five, or Memento. Regardless of the order, the flow of memories still represent a sequence since we can't consider the whole at once. Human memory is inherently sequential.

One additional curious observation of human memory is that when trying to recall dreams, it is best to hold onto the last recollections before waking. Revealing the rest of the dream is discovered by always asking "what happened just before that?" By keeping the eyes closed and slowly rolling back into the positions during sleep, dream memory recall is enhanced. Is there some link between dream memories and the kinesthetic sixth sense of body position? Does our faculty of knowing the position, orientation, etc of our body parts, hold special significance for dream memories? I can't help but see an interesting connection between this positional properception (capturing the world and the internal needs) and perception (capturing the world and the external stimulus). Internal senses play a role is dream memories while external senses play a role in conscious memory.

Auto-associative Recall
This refers to the ability to recall complete patterns given only partial or inexact input. The human brain performs mental shorthand and fills in the details. Any sight, smell, taste, or other sensory input can trigger an associative recall. For example, smells are known to be very powerful associate stimuli. Today while cycling though an industrial area a whiff brought forth a series of memories of a high school factory summer job making printed circuit boards. This set of memories was just one in a stream of consciousness during the bike ride. I encounter associative leaps tying sensory feelings and fragmentary thoughts. Conversations also provide a rich stream of associated memories. In essence this associative string of memories is thought, or at least one form of thought.

This leads me to the connection between thought and consciousness. One part of the brain is believed to be the center of arousal and motivation. This so called the reticular activating system is also involved in stereotypical actions, such as walking, sleeping, and lying down. Being one of the oldest parts of the brain, its activity is crucial for maintaining the state of consciousness. Is thought consciousness? Can there be one without the other. Is there a difference between though and intelligent thought? Is there a distinction between thought and self-awareness?

Invariant Representations
Unlike a lossless copy of a computer file, human memory is "lossy". The focus of human memory is on relationships and not the details. Babies recognition of their parents is one example of this invariance. This can involve sight and sound. Reaching into a toy chest to extricate a desired object is another. This primarily involves touch. Playing catch with a baseball is yet another example, where motor skill memories play the most significant role. Recognition of a song is a great example. Here the key and tempo can change and the piece can be performed poorly or out of tune and we still can recognize the melody since the invariant here the intervals between note pitch and duration.

I believe that the author missed the point quite a bit with some of his examples, especially regarding putting a key in an ignition by having to deal with seat position and other variables. The example completely ignores any feedback system. In particular visual feedback would be an obvious ingredient. Automated space vehicle docking is a close analogy. The FIRST Vex Challenge robot competition (well, at least the autonomous period portion) demonstrates comparable types of autonomous motor skill capabilities.

Futher, I speculate that although reproduction of information is not the primary retrieval means of the human mind, there are alternative means to access such detail. Hypnosis and Dianetics are two such techniques. In Dianetics, the human mind is described as a collection of "mental image pictures," which contain the recorded memory of a past moment, including all sensory perceptions and feelings involved. Although debunked as pseudo-science or simply a form of hypnotism, Dianetics does offer some intriguing possibilities. That is, as long as you separate the pseudo-religion from this pseudo-science technique. Nevertheless, direct recall is one area where cyber-assistance would definitely be of value. Google is already trying to do this with its user history of internet activity and desktop search capabilities. After all, the Google search is really the 21st century version of the command prompt.

Patterns in a Hierarchy
Integrating the senses, information flows up the hierarchy as observed patterns and predictions flow down to create a unified sensory experience. At the low end of this hierarchy, patterns change at a faster rate. Conversely, more constant patterns occur at the top. Bottom up classifications and top down sequences are constantly interacting and changing through experiences. This is the essence of learning.

Using speech as an example, phonemes are recognized at the low end and progress upwards in the hierarchy to form syllables, then words, and so on up the hierarchy to recognize and anticipate phrases, sentences and ultimately ideas. One measure of intelligence is the capacity to recall and predict patterns. As the brain receives sensory input, patterns are recognized and predictions are made by combining recall of past patterns matched to the current situation. Speech recognition is a great example, since it has been one of the most difficult tasks of AI (vision and "smart cars" are two other examples cited by Hawkins).

A hierarchy of nested sequences allows the sharing and reuse of lower level objects in a remarkably efficient manner, but very different from how computers work. Another interesting observation is that through repeated training, representations can move down the hierarchy. Sequences of sequences form, each having a repertoire of sequences it knows, analogous to a repertoire of songs.

This structure of patterns in a hierarchy integrates the other three basic differences between human brains and computers described here. Coping with your environment, you need tomake predictions of future events, even those a micro-second in the future. The neocortex stores sequences of patterns in an invariant form that are retrieved via auto-associative recall so that the knowledge of past events can be applied to new situations that are similar but not identical to the past.

Commentary
Here are some random notes from the text that I found particularly interesting.

• Visual comprehension involves a process of rapid eye movement (saccade) and stops (fixation). In spite of this jerkiness, we experience vision as continuous and stable, Similarly we smooth out the vision from both eyes to form a single image.
• Similar to speech comprehension progression from phonemes through ideas, music progresses from notes to form intervals, then melodic phrases, melodies, and so on to songs or movement and upward to symphonies or Broadway musical scores. I purposely exclude the progression from songs to albums, since I think this is an artificial association that has rarely had much significance beyond some theme albums that mostly occurred in the 60's such as Sgt. Pepper, Dark Side of the Moon, etc.
  
• Time is a critical element sensory perception. It allows the recognition of melodies, speech, or even touch. Neural activity must last longer in time than the individual input patterns. The higher up tin the cortex you go, the fewer changes over time you should see.
• Predictability is synonymous with reality. A predictable sequence of patterns is only possible if it is part of a larger object that really exists.
  
• Handwritten words are often unintelligible out of context, but very readably within a full written sentence. Most of the time you aren't aware that you are filling in the ambiguous or incomplete information from you memories of sequences. You hear what you expect to hear and see what you expect to see - at least when what you hear and see fits into past experience.
• What we see, hear, feel is highly dependent on our own actions: eye movement, turning the head to hear, moving limbs and fingers.
• Unexpected input requires involvement at the higher parts of the hierarchy. New predictions propagate down the hierarchy as far as it can go. If the new prediction is not right, an error will be detected and again climb back up until some region can interpret it as part of its currently active sequence. This can lead to the"aha" moment of sudden conprehension.
• Hebbian learning principles can explain most cortical behavior - that is when two neurons fire at the same time, the synapses between them get strengthened.
• Three epochs of memory: 1) when species used DNA as the medium for memory. 2) when nature invented modifiable nervous systems that could quickly form memories. 3) the invention of language and the expansion of our large neocortex.
• Shakespeare's metaphors (e.g. "there's dagggers in men's smiles.") become obvious when you see them, but they're very hard to invent. This is genius.
• Deja vu, or a twinge of recognition is just an invariant representation in the brain that was activated by a novel situation that made an analogy between two normally unrelated events.
• To be more creative: 1) assume up front that there is an answer to what you are trying to solve. 2). let you mind wander. Ponder often, but do other things so the cortex will have the opportunity to find an analogous memory.
• Imagination is letting your predictions turn around and become inputs. without physically doing anything, you can follow the consequences of your predictions.
• Hawkins invented Graffiti for the Palm Pilot. It was successful because our brains prefer systems that are consistent and predictable and we like learning new skills.
  
• Consciousness is simply what it feels like to have a neocortex.Two major categories: 1) similar to self-awareness the everyday notion of being conscious. 2) qualia - the idea that feelings associated with sensation are sonehow independent of sensory input.
• If your memory of the last two hours is erased, you would claim to have been unconscious and not responsible for your actions during that time. Thus memory erasure can cange the meaning of cconsciousness.
• For an intelligent machine to pass the Turing test, it would have to have most of the experiences and emotions of a real human, and to live a humanlike life. This is because the human mind is created not only by the neocortex but also the emotional systems of the old brain and by the complexity of the human body.
  
• Intelligent machine will likely have a different set of senses than humans. They very likely will "exist" in a very different world. Its model of reality will be based on its own world using its own senses. The physical embodiment of an intelligent machine could take many forms. These forms can even change over time, possibly being closer to a hive mentality than a single biological unit. An amalgamation of several robots would form, merge, and reform so that the concept of a single intelligence would be a fluid concept. The major invariant defining this intelligence would be how it understands and interacts with its world via a hierarchical memory model.
• Unlike current computer chips, intelligent memory chips will be inherently of fault tolerant. One side effect of this ability will be the capability to build chips that are significantly larger and denser that today.
  
• Ray Kurzweil talks about nano-robots that will crawl within your brain recording every synapse and every connection and then report all the information to a supercomputer, which will reconfigure itself into you.
• Building intelligent machines is not the same as building self-replicating machines. Ther is no logical connection whatsoever between them. Neither brains nor computers can directly self-replicate.
• Unlike human learning, intelligent machines will be able to share components of learning and knowledge. They can also be reprogrammed with a new set of connections to lead to different behavior.