Thursday, June 21, 2007
Hierarchical Temporal Memory
I have continued research on intelligent machines. In particular, perusing the forums for the On Intelligence book site. I also downloaded but have yet to install the Numenta software - Platform for Intelligent Computing (NuPIC) . I have also started looking at few other sites of interest:
Being new to the whole HTM model, I have been mostly struggling with the sensory data. It strikes me that the inferencing can be greatly enhanced if one can discount real time processing of data. My first idea for an HTM application was my lifelong ambition to develop an intelligent home. However that is far too grand a scale to begin with. I already envision a dozen or more HTM cooperating entities. For example, I had two basic ideas that involve simplification of sensor input. The first one was constructing an animated version of a web camera - sort of a "rotoscoped" version to simplify the scene. This would focus on shapes and colors. If you've seen the movie 'A Scanner Darkly' you get the general idea. For sound, particularly music, the analogy would be to represent the sound as a MIDI sequence. This represents the second sensor input stream.
The classification tasks that I envision harken back two decades for me to my initial work with expert systems. Those experiences still hold considerable influence over my mindset. This is one of the reasons that I was drawn to the Oracle implementation for an HTM. There is a potential synergy of HTMs with the RDF Semantic Web capabilities built into 10gR2. I find it interesting that Oracle has begun to morph the 'g' from 'CPU grid' to 'information grid' as demonstrated here:
Whereas Numenta seems to be focused on discovering and inferring causes, I have been concentrating on a "more macro" approach, Using this more holistic macro view, I have a slightly different take on the transition of the construction of a "belief" to the "make predictions" phase. Since a "belief" results from the distribution of possible causes, it strikes me that the transition is more of a "build hypothesis" phase that will "direct behavior" by querying a semantic web with the result being considered new sensory input that then gets translated into higher order "causes".
For example, I a see a ball rolling across the floor, my prediction will naturally be that it will continue to roll in the same direction. Big deal, this is so micro as to be of little interest to me. If I was an intelligent machine I would want to be "trained" as quickly as possible, performing my own research. This is how I would see it playing out between an intelligent machine (IM) and a human (H).
IM: what was that?
H: what was what? show me?
(IM replays raw webcam video of a ball rolling across the floor)
H: Oh, that's a ball rolling across the floor
(IM activates a search for "ball rolling across the floor", this leads to a semantic connection to object moving and eventually to Newton's Three Laws of Motion)
At this point the intelligent machine applies mathematical principles and can now make a prediction as to exactly where the ball will travel. I must add here that this is the one of the reasons for my fascination with animation, particularly computer animation based on wireframes and rigging. The mathematical models for almost all significant actions have already been worked out for humans, animals, liquids, motion of objects, smoke, explosions, you name it. These are all stored in libraries and can be accessed by name.
The challenge here is to apply the correct context frame. In HTM terminology this would be "covert attention". I believe that training the HTM in an efficient manner for higher level nodes will require human interaction to monitor the generated beliefs and direct them to appropriate context frames by giving names to the beliefs so that the power of the semantic web can be pulled in. Granted all of this in not in place yet, but I am convinced that it is already converging.
There is one example of an intelligent machine that sticks in my mind. The author related to the difficulty that an intelligent machine would have playing a simple game of hangman. This is because it deals with invariants and incomplete data. What would really make the machine intelligent, he wrote, would be for it to realize that it had a word dictionary available to it via a word processing application and use that to play the game, For the machine to realize what resources it could throw at the problem and utilize them would truly qualify it as intelligent. I am hypothesizing that training an HTM will require metadata available via the sematic web, particularly for all but the lowest level nodes. I also suspect that it will be essential to its continued growth and development.
- The recently released CLARAty Reusable Robotic Software from NASA
- White papers and other materials at the Singularity Institute
- Richard Bowles' excellent An Idiot's Guide to Neural Networks
- Neural Networks source code
- The memory-prediction framework Project Neocortex
Being new to the whole HTM model, I have been mostly struggling with the sensory data. It strikes me that the inferencing can be greatly enhanced if one can discount real time processing of data. My first idea for an HTM application was my lifelong ambition to develop an intelligent home. However that is far too grand a scale to begin with. I already envision a dozen or more HTM cooperating entities. For example, I had two basic ideas that involve simplification of sensor input. The first one was constructing an animated version of a web camera - sort of a "rotoscoped" version to simplify the scene. This would focus on shapes and colors. If you've seen the movie 'A Scanner Darkly' you get the general idea. For sound, particularly music, the analogy would be to represent the sound as a MIDI sequence. This represents the second sensor input stream.
The classification tasks that I envision harken back two decades for me to my initial work with expert systems. Those experiences still hold considerable influence over my mindset. This is one of the reasons that I was drawn to the Oracle implementation for an HTM. There is a potential synergy of HTMs with the RDF Semantic Web capabilities built into 10gR2. I find it interesting that Oracle has begun to morph the 'g' from 'CPU grid' to 'information grid' as demonstrated here:
The resources in the information grid span all the data in the organization, as well as all the metadata required to make that data meaningful. This data may be structured, semi-structured, or unstructured, stored in any location, such as databases, local file systems, or email servers, and created by any application. The vision for the information grid builds on the vision of the semantic web; the goal is to enable organizations to view all their assets in a smooth continuum, from the Internet to the Intranet, with uniform semantically rich access... An information grid is self-describing. Application modules can discover what sources exist, what data they possess, what the life cycle of that data is, and how that data should be interpreted. The information grid builds on the infrastructure and application grids.
Whereas Numenta seems to be focused on discovering and inferring causes, I have been concentrating on a "more macro" approach, Using this more holistic macro view, I have a slightly different take on the transition of the construction of a "belief" to the "make predictions" phase. Since a "belief" results from the distribution of possible causes, it strikes me that the transition is more of a "build hypothesis" phase that will "direct behavior" by querying a semantic web with the result being considered new sensory input that then gets translated into higher order "causes".
For example, I a see a ball rolling across the floor, my prediction will naturally be that it will continue to roll in the same direction. Big deal, this is so micro as to be of little interest to me. If I was an intelligent machine I would want to be "trained" as quickly as possible, performing my own research. This is how I would see it playing out between an intelligent machine (IM) and a human (H).
IM: what was that?
H: what was what? show me?
(IM replays raw webcam video of a ball rolling across the floor)
H: Oh, that's a ball rolling across the floor
(IM activates a search for "ball rolling across the floor", this leads to a semantic connection to object moving and eventually to Newton's Three Laws of Motion)
At this point the intelligent machine applies mathematical principles and can now make a prediction as to exactly where the ball will travel. I must add here that this is the one of the reasons for my fascination with animation, particularly computer animation based on wireframes and rigging. The mathematical models for almost all significant actions have already been worked out for humans, animals, liquids, motion of objects, smoke, explosions, you name it. These are all stored in libraries and can be accessed by name.
The challenge here is to apply the correct context frame. In HTM terminology this would be "covert attention". I believe that training the HTM in an efficient manner for higher level nodes will require human interaction to monitor the generated beliefs and direct them to appropriate context frames by giving names to the beliefs so that the power of the semantic web can be pulled in. Granted all of this in not in place yet, but I am convinced that it is already converging.
There is one example of an intelligent machine that sticks in my mind. The author related to the difficulty that an intelligent machine would have playing a simple game of hangman. This is because it deals with invariants and incomplete data. What would really make the machine intelligent, he wrote, would be for it to realize that it had a word dictionary available to it via a word processing application and use that to play the game, For the machine to realize what resources it could throw at the problem and utilize them would truly qualify it as intelligent. I am hypothesizing that training an HTM will require metadata available via the sematic web, particularly for all but the lowest level nodes. I also suspect that it will be essential to its continued growth and development.
Labels: HTM robotics intelligent-machines
