Robot behavior, Babies, Careers and HTMs

Greetings everyone,

The crew had to disband to address career and lifestyle changes. Trevor had a baby, Rob is the .NET architect and team lead for a rapidly growing company and Bert picked up a heavy consulting gig for 5 months ... but I'm back in the AI game and ready to start blogging again.

The crew still gets together occasionally and we try to work on the behavior AI code as much as possible. Here's a quickie update from the past year.

• We've learned ALOT from trial and error and have decided to perform a bottom-up approach to a generalized AI behavior system.
• We've ditched MS Robotics Studio (for now - steep learning curve) but we still utilize the CCR (Concurrency and Coordination Runtime) to help us coordinate threading in our code.

We've written custom code to integrate to robotic substrates (Lego and iRobot). I will be publishing a C#.NET-based implementation of the iRobot Create specification in a future blog posting.

• Integration of our custom HTM code (Hierarchical Temporal Memory) has been deferred to a more appropriate "strata" of our AI behavior framework. (see Mammalian layer in diagram below)
  
• We're pursuing a more biologically inspired approach by focusing on evolutionary models of animal behavior.
  
• We've also added an emotion machine implementation to our framework (not shown below)
  

This is obviously a simplified object model of our new framework. Since we're not publishing our code publicly - I cannot provide any more details on the object model (the true object model has 40-50 classes). But this gives you a good idea on the layered hierarchical approach we're using for our AI behavior system.

I will be discussing the inspiration behind this new design in the next post.
Great to be back.

RoboCup 2007 - Quarter Finals

The following is an excellent video cut between Team Nimbro and the Darmstadt Dribblers in the humanoid 2 on 2 quarter finals.



The following is the entire game between Carnegie Mellon and TJark (Tonji University) from China (approx 30 mins). Quarterfinal match held July 6, 2007 at RoboCup 2007 in Atlanta, GA.

RoboCup 2007 - Videos - Thursday

2 on 2 Humanoid Robot Soccer Preliminaries


Girl performs unit testing on soccer robot.

RoboCup 2007 - Video - Wednesday

RoboCup 2007 - Georgia Tech - Atlanta

Here are some pix from this year's RoboCup 2007 in Atlanta, GA. International teams are well represented.

Most pictures in this post are from the Junior Soccer preliminary competitions and mid-size / humaniod soccer league prelims.

Here are a bunch of random pics.

Alot of the robots from the junior leagues spun in place when they couldn't detect the ball.

Hierarchical Temporal Memory

We've completed a functional (and much better) version of our .NET-based Hierarchical Temporal Memory (HTM) engines (great job Rob). We're also still working on an HTM based robotic behavioral framework (and our 1st quarter goal -- yikes - we're late). Also, we are NOT using Numenta's recently released run-time and/or code... since we're professional .NET consultants/developers, we decided to author our own implementation from initial prototypes authored over the summer of 2006 during an infamous sabbatical -- please don't ask about the "Hammer" stories.

I've been feeling that the team has not been in synch in terms of HTM concepts, theory and implementation. We decided to spend the last couple of meetings purely focused on discussions concerning HTMs. This has resulted in a new HTM based initiative (in line with our charter) that utilizes HTMs as the basis of chess playing game engine and in the team rededicating itself to gain deeper insights into HTM-based networks and AIs. I decided to author a simple HTM narrative (below) with an example. I wanted the example to illuminate the potential power in a simple HTM network when attached to a set of sensors. We have used this example as a powerful springboard into deeper analysis and discussions especially regarding multi-level hierarchies (2 - 4 levels).

We have divided our HTM node implementation into 2 high level types. 1) Sensor Node and 2) Cortical Node. Only sensor nodes are tied directly to sensory input. Cortical nodes can have either sensor nodes or other cortical nodes as children. Sensor Nodes retrieve inputs directly from the sensor sources while cortical nodes receive the temporally processed inputs of their children. You can learn the basics from this whitepaper (PDF) from Numenta.

SIMPLE HTM NARRATIVE
An HTM sensor node provides a mechanism to memorize sensor inputs and sequences of those inputs. More generically, an HTM sensor node provides a memory of input and a memory of the temporal contexts for those inputs.

An HTM cortical node provides a mechanism to memorize and associate the temporal contexts of its children (as a spatial memory) AND also stores the temporal-based trends of these associations as a temporal memory.

EXAMPLE
temp = temperature sensor
pressure = barometric sensor
light = luminousity sensor
moisture = humidity sensor

TEMP Sensor
The temperature HTM sensor node is memorizing temperatures and also the temporal trends of those temperatures
Spatial Analysis: High temperature
Temporal Analysis: Temperatures have been getting hotter

PRESSURE Sensor
The pressure HTM sensor node is memorizing pressures and also the temporal trends of those pressure values
Spatial Analysis: Low Pressure
Temporal Analysis: Pressure has been dropping slowly

LIGHT Sensor
The light HTM sensor node is memorizing light intensities and also the temporal trends of those light intensities
Spatial Analysis: Low Light
Temporal Analysis: It has been getting darker

MOISTURE Sensor
The moisture HTM sensor node is memorizing moisture values and also the temporal trends of those moisture values
Spatial Analysis: Some moisture
Temporal Analysis: Some moisture (but more than last time)

HTM Cortical Node
The cortical HTM node is memorizing the simulateneous trends of the temperature, pressure and light intensity sensors AND

The cortical HTM node is ALSO memorizing the sequences of these simulateneous trends of temperature, pressure and light intensity -- over time

Spatial Analysis: (Temps are increasing WHILE pressure is low WHILE it’s getting darker WHILE some moisture) = STORM IS FAR BUT APPROACHING
Temporal Analysis: (Temp Hotter Trend AND Pressure Lowering Trend and Darker Trend AND wetter) = STORM IS NEAR AND APPROACHING

CONCLUSION
From just a simple set of sensors tied to HTM sensor nodes which are themselves children of a single parent cortical HTM node, you can inspect the temporal memory (Temporal Analysis) of the cortical node and get a strong correlated "abstraction" of what's going on (STORM IS NEAR AND APPROACHING ).

The strength of this conclusion stems from the fact that the 4 sensors (and their values) are correlated (associated) by their common temporal contexts and by their inherent association via the single parent cortical node. Said another way, the abstraction of "STORM IS NEAR AND APPROACHING" is a strong belief because the 4 sensor inputs were gathered and associated during the same time, which provides the causal strength of the basis of the conclusion.... whereas, if the sensor values were NOT gathered at the same time (i.e. temperature and humidity were taken hours apart) then the correlated basis of the conclusion would be much weaker.

We've decided to use narratives like this one to help us understand what the HTM nodes are actually comparing and correlating - without getting into any hairy technical or mathematical analysis .... I feel strongly that these dicussions have helped the team grasp deeper understandings of its HTMs implementations.

We have also reminded each other the fact that until we have anything working -- we're just pontificating like a bunch of old philosophers. Since we're all software professionals, we understand that we don't have shit until "its in production"....

but is fun to dream once in a while ... I just wish we had more time to work on this stuff :( ....

HTMs - Memory, Machines & Motivation

This blog posting will focus on Rob's presentation on Hierarchical Temporal Memory (HTM) as defined by Jeff Hawkins of Numenta in his onIntelligence book (published in 2004). HTMs, sometimes called memory prediction frameworks, will play a central role in the team's goal of buidling a generalized AI framework and the basis of a robotic nervous system. This is just a high level overview of HTMs being used for our purposes, more details are available here and here.


Prediction Machine: Rob described how HTMs are modeled after a common cortical structure present in the human neocortex. In a nutshell, the human brain stores sensory input in a layered hierarchial memory structure. Each computational unit of this memory framework processes spatial patterns (patterns over space) and temporal patterns (patterns over time). Current sensory patterns are matched against patterns in memory and the closest matching pattern category is sent up the memory hierarchy. As categories of patterns continue traveling up the hierarchy an invariant view of the sensory data is formulated. Spatial and temporal patterns are stored in memory with more detailed/specific patterns existing in the lower levels of the hierarchy and generic/invariant models living in the higher levels.

This invariant model of the world is then used to form predictions based on prior and current sensory input patterns. Current sensory inputs are constantly checked against predictions forming the initial basis for a behavioral action (motor or otherwise). Predictions also allow the brain (HTM) to supplement current sensory input to form a complete picture from sensory data that may be missing information (i.e. "fill in the blanks").

Exception & Attention Machine: A prediction that fails miserably against current sensory input can be considered a violation of expectation or an exceptional event. In computer science, exceptional events are called exceptions and are usually handled immediately or allowed to "bubble up" to a high-level handler for processing. The brain also seems to have an "exception handling" mechanism of its own. When current sensory input "surprises" ... the brain commands the body to pay special attention to the source of the surprising sensory input. A good example is when you see an unrecognized motion from your peripheral field of vision. You immediately turn your head to focus your eyes (and senses) to the source. Attention was probably a very old adaptation of the reptillian brain to focus an organism's full resources to detection of a predator (or prey).

MOTIVATION & MOTOR BEHAVIOR
What do HTMs have to do with robots? Well... we're seeking to use HTMs as a primary tool for a generalized AI system that can drive robot behavior. We're writing code to simulate the common cortical algorithm defined by Jeff Hawkins of Numenta. Motor behavior will be implemented from processing the HTMs predictions and sending commands to robotic servos. We're still in discussions about how to model a robotic nervous system with HTMs as a central component. Team members have discussed potentially modeling innate behaviorial control mechanisms such as pain and hunger. The team's goal is to get robotic behavior to emerge as a response to its environment versus explicity coding specific goals and behaviors into the robot. This is one of our prime directives.

LIMITATIONS & REALITIES
The obvious limitations to the implementation of HTMs are the amount of memory and CPU resources required to effectively simulate the neocortex and the amount of training necessary to get the robot/HTM anywhere near operational (i.e. a baby). We're using Microsoft .NET to implement the HTM and the MS Robotics Studio to tie the HTM to the robot hardware. We will use Microsoft's provider model when implementing the memory structures... since we know initially, we wont have enough RAM to store the memory component .. we will probably persist to Flash memory or hard drive. Other specific goals of our HTM development is complete serialization/deserialization of memory structures and multithreading the processing of sensor nodes.

WRAP-UP

Some thoughts brought up by team members.

Hawkins talks about the massive amount of feedback that seems to take place going down the memory hierarchy. Trevor mentioned that this is probably due to a set of predictions being sent down (versus just the best prediction) to better correlate (in a bayesian fashion) with the current sensory input.

Another interesting point brought up by Rob was specific to an HTM implementation issue regarding categorization of the temporal patterns. Bert prototyped an HTM implementation that was string based but Rob was wondering if a pre-quantized numeric based category structure will be better for the HTM sensor nodes ... this will overcome a nuance of Hawkins design that forces you to evolve the categories over time. Rob's design will require knowing a priori the range of values for sensor input but may provide an interesting alternative.

Team talked about how inhibition should be modeled specifically in an HTM implementation. Maybe an HTM node can inhibit the temporal pattern categories of other HTM nodes by some unknown mechanism ...can probably gather some ideas from modern neural networks.