DARPA and hierarchical temporal memory

In a recent comment, I linked to an article entitled "Deep Machine Learning- A new Frontier in Artificial Intelligence Research.".

In it, the creators of the DESTIN architecture, who I have mentioned before in this blog, attempt to summarize the field of work going on with Deep Machine Learning, or the idea of using hierarchies to learn in a more brain-like manner. What was interesting to me about the article was that it mentioned a DARPA research effort involving deep machine learning architectures. In April 2009, DARPA put out a request for proposals for deep machine learning architectures. The military is increasingly worried about the vast amount of data that it collects that goes unanalyzed due to the sheer volume of data that humans do not have the time to analyze. DARPA seeks an HTM-like algorithm that will find patterns in this vast data. The DARPA announcement closed in April 2010, and to my shock, I don't see any indication that Numenta put in a proposal (among others, it appears that the DESTIN folks did). In a briefing, DARPA set out a list of desirable properties that would be features of the algorithms resulting from the multi-year research effort. Here is the list:

1. Single learning algorithm and uniform architecture for all applications

2. Unsupervised and supervised learning, including with attention mechanisms

3. Increasingly complex representations as you ascend the hierarchy. Sparse representations were mentioned here

4. The ability to learn sequences and recall them auto-associatively

5. Recognize novelties at each level and escalate them in the hierarchy

6. Feedback for predictions to fill-in missing input

7. Online learning

8. Parameters set themselves and need no tweaking

9. Neuroscience insights to inform the algorithms and architecture

Essentially, that list of desirable features in DARPA's envisioned software is a description of the HTM algorithms. Its difficult to imagine why Numenta didn't throw their hat in the ring given the amount of money potentially involved if the technology catches the eye of the military. In any event, DARPA's document was very interesting reading.