Narrow versus broad AI

One interesting question to think about is whether Numenta's current focus on implementing its HTM algorithms on narrow AI problems actually hurts its usefulness. I say that because AI is actually getting pretty good at certain narrow AI problems (handwriting and speech recognition, certain types of computer vision, game playing, etc.). For instance, in a recent study, HTM's were in the middle of the pack in recognition accuracy when tested on optical character recognition. Of course, the counterargument is that HTM's don't just do character recognition, but can be used for many, many other applications. Yet, that gets to my point. One wonders if, for any narrow AI endeavor, HTM's will be outflanked by competing AI techniques designed specifically for that one, narrow task.

If this is truly a problem, the obvious solution would be for Numenta to ultimately focus HTM's on broad competence AI (i.e. robots that can carry on a conversation with you, reason intelligently on novel problems, make independent decisions, and otherwise learn like a human). Yet, Jeff Hawkins himself in his book envisioned little or no role of this type for his technology, at least in terms of robots.

The difference between the human brain and any computer is not that the human brain can do all thinking/reasoning tasks better than all computers. In fact, computers are now much better at certain tasks than humans, such as number crunching and playing chess. What is different is that the human chess player can do that and a million other things, and can understand how the million different things all relate together in a complete world model.

Dileep George of Numenta has actually publicly touched on a related topic in recent months. He discussed the No Free Lunch theorem, which states that no learning algorithm is inherently superior to other learning algorithms. If an algorithm seems to be better at a certain task than other algorithms, it is only because that algorithms is written based on certain assumptions about the world that are applicable to that task. The more assumptions that an algorithm makes about the world, the better it will be at a task in an area where the algorithm can exploit its world assumptions. Yet, that also means that while the algorithm will be better at specific problems where those assumptions are applicable, it will be worse at other problems where you can't make those assumptions.

Numenta believes that HTM's take advantage of two properties of the world that are also exploited by the human brain: 1) the world is hierarchical in nature, and 2) all learning must be done through time. In other words, HTM's work because we live in a hierarchical world in space and time. In narrow domains such as chess playing, the AI algorithms are designed specifically and only for that single task. The engineer thus makes many, many assumptions about the world in coding the algorithm. For this reason, a chess playing computer can beat the world's best human at chess, but knows absolutely nothing else about the world.

The No Free Lunch theorem brings me back to my initial point. Would Numenta be better off focusing on broad competence AI? Assuming that HTM's are based on the same assumptions about the world as the human brain, HTM's will be the best means of emulating human-level AI. On the other hand, for the many, many different narrow AI problems out there, the algorithms being developed specifically for such problems might be better. Time will tell. So far, Numenta has mostly focused on computer vision, and that might actually be a broad rather than narrow AI problem, given the huge amount a computer needs to know about the world to truly be able to understand what it is seeing as well as a human. It will be interesting to see the direction Numenta takes in coming years.

Vitamin D's surveillance software no longer in beta

Vitamin D has reached a milestone by becoming the first company to create a working product out of HTM technology that can now be purchased (it was released in beta form in November). See here for pricing information: http://vitamindinc.com/store/pricing.php

It looks like a single webcam is still free, while for $49 you get support for two webcams in either 320 by 240 or full VGA resolution. For $199, you get support for an unlimited number of cameras at full VGA resolution (although the company does not recommend using more than 3-4 cameras for a dual core PC or 6-8 cameras per quad-core PC).

For now, the software recognizes the presence of humans in videos (as opposed to other moving objects like cars, animals, or tree brances). The future of this software will be quite interesting, as Vitamin D has already noted that it plans to upgrade the software in the future to detect more sophisticated actions in video.

New version of HTM algorithms in October

A couple of weeks ago, Numenta sent out a newsletter in which it revealed that it plans a major new release of its software implementation of HTM. It will be released in October 2010. The newsletter said that the company has had some recent insights into the HTM learning algorithms based on a deeper understanding of the biology of the neocortex. Numenta said that the new algorithms have the potential for a "large" increase in scalability and robustness.

One of the things that makes Numenta such a solid AI company is that when they run into problems with issues like scalability and robustness, they look to the brain itself for solutions. Even to a non-expert like me, it is obvious that the whole field of artificial intelligence has floundered for the past 70 years precisely because it has ignored the only known example of real intelligence, the neocortex of the mammalian brain. Jeff Hawkins' book made this very point. He decided in the mid 1980's that he wanted to enter a PHD program to create intelligent machines using the brain as his guide to doing it. He tried to apply to MIT, the leading AI lab in the country, and they basically laughed him out of the building for believing that it was necessary to understand how the brain works to create real AI. Now, 25 years later, MIT has a research group doing exactly what Hawkins suggested as a graduate student.

Hawkins' ideas may not all be correct, but the progress that has been made in AI in the last five years or so seems to be much more biologically grounded than 20 years ago, so Numenta is clearly on the right track by emulating the brain. If the new software really is a large improvement in its ability to scale (currently it is quite limited in many ways), we might actually begin to see the software begin to approach human-level ability at certain tasks, such as visual pattern recognition.