Big Numenta news today. Numenta's website indicates that Dileep George is on an extended "personal" leave of absence. He, along with Jeff Hawkins, co-founded Numenta back in 2005. It is hard to overstate his importance to the company over the years. He was the guy who read "On Intelligence" and figured out how to turn Hawkins' neuroscience theories into a mathematical expression that could be created in software. I went to George's website, and he says there that he left Numenta so that he could form a new company focused more on applications of the HTM technology.
I am not sure what to think about this. On the one hand, this could signal that George simply thinks that the technology is finally in a state that serious commercial applications can now be created with HTM. Numenta has always been more about the basics of the theory than applications, so it might be that Dileep just wants to hurry along the commercialization process. This could be a signal that the new algorithms really are going to be that big of a step forward for AI.
Hopefully this move doesn't mean some type of rift has opened between George and the company. Numenta could really use his talents down the road. Given George's stated reason for leaving, and given that it is called a "leave of absence" rather than outright resignation, I am inclined to go with the more optimistic interpretation.
Wednesday, May 19, 2010
Friday, May 7, 2010
May 2010 Numenta newsletter
In case you missed it, this week Numenta issued a newsletter with an update on the status of the new algorithms. Its an interesting read, with a write up by Jeff Hawkins himself. He talks about how last fall they decided to take a fresh look at their node learning algorithms, realizing that the current version had shortcomings that could not be overcome. They went back and looked at the brain for inspiration on how to improve the learning capabilities. Here are a few keys points I pulled out of the article:
1. The new algorithms can be learning and inferring at the same time. The old algorithms had a separate learning and inference stage, so this will be a significant improvement for many applications with real time data where the system needs to be able to learn, infer, and predict in real time (like a real brain).
2. The sparse distributed nature of the system makes it scale much better to large problems, and makes it very robust to noise. In other words, the system will work very well with messy, incomplete data.
3. Variable order sequence learning- A real brain can start listening to a song midway through and almost immediately identify the song. Likewise, we can predict the future based on learned sequences of various lengths that occurred a short time ago or years ago. The new software will be useful in doing these types of things.
4. Much more biologically realistic- This is the first version of the software that will basically be emulating the cortex at the level of neurons and synapses. Of course, the downside is the higher system requirements. Hawkins notes that Numenta is having to spent a great deal more time optimizing the software to be able to work on something that isn't a supercomputer.
As an aside, I am surprised at the level of skepticism of some of the mainstream AI people regarding Numenta. Ben Goertzel, for one, seems determined to believe that Numenta is on the wrong track. He went out of his way recently to claim that Itamar Arel's DESTIN system is a better hierarchical pattern recognition system. I have looked into DESTIN, and it actually seems very similar to Numenta's work. It learns temporal sequences of spatial patterns in a hierarchical nature, and performs bayesian inference. I have not been able to find any evidence showing that DESTIN has, so far, done more in the computer vision arena than HTM. If I am wrong, someone can correct me. For instance, in a December 2009 paper regarding DESTIN, Arel noted that they had conducted an experiment showing character recognition. It was recognition of letters in a binary (black or white) environment. Numenta was demonstrating that level of work at least three years ago. My sense is that DESTIN is on the right track, and perhaps Arel and Hawkins will collaborate at some point (maybe they already are) but I have no idea how Goertzel reaches his conclusion.
I was happy to see that Shane Legg (another AI critic of Numenta) seemed to change his mind about Numenta after seeing Hawkins' recent talk in March on the new algorithms. If Numenta can come through in a big way with its next software release, I think that there will be many more converts to the HTM theory of AI.
1. The new algorithms can be learning and inferring at the same time. The old algorithms had a separate learning and inference stage, so this will be a significant improvement for many applications with real time data where the system needs to be able to learn, infer, and predict in real time (like a real brain).
2. The sparse distributed nature of the system makes it scale much better to large problems, and makes it very robust to noise. In other words, the system will work very well with messy, incomplete data.
3. Variable order sequence learning- A real brain can start listening to a song midway through and almost immediately identify the song. Likewise, we can predict the future based on learned sequences of various lengths that occurred a short time ago or years ago. The new software will be useful in doing these types of things.
4. Much more biologically realistic- This is the first version of the software that will basically be emulating the cortex at the level of neurons and synapses. Of course, the downside is the higher system requirements. Hawkins notes that Numenta is having to spent a great deal more time optimizing the software to be able to work on something that isn't a supercomputer.
As an aside, I am surprised at the level of skepticism of some of the mainstream AI people regarding Numenta. Ben Goertzel, for one, seems determined to believe that Numenta is on the wrong track. He went out of his way recently to claim that Itamar Arel's DESTIN system is a better hierarchical pattern recognition system. I have looked into DESTIN, and it actually seems very similar to Numenta's work. It learns temporal sequences of spatial patterns in a hierarchical nature, and performs bayesian inference. I have not been able to find any evidence showing that DESTIN has, so far, done more in the computer vision arena than HTM. If I am wrong, someone can correct me. For instance, in a December 2009 paper regarding DESTIN, Arel noted that they had conducted an experiment showing character recognition. It was recognition of letters in a binary (black or white) environment. Numenta was demonstrating that level of work at least three years ago. My sense is that DESTIN is on the right track, and perhaps Arel and Hawkins will collaborate at some point (maybe they already are) but I have no idea how Goertzel reaches his conclusion.
I was happy to see that Shane Legg (another AI critic of Numenta) seemed to change his mind about Numenta after seeing Hawkins' recent talk in March on the new algorithms. If Numenta can come through in a big way with its next software release, I think that there will be many more converts to the HTM theory of AI.
Tuesday, April 20, 2010
2nd public commercial HTM application
Looks like a press release was issued today by EDSA Power Analytics, noting that it is teaming up with Numenta to develop autonomous monitoring of electrical power systems. Apparently, for some applications, electrical power failures are hugely expensive. EDSA develops software to monitor power systems to attempt to prevent power failures. HTM can be used by EDSA to learn the difference between normal and non-normal electrical activity, such that the HTM becomes increasingly able to predict a future power failure.
http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20100420005529&newsLang=en
http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20100420005529&newsLang=en
Thursday, March 25, 2010
Hawkins explains next generation HTM algorithms
On March 18, Jeff Hawkins gave a talk to the computer science department at the University of British Columbia. In that talk, for the first time, he gave a detailed explanation regarding the upcoming HTM algorithms. To be honest, parts of it were difficult for a layperson (like me) to fully understand. It is a very interesting talk, though, and showed that Numenta is getting ever closer to a truly intelligent computer. This version of the algorithms appears to much more closely mimic the brain's method of learning than ever before. Here is a link to the speech:
http://www.youtube.com/watch?v=TDzr0_fbnVk
http://www.youtube.com/watch?v=TDzr0_fbnVk
Wednesday, February 17, 2010
Narrow versus broad AI
One interesting question to think about is whether Numenta's current focus on implementing its HTM algorithms on narrow AI problems actually hurts its usefulness. I say that because AI is actually getting pretty good at certain narrow AI problems (handwriting and speech recognition, certain types of computer vision, game playing, etc.). For instance, in a recent study, HTM's were in the middle of the pack in recognition accuracy when tested on optical character recognition. Of course, the counterargument is that HTM's don't just do character recognition, but can be used for many, many other applications. Yet, that gets to my point. One wonders if, for any narrow AI endeavor, HTM's will be outflanked by competing AI techniques designed specifically for that one, narrow task.
If this is truly a problem, the obvious solution would be for Numenta to ultimately focus HTM's on broad competence AI (i.e. robots that can carry on a conversation with you, reason intelligently on novel problems, make independent decisions, and otherwise learn like a human). Yet, Jeff Hawkins himself in his book envisioned little or no role of this type for his technology, at least in terms of robots.
The difference between the human brain and any computer is not that the human brain can do all thinking/reasoning tasks better than all computers. In fact, computers are now much better at certain tasks than humans, such as number crunching and playing chess. What is different is that the human chess player can do that and a million other things, and can understand how the million different things all relate together in a complete world model.
Dileep George of Numenta has actually publicly touched on a related topic in recent months. He discussed the No Free Lunch theorem, which states that no learning algorithm is inherently superior to other learning algorithms. If an algorithm seems to be better at a certain task than other algorithms, it is only because that algorithms is written based on certain assumptions about the world that are applicable to that task. The more assumptions that an algorithm makes about the world, the better it will be at a task in an area where the algorithm can exploit its world assumptions. Yet, that also means that while the algorithm will be better at specific problems where those assumptions are applicable, it will be worse at other problems where you can't make those assumptions.
Numenta believes that HTM's take advantage of two properties of the world that are also exploited by the human brain: 1) the world is hierarchical in nature, and 2) all learning must be done through time. In other words, HTM's work because we live in a hierarchical world in space and time. In narrow domains such as chess playing, the AI algorithms are designed specifically and only for that single task. The engineer thus makes many, many assumptions about the world in coding the algorithm. For this reason, a chess playing computer can beat the world's best human at chess, but knows absolutely nothing else about the world.
The No Free Lunch theorem brings me back to my initial point. Would Numenta be better off focusing on broad competence AI? Assuming that HTM's are based on the same assumptions about the world as the human brain, HTM's will be the best means of emulating human-level AI. On the other hand, for the many, many different narrow AI problems out there, the algorithms being developed specifically for such problems might be better. Time will tell. So far, Numenta has mostly focused on computer vision, and that might actually be a broad rather than narrow AI problem, given the huge amount a computer needs to know about the world to truly be able to understand what it is seeing as well as a human. It will be interesting to see the direction Numenta takes in coming years.
If this is truly a problem, the obvious solution would be for Numenta to ultimately focus HTM's on broad competence AI (i.e. robots that can carry on a conversation with you, reason intelligently on novel problems, make independent decisions, and otherwise learn like a human). Yet, Jeff Hawkins himself in his book envisioned little or no role of this type for his technology, at least in terms of robots.
The difference between the human brain and any computer is not that the human brain can do all thinking/reasoning tasks better than all computers. In fact, computers are now much better at certain tasks than humans, such as number crunching and playing chess. What is different is that the human chess player can do that and a million other things, and can understand how the million different things all relate together in a complete world model.
Dileep George of Numenta has actually publicly touched on a related topic in recent months. He discussed the No Free Lunch theorem, which states that no learning algorithm is inherently superior to other learning algorithms. If an algorithm seems to be better at a certain task than other algorithms, it is only because that algorithms is written based on certain assumptions about the world that are applicable to that task. The more assumptions that an algorithm makes about the world, the better it will be at a task in an area where the algorithm can exploit its world assumptions. Yet, that also means that while the algorithm will be better at specific problems where those assumptions are applicable, it will be worse at other problems where you can't make those assumptions.
Numenta believes that HTM's take advantage of two properties of the world that are also exploited by the human brain: 1) the world is hierarchical in nature, and 2) all learning must be done through time. In other words, HTM's work because we live in a hierarchical world in space and time. In narrow domains such as chess playing, the AI algorithms are designed specifically and only for that single task. The engineer thus makes many, many assumptions about the world in coding the algorithm. For this reason, a chess playing computer can beat the world's best human at chess, but knows absolutely nothing else about the world.
The No Free Lunch theorem brings me back to my initial point. Would Numenta be better off focusing on broad competence AI? Assuming that HTM's are based on the same assumptions about the world as the human brain, HTM's will be the best means of emulating human-level AI. On the other hand, for the many, many different narrow AI problems out there, the algorithms being developed specifically for such problems might be better. Time will tell. So far, Numenta has mostly focused on computer vision, and that might actually be a broad rather than narrow AI problem, given the huge amount a computer needs to know about the world to truly be able to understand what it is seeing as well as a human. It will be interesting to see the direction Numenta takes in coming years.
Wednesday, February 10, 2010
Vitamin D's surveillance software no longer in beta
Vitamin D has reached a milestone by becoming the first company to create a working product out of HTM technology that can now be purchased (it was released in beta form in November). See here for pricing information: http://vitamindinc.com/store/pricing.php
It looks like a single webcam is still free, while for $49 you get support for two webcams in either 320 by 240 or full VGA resolution. For $199, you get support for an unlimited number of cameras at full VGA resolution (although the company does not recommend using more than 3-4 cameras for a dual core PC or 6-8 cameras per quad-core PC).
For now, the software recognizes the presence of humans in videos (as opposed to other moving objects like cars, animals, or tree brances). The future of this software will be quite interesting, as Vitamin D has already noted that it plans to upgrade the software in the future to detect more sophisticated actions in video.
It looks like a single webcam is still free, while for $49 you get support for two webcams in either 320 by 240 or full VGA resolution. For $199, you get support for an unlimited number of cameras at full VGA resolution (although the company does not recommend using more than 3-4 cameras for a dual core PC or 6-8 cameras per quad-core PC).
For now, the software recognizes the presence of humans in videos (as opposed to other moving objects like cars, animals, or tree brances). The future of this software will be quite interesting, as Vitamin D has already noted that it plans to upgrade the software in the future to detect more sophisticated actions in video.
Friday, February 5, 2010
New version of HTM algorithms in October
A couple of weeks ago, Numenta sent out a newsletter in which it revealed that it plans a major new release of its software implementation of HTM. It will be released in October 2010. The newsletter said that the company has had some recent insights into the HTM learning algorithms based on a deeper understanding of the biology of the neocortex. Numenta said that the new algorithms have the potential for a "large" increase in scalability and robustness.
One of the things that makes Numenta such a solid AI company is that when they run into problems with issues like scalability and robustness, they look to the brain itself for solutions. Even to a non-expert like me, it is obvious that the whole field of artificial intelligence has floundered for the past 70 years precisely because it has ignored the only known example of real intelligence, the neocortex of the mammalian brain. Jeff Hawkins' book made this very point. He decided in the mid 1980's that he wanted to enter a PHD program to create intelligent machines using the brain as his guide to doing it. He tried to apply to MIT, the leading AI lab in the country, and they basically laughed him out of the building for believing that it was necessary to understand how the brain works to create real AI. Now, 25 years later, MIT has a research group doing exactly what Hawkins suggested as a graduate student.
Hawkins' ideas may not all be correct, but the progress that has been made in AI in the last five years or so seems to be much more biologically grounded than 20 years ago, so Numenta is clearly on the right track by emulating the brain. If the new software really is a large improvement in its ability to scale (currently it is quite limited in many ways), we might actually begin to see the software begin to approach human-level ability at certain tasks, such as visual pattern recognition.
One of the things that makes Numenta such a solid AI company is that when they run into problems with issues like scalability and robustness, they look to the brain itself for solutions. Even to a non-expert like me, it is obvious that the whole field of artificial intelligence has floundered for the past 70 years precisely because it has ignored the only known example of real intelligence, the neocortex of the mammalian brain. Jeff Hawkins' book made this very point. He decided in the mid 1980's that he wanted to enter a PHD program to create intelligent machines using the brain as his guide to doing it. He tried to apply to MIT, the leading AI lab in the country, and they basically laughed him out of the building for believing that it was necessary to understand how the brain works to create real AI. Now, 25 years later, MIT has a research group doing exactly what Hawkins suggested as a graduate student.
Hawkins' ideas may not all be correct, but the progress that has been made in AI in the last five years or so seems to be much more biologically grounded than 20 years ago, so Numenta is clearly on the right track by emulating the brain. If the new software really is a large improvement in its ability to scale (currently it is quite limited in many ways), we might actually begin to see the software begin to approach human-level ability at certain tasks, such as visual pattern recognition.
Subscribe to:
Posts (Atom)