Looks like a press release was issued today by EDSA Power Analytics, noting that it is teaming up with Numenta to develop autonomous monitoring of electrical power systems. Apparently, for some applications, electrical power failures are hugely expensive. EDSA develops software to monitor power systems to attempt to prevent power failures. HTM can be used by EDSA to learn the difference between normal and non-normal electrical activity, such that the HTM becomes increasingly able to predict a future power failure.
http://www.businesswire.com/portal/site/home/permalink/?ndmViewId=news_view&newsId=20100420005529&newsLang=en
Tuesday, April 20, 2010
Thursday, March 25, 2010
Hawkins explains next generation HTM algorithms
On March 18, Jeff Hawkins gave a talk to the computer science department at the University of British Columbia. In that talk, for the first time, he gave a detailed explanation regarding the upcoming HTM algorithms. To be honest, parts of it were difficult for a layperson (like me) to fully understand. It is a very interesting talk, though, and showed that Numenta is getting ever closer to a truly intelligent computer. This version of the algorithms appears to much more closely mimic the brain's method of learning than ever before. Here is a link to the speech:
http://www.youtube.com/watch?v=TDzr0_fbnVk
http://www.youtube.com/watch?v=TDzr0_fbnVk
Wednesday, February 17, 2010
Narrow versus broad AI
One interesting question to think about is whether Numenta's current focus on implementing its HTM algorithms on narrow AI problems actually hurts its usefulness. I say that because AI is actually getting pretty good at certain narrow AI problems (handwriting and speech recognition, certain types of computer vision, game playing, etc.). For instance, in a recent study, HTM's were in the middle of the pack in recognition accuracy when tested on optical character recognition. Of course, the counterargument is that HTM's don't just do character recognition, but can be used for many, many other applications. Yet, that gets to my point. One wonders if, for any narrow AI endeavor, HTM's will be outflanked by competing AI techniques designed specifically for that one, narrow task.
If this is truly a problem, the obvious solution would be for Numenta to ultimately focus HTM's on broad competence AI (i.e. robots that can carry on a conversation with you, reason intelligently on novel problems, make independent decisions, and otherwise learn like a human). Yet, Jeff Hawkins himself in his book envisioned little or no role of this type for his technology, at least in terms of robots.
The difference between the human brain and any computer is not that the human brain can do all thinking/reasoning tasks better than all computers. In fact, computers are now much better at certain tasks than humans, such as number crunching and playing chess. What is different is that the human chess player can do that and a million other things, and can understand how the million different things all relate together in a complete world model.
Dileep George of Numenta has actually publicly touched on a related topic in recent months. He discussed the No Free Lunch theorem, which states that no learning algorithm is inherently superior to other learning algorithms. If an algorithm seems to be better at a certain task than other algorithms, it is only because that algorithms is written based on certain assumptions about the world that are applicable to that task. The more assumptions that an algorithm makes about the world, the better it will be at a task in an area where the algorithm can exploit its world assumptions. Yet, that also means that while the algorithm will be better at specific problems where those assumptions are applicable, it will be worse at other problems where you can't make those assumptions.
Numenta believes that HTM's take advantage of two properties of the world that are also exploited by the human brain: 1) the world is hierarchical in nature, and 2) all learning must be done through time. In other words, HTM's work because we live in a hierarchical world in space and time. In narrow domains such as chess playing, the AI algorithms are designed specifically and only for that single task. The engineer thus makes many, many assumptions about the world in coding the algorithm. For this reason, a chess playing computer can beat the world's best human at chess, but knows absolutely nothing else about the world.
The No Free Lunch theorem brings me back to my initial point. Would Numenta be better off focusing on broad competence AI? Assuming that HTM's are based on the same assumptions about the world as the human brain, HTM's will be the best means of emulating human-level AI. On the other hand, for the many, many different narrow AI problems out there, the algorithms being developed specifically for such problems might be better. Time will tell. So far, Numenta has mostly focused on computer vision, and that might actually be a broad rather than narrow AI problem, given the huge amount a computer needs to know about the world to truly be able to understand what it is seeing as well as a human. It will be interesting to see the direction Numenta takes in coming years.
If this is truly a problem, the obvious solution would be for Numenta to ultimately focus HTM's on broad competence AI (i.e. robots that can carry on a conversation with you, reason intelligently on novel problems, make independent decisions, and otherwise learn like a human). Yet, Jeff Hawkins himself in his book envisioned little or no role of this type for his technology, at least in terms of robots.
The difference between the human brain and any computer is not that the human brain can do all thinking/reasoning tasks better than all computers. In fact, computers are now much better at certain tasks than humans, such as number crunching and playing chess. What is different is that the human chess player can do that and a million other things, and can understand how the million different things all relate together in a complete world model.
Dileep George of Numenta has actually publicly touched on a related topic in recent months. He discussed the No Free Lunch theorem, which states that no learning algorithm is inherently superior to other learning algorithms. If an algorithm seems to be better at a certain task than other algorithms, it is only because that algorithms is written based on certain assumptions about the world that are applicable to that task. The more assumptions that an algorithm makes about the world, the better it will be at a task in an area where the algorithm can exploit its world assumptions. Yet, that also means that while the algorithm will be better at specific problems where those assumptions are applicable, it will be worse at other problems where you can't make those assumptions.
Numenta believes that HTM's take advantage of two properties of the world that are also exploited by the human brain: 1) the world is hierarchical in nature, and 2) all learning must be done through time. In other words, HTM's work because we live in a hierarchical world in space and time. In narrow domains such as chess playing, the AI algorithms are designed specifically and only for that single task. The engineer thus makes many, many assumptions about the world in coding the algorithm. For this reason, a chess playing computer can beat the world's best human at chess, but knows absolutely nothing else about the world.
The No Free Lunch theorem brings me back to my initial point. Would Numenta be better off focusing on broad competence AI? Assuming that HTM's are based on the same assumptions about the world as the human brain, HTM's will be the best means of emulating human-level AI. On the other hand, for the many, many different narrow AI problems out there, the algorithms being developed specifically for such problems might be better. Time will tell. So far, Numenta has mostly focused on computer vision, and that might actually be a broad rather than narrow AI problem, given the huge amount a computer needs to know about the world to truly be able to understand what it is seeing as well as a human. It will be interesting to see the direction Numenta takes in coming years.
Wednesday, February 10, 2010
Vitamin D's surveillance software no longer in beta
Vitamin D has reached a milestone by becoming the first company to create a working product out of HTM technology that can now be purchased (it was released in beta form in November). See here for pricing information: http://vitamindinc.com/store/pricing.php
It looks like a single webcam is still free, while for $49 you get support for two webcams in either 320 by 240 or full VGA resolution. For $199, you get support for an unlimited number of cameras at full VGA resolution (although the company does not recommend using more than 3-4 cameras for a dual core PC or 6-8 cameras per quad-core PC).
For now, the software recognizes the presence of humans in videos (as opposed to other moving objects like cars, animals, or tree brances). The future of this software will be quite interesting, as Vitamin D has already noted that it plans to upgrade the software in the future to detect more sophisticated actions in video.
It looks like a single webcam is still free, while for $49 you get support for two webcams in either 320 by 240 or full VGA resolution. For $199, you get support for an unlimited number of cameras at full VGA resolution (although the company does not recommend using more than 3-4 cameras for a dual core PC or 6-8 cameras per quad-core PC).
For now, the software recognizes the presence of humans in videos (as opposed to other moving objects like cars, animals, or tree brances). The future of this software will be quite interesting, as Vitamin D has already noted that it plans to upgrade the software in the future to detect more sophisticated actions in video.
Friday, February 5, 2010
New version of HTM algorithms in October
A couple of weeks ago, Numenta sent out a newsletter in which it revealed that it plans a major new release of its software implementation of HTM. It will be released in October 2010. The newsletter said that the company has had some recent insights into the HTM learning algorithms based on a deeper understanding of the biology of the neocortex. Numenta said that the new algorithms have the potential for a "large" increase in scalability and robustness.
One of the things that makes Numenta such a solid AI company is that when they run into problems with issues like scalability and robustness, they look to the brain itself for solutions. Even to a non-expert like me, it is obvious that the whole field of artificial intelligence has floundered for the past 70 years precisely because it has ignored the only known example of real intelligence, the neocortex of the mammalian brain. Jeff Hawkins' book made this very point. He decided in the mid 1980's that he wanted to enter a PHD program to create intelligent machines using the brain as his guide to doing it. He tried to apply to MIT, the leading AI lab in the country, and they basically laughed him out of the building for believing that it was necessary to understand how the brain works to create real AI. Now, 25 years later, MIT has a research group doing exactly what Hawkins suggested as a graduate student.
Hawkins' ideas may not all be correct, but the progress that has been made in AI in the last five years or so seems to be much more biologically grounded than 20 years ago, so Numenta is clearly on the right track by emulating the brain. If the new software really is a large improvement in its ability to scale (currently it is quite limited in many ways), we might actually begin to see the software begin to approach human-level ability at certain tasks, such as visual pattern recognition.
One of the things that makes Numenta such a solid AI company is that when they run into problems with issues like scalability and robustness, they look to the brain itself for solutions. Even to a non-expert like me, it is obvious that the whole field of artificial intelligence has floundered for the past 70 years precisely because it has ignored the only known example of real intelligence, the neocortex of the mammalian brain. Jeff Hawkins' book made this very point. He decided in the mid 1980's that he wanted to enter a PHD program to create intelligent machines using the brain as his guide to doing it. He tried to apply to MIT, the leading AI lab in the country, and they basically laughed him out of the building for believing that it was necessary to understand how the brain works to create real AI. Now, 25 years later, MIT has a research group doing exactly what Hawkins suggested as a graduate student.
Hawkins' ideas may not all be correct, but the progress that has been made in AI in the last five years or so seems to be much more biologically grounded than 20 years ago, so Numenta is clearly on the right track by emulating the brain. If the new software really is a large improvement in its ability to scale (currently it is quite limited in many ways), we might actually begin to see the software begin to approach human-level ability at certain tasks, such as visual pattern recognition.
Wednesday, December 9, 2009
How to make computers that work like the brain
This is the interesting title of a short article presented by Dileep George (co-founder of Numenta) back in July 2009. It is available at the following link, but you need access to an educational database (such as through a University) to get to it:
http://portal.acm.org/citation.cfm?id=1630024
A couple of interesting points were made in the article: As was mentioned publicly by Subutai Ahmad in September, Numenta's third generation of algorithms are now being developed. This means that the algorithms are beginning to become more mature. As a result, Numenta is finally starting to think seriously about implementing HTM in hardware. Until now, HTM was such a new technology, and the algorithms have been so early in their development, that it was not feasible to use hardware.
It turns out that HTM in its software configuration is very CPU hungry, so a hardware implementation could be quite useful. For instance, Vitamin D's new beta webcam software that can recognize people in video can only analyze two 320 by 240 video feeds for each dual core 2 GHZ processor. Think about that. That means that if Vitamin D were to use its HTM-based software to analyze 640 by 480 video, it would require a quad core 2 ghz processor for EACH webcam feed. Extrapolating from that, it would be difficult to find a home computer powerful enough to analyze a single 1080P or even a single 720P video feed.
Perhaps software versions of HTM would be sped up significantly with GPU computing, however. HTM can apparently scale to a very large number of parallel CPU's, so one would think that a GPU could improve significantly over a CPU's performance.
http://portal.acm.org/citation.cfm?id=1630024
A couple of interesting points were made in the article: As was mentioned publicly by Subutai Ahmad in September, Numenta's third generation of algorithms are now being developed. This means that the algorithms are beginning to become more mature. As a result, Numenta is finally starting to think seriously about implementing HTM in hardware. Until now, HTM was such a new technology, and the algorithms have been so early in their development, that it was not feasible to use hardware.
It turns out that HTM in its software configuration is very CPU hungry, so a hardware implementation could be quite useful. For instance, Vitamin D's new beta webcam software that can recognize people in video can only analyze two 320 by 240 video feeds for each dual core 2 GHZ processor. Think about that. That means that if Vitamin D were to use its HTM-based software to analyze 640 by 480 video, it would require a quad core 2 ghz processor for EACH webcam feed. Extrapolating from that, it would be difficult to find a home computer powerful enough to analyze a single 1080P or even a single 720P video feed.
Perhaps software versions of HTM would be sped up significantly with GPU computing, however. HTM can apparently scale to a very large number of parallel CPU's, so one would think that a GPU could improve significantly over a CPU's performance.
Sunday, November 29, 2009
Turning point for HTM- Vitamin D releases first commercial HTM app.
Earlier this month, Vitamin D, Inc. issued a press release noting that it released a beta of video surveillance software based on Numenta's hierarchical temporal memory. This is the first commercial application of HTM, which represents a significant milestone for the technology. The software has been trained to recognize the presence of people in videos, which marks a significant improvement over less intelligent technologies that simply look for motion in video. It sounds as though future versions of the software will be able to distinguish different actions taken by people, which should open up many further commercial HTM possibilities. This is only the beginning for commercial HTM apps.
Subscribe to:
Posts (Atom)