It’s the software

It’s the software that’s holding us back in the AI realm, not hardware.

Kevin Drum does say things that are incredibly smart at times as long as he stays away from economics, the stock market, or related. It’s why I keep reading his site. Or at least skimming it.

If anything, software has more potential for exponential growth right now than hardware, and lots of people are working on this.

For probably 20-30 years, lack of software has been the main hindrance of AI, not hardware. We’ve had good enough hardware for near-human-level AI for a while now, but not the first clue how to produce software to run on it.

Also true:

As for the argument about sociability, I find this even more baffling. It seems to spring from a desire to believe that there just has to be something unique about human beings. I don’t really see why. The human brain is, in a sense, an existence proof that it’s possible to construct a human brain. But if it’s possible at all, why shouldn’t it be possible to construct one using solid state electronics rather than organic chemistry? And if it’s possible to build one using solid state electronics, why shouldn’t it be able to learn sociability just like human children do?

By the way, all the hoopla you’ve heard about processor speed improvements and Moore’s law? While not unimportant, algorithmic optimization has absolutely trounced any hardware-based gains. It’s not even remotely close.

The algorithms that we use today for speech recognition, for natural language translation, for chess playing, for logistics planning, have evolved remarkably in the past decade. It’s difficult to quantify the improvement, though, because it is as much in the realm of quality as of execution time.

In the field of numerical algorithms, however, the improvement can be quantified. Here is just one example, provided by Professor Martin Grötschel of Konrad-Zuse-Zentrum fur Informationstechnik Berlin. Grötschel, an expert in optimization, observes that a benchmark production planning model solved using linear programming would have taken 82 years to solve in 1988,using the computers and the linear programming algorithms of the day. Fifteen years later – in 2003 – this same model could be solved in roughly 1 minute, an improvement by a factor of roughly 43 million. Of this, a factor of roughly 1,000 was due to increased processor speed, whereas a factor of roughly 43,000 was due to improvements in algorithms! Grötschel also cites an algorithmic improvement of roughly 30,000 for mixed integer programming between 1991 and 2008.

Also unknown to most, we are still making such algorithmic improvements today.

This will not get us to AI but it is part of the path.

The world will change more than most people realize in the next 50 years. Perhaps not for the better, but it will certainly not be the steady state that is now imagined — not in the realm of AI, nor in climate change, nor in biotech.