In his AI Speaker Series presentation at Sutter Hill Ventures, Mark Horowitz discussed the current state of hardware chip design and scaling along with significant challenges as Moore's law comes to an end. Here's my notes from his talk:
- We don't recognize is how pervasive our notion that computing will get cheaper in the future is. Everybody's building more complicated models that take longer to compute and the expectation is that that's OK, computers will be able to compute it.
- The driver of this expectation is Moore's law. but most people don't understand that Moore's law is really about cost per function. That cost scaling is not what it used to be.
- When transistor cost scale, making the same product in new technology is cheaper to do. That means you always moved all products to the most advanced technology.
- But that's not happening anymore so Moore's law has ended. Cost per transistor is on a linear scale. It's supposed to be log.
- If you plot the cost per bit over the past 60 years, DRAM prices are relatively flat. The hard drive has really bottomed out in terms of cost per bit. The only thing that's still scaling is SSDs.
- So scaling today is just a marketing label. People are expecting better performance but basic technology is not the way to go.
- What we need to do is increase efficiency and the only way we know how to increase efficiency is to increase customization for a particular end application. We need to tailor certain things for certain markets.
- We used to be able to build a universal thing and now we need build these little different products without bankrupting ourselves.
- Chiplets are not the answer. They are interesting and useful technology but won't solve the base problem.
- We need to do application optimization but who's going to do this optimization?
- Groups of students in the past could build a whole microprocessor.
- The systems that are competitive today require investments of hundreds of millions of dollars. Most of the cost is the firmware, the basic interfaces because that's where all the complexity is.
- Because of this complexity, the number of companies in the silicon space is decreasing. And student interest in hardware is decreasing because it's so opaque to them and the level of complexity that they need to make contributions is very high.
- Given the situation, we need innovation now more than ever before. And we have nobody to do it.
- To get great improvements in application optimization, we need radical thinking. In 99% or 95% of the cases it doesn't work but in 5% of cases, there's something interesting.
- If every experiment costs you $100 million, you're not going to find a few percent of ideas that actually are good.
- So can we make this exciting again and bring in new people? And make it cheap so that people can actually do it?
- The good news is this happened before. In the 70s there were only custom chip designs. A bunch of crazy people in the 80s had this idea not to help the custom designers, but to enable another group of people who were interested in hardware design to basically build chips. Those were the logic designers, the people who used our chips and put them on boards.
- To do that they had to create a whole different level of tooling that interfaced with people not thinking about chip architecture. These tools created really crappy chips but in 10 years it enabled a vibrant design community and the tools improved.
- Now nobody does custom.
- So we need a new group of people to throw spaghetti against the wall, because some of it might actually be useful.
- We need to do hardware software co-design with performance engineers in the application space. They have no knowledge of hardware they know about locality, parallelism and metrics.
- These users need to have a system to interact with an open interface to a proprietary platform for people to make money. We need to figure out how to map an application to hardware automatically.
- The application designer need feedback at the level of their software about where the bottlenecks are.
- All this stuff is hard but it doesn't seem like it is impossible. And if we're going to make forward progress, we really do fundamentally need to change the way we think about design.
- Because chip design and designers are a smaller group of people that are getting more gray hair. And we're going to end up in a universe, where we all are dependent on technology that none of us understand how it works.