In my job at Intel, I get to travel far and wide to meet with research groups working on parallel computing. And as travel, I am constantly struck by the differences between the state of HW and SW in parallel computing.On the HW front, we are sweating the details. We know there will be many cores. We know the cores in a single processor may be different. We know there will be a scalable on-die interconnect and that the caches will need to adapt to the workloads to maximize locality. I’ve spoken to countless HW architects working on many core systems and frankly, the “big picture” view seems pretty well locked up. But on the software front, it’s chaos. • Will we need new languages? Or will it be enough to extend existing languages? • Will data parallel programming languages allow general purpose programmers to write parallel code? Or are they the nightmare of HPF (High Performance Fortran) all over again and hence an utter waste of time? • All highly scalable codes use message passing … or they use a shared memory system with a message-passing-like approach. Can we make message passing easy enough for the general programmer to use? Or is it too hard for programmers and will never be adopted outside of HPC? • Frameworks such as Visual Basic revolutionized PC-applications. Can we design a framework-based approach for parallel programming? • How can we teach programmers to “think parallel”? I’m told by people I greatly respect that I shouldn’t worry. With many core chips going mainstream, there will be countless software engineers motivated to work on parallel programming. Someone will solve the problems for us. But I do worry. The high performance computing community has been working on the parallel programming problem for over 25 years. And frankly, we don’t have much to show for it. I hope the optimists are right and this problem will take care of itself, but this seems like faith-based reasoning to me. This problem is too big and the consequences of failure too dire to leave this to faith. We need to ask the hard questions and find a deliberate and systematic solution. And laying out a path to that solution will be the goal of my blog. I’ve spend over 20 years working in this field. I’ve been beaten down and battered in the battles over parallel-programming languages. As an application programmer, I’ve used all the major classes of parallel programming notations “out there”. I have a unique perspective on this problem and I am just arrogant enough to think I know how to “get it right” this time around. And I hope you, the readers of this blog, will join me in this journey. With your patience and your comments, I’m hoping we can hold a productive dialog and working together, blaze a trail to a future where “sequential programs are rare”. [Editors note: please see Tim's next blog for responses to comments below]
Connect With Us
- gta on What makes a super computer become a super computer?
- gk-edv on The Internet of Things will overtake you only if you let it
- Negin Owliaei on The Internet of Things will overtake you only if you let it
- website packages on Ask the Expert: The Internet of Things
- Jordan on Ask the Expert: The Internet of Things
Tags#IntelR&Dday @idf08 Big Data circuits Cloud Computing Ct CTO energy efficient Future Lab Future Lab Radio HPC IDF IDF2008 IDF 2010 Immersive Connected Experiences innovation Intel Intel Labs Intel Labs Europe Intel Research ISSCC Justin Rattner many core microprocessor mobility multi-core parallel computing parallel programming radio Rattner ray tracing research Research@Intel Research At Intel Day Robotics security silicon photonics software development Stanford technology terascale virtual worlds Wi-Fi WiMAX wireless