Matt3 pointed us to this helpful document by Agner Fog, “Optimizing software in C++ An optimization guide for Windows, Linux and Mac platforms.”
More here. Enjoy!
[...] A rather detailed manual on software optimization (C++) and other optimization resources; via Andrew at Statistical Modeling, Causal inference, and Social Science. [...]
Nice. I like the low level and system specific information. Personally I’ve found “Techniques in scientific C++” (http://desktop.jncasr.ac.in/uploaded/ansumali/ScientificC++(1).pdf) to be very helpful in the design and architecture of my runtime sensitive applications.
That’s a doc by Veldhuizen, the inventor of expression templates! The book on C++ templates by Vandervoorde and Josuttis covers much of the same material.
This all falls under what Agner calls methods for eggheads that should be avoided. While I’ll admit to being an egghead, I think avoiding template metaprogramming is a big mistake for performance-critical code. We wouldn’t even be able to do auto-dif without it, much less write efficient vectorized distribution functions and gradients. And Eigen leans on it heavily to optimize their matrix library along lines originally introduced by Veldhuizen. This makes C++ matrix libs faster than their C counterparts and analogous to what Fortran compilers can achieve.
@Bob Carpenter: I’m very new to fortran, so I’m not sure what you mean by “This makes C++ matrix libs faster than their C counterparts and analogous to what Fortran compilers can achieve.” Do you mean in terms of getting performance like the element-wise operations that you’d have to write in equivalent Fortran code? But that isn’t related to what the Fortran compiler does, just limitations/features of the language.
Am I missing something?
The short story is that Fortran compilers can do a lot of static optimization because the language is so restricted. So you could argue that’s a restriction of the language that makes it so fast. And C++ is introducing declarations and programming techniques to do some of the same things.
For background, see Veldhuizen’s 1997 piece:
There’s also a nice Stack Overflow thread here:
You can get the same sort of advantage in static optimization versus C++ for functional languages like ML.
I missed Agner’s “egghead” assessment. It seems odd to view meta-programming as problematic with regard to code maintenance in the same document that covers processor specific vectorization.
The URL in Ian’s comment is split. Try http://bit.ly/MClsHV