I'd say Python's runtime only has a small fixed startup cost that prevent it from beating natively-compiled tools in no-op rebuilds (when no source file has changed). Some tools like tup or ninja reach insanely low times that are below the startup time of Python itself. But at such low times, it doesn't matter anymore whether it's 10ms or 100ms.
The cmake benchmarks used the make backend on linux and cygwin, and the vcbuild, msbuild and mscv-ide backend on windows/msvc.
The ninja backend is indeed faster than the make one. An update to these benchmarks is needed to take it into account.
PS: I fixed the broken links in that benchmark page.
I take note that I should put a date on the page!
I actually ran again some of these benchmarks again over the years (last time about one years ago), on the same hardware and updated software, and results didn't change significantly.
FWIW, at the time these results were first published, I had my working dir on ext2 because it had higher performance than ext3.
Hi, I'm the author of these benchmarks, and yes, they are getting old. One of Scons' maintainers kindly asked me a few days ago to try again with current Scons version, so I've started to update and run these benchmarks again with current software and hardware (SSD).
It takes some time to do it right (blame API breakage!) and collect all the results, so I won't be able to publish them before at least a month or so.
I will include the meson+ninja and cmake+ninja combinations, and the new CBS tool (an experimental build system from Qt).
