I recently had the opportunity to revisit some old Octave code most of whose runtime was inside a for-loop with an if-else tree inside it, with very few library function calls and no opportunity for vectorization or broadcasting. hilb_bench.m (3.1 KB)
I used it as a benchmark on different versions of Octave and got the following results:
- Octave 5.2.0: 281 ± 3 seconds.
- Octave 7.1.0: 243 ± 3 seconds.
- Octave 8.0.0: 234 ± 3 seconds.
Anyway, I thought it was a small but nice demo of continued improvement over time for the part of Octave that does not benefit from compiled C++ functions, optimized libraries like OpenBlas, or techniques like vectorization or broadcasting. If we get JIT working again in future, this sort of Octave code would be the first to benefit.
(I would have tested more versions but old versions of Octave no longer build easily for me, and version 6.x does not build because of the Sundials library API change. All versions were built afresh.)
BTW, if anyone is interested in creating a proper benchmark suite to test for performance regressions, I’m happy to contribute.