I am revisiting the JIT work done in the past. To my knowledge, there was a GSoC project in 2012 with Max Brister to add initial LLVM support but it was difficult to keep it maintained against the always-changing LLVM API. Outdated page here: JIT - Octave
There was some effort in 2014 by David Malcolm to add libgccjit support to Octave but that effort also did not yield much success at the time. GNU Octave - Patches: patch #8395, Work-in-progress port of Octave... [Savannah]
I see that libgccjit has been classified as production quality starting with GCC 11 (released a year ago). It says its C API is mature and its C++ API wraps around the C API. JIT - GCC Wiki
There are some tutorials including how to add JIT to an existing (toy) interpreter Tutorial part 4: Adding JIT-compilation to a toy interpreter — libgccjit 13.0.0 (experimental ) documentation.
I’ll reach out to David Malcolm and see if he can share his thoughts on what it would take to bring JIT functionality to Octave. Personally I like the fact that it is classified as mature – hopefully we won’t have to do battle again with an ever-changing API.
5 posts were split to a new topic: VM for Octave interpreter
Hi - I’m the author and maintainer of libgccjit.
The libgccjit C API is mature - and I’ve kept the API and ABI stable for many releases (merely adding new entrypoints).
However the libgccjit C++ API is not mature - I wrote it as I went along as I was working on that old patch for Octave, and I haven’t touched the API in years. It might be suitable for your needs; I’m not sure.
libgccjit is rather poorly named, in that the two projects using it successfully (Emacs and rustc_codegen_gcc) are both using it to do ahead-of-time compilation. The overhead of gcc’s code generator is so high that unless you’re going to reuse the result, it’s probably overkill for just-in-time compilation.
You might want to look at the emacs compilation project:
since it may share some of the characteristics of Octave (interpreted, but often running the same code over and over again, hence caching the generated code is a big win). It also gains a lot of speed by applying language-specific optimizations on an intermediate representation.
I’m not an expert on Octave, and most of my cycles these days are spent on the GCC static analyzer, so I’ll only be able to provide a little help with Octave.
I should also recommend looking at “Mir”, which a colleague of mine is working on, with a view to optimizing the Ruby interpreter; see his blog posts here:
Vladimir Makarov | Red Hat Developer
(Vladimir maintains GCC’s register allocator, and is far more skilled at code performance than I am)
Hope this is constructive; good luck!
Thank you @davidmalcolm for your perspective and for the links to alternatives. I did wonder if the C++ API was developed for Octave in your 2014 work, and you confirmed it.
I have some reading to do from the links you sent. I see that Vladimir Makarov describes MIR as language-agnostic JIT support for different dynamic languages, which certainly sounds promising.
I am not familiar with Octave interpreter but I can guess that vector/matrix operations are the most important for overall Octave performance and such high-level operations are the interpreter insns.
Using any JIT for such high-level insns will hardly improve Octave performance as insn operand decoding and interpreter dispatching are tiny parts of of the overall high level insn execution.
Probably you have low-level (scalar) interpreter insns too and JIT can improving their performance.
So imho if a typical Octave program mostly spends time in execution of high level insns, you probably don’t need JIT.
As for MIR, it is written on C and can be used as static or dynamic library. I don’t see obstacles to use this libraries in C++ code. MIR is light-weight JIT (small and having only basic optimizations). LLVM or GCC based JITs can generate 10-20% faster code than MIR based one JIT. For example, MIR JIT does not have and probably will never have autovectorization.
State of the art JIT still can improve vector/matrix insn performance, if you generate specific code for given size vector or matrix (by generating code with memory optimizations specifically tailored for given vector/matrix size). But in this case you should use GCC or LLVM based JIT, not MIR one. That is because memory optimization improvements will be eaten by generating worse code with MIR.