Suitability of libgccjit for Octave

I am revisiting the JIT work done in the past. To my knowledge, there was a GSoC project in 2012 with Max Brister to add initial LLVM support but it was difficult to keep it maintained against the always-changing LLVM API. Outdated page here: JIT - Octave

There was some effort in 2014 by David Malcolm to add libgccjit support to Octave but that effort also did not yield much success at the time. GNU Octave - Patches: patch #8395, Work-in-progress port of Octave... [Savannah]

I see that libgccjit has been classified as production quality starting with GCC 11 (released a year ago). It says its C API is mature and its C++ API wraps around the C API. JIT - GCC Wiki

There are some tutorials including how to add JIT to an existing (toy) interpreter Tutorial part 4: Adding JIT-compilation to a toy interpreter — libgccjit 13.0.0 (experimental ) documentation.

I’ll reach out to David Malcolm and see if he can share his thoughts on what it would take to bring JIT functionality to Octave. Personally I like the fact that it is classified as mature – hopefully we won’t have to do battle again with an ever-changing API.

2 Likes

5 posts were split to a new topic: VM for Octave interpreter

Hi - I’m the author and maintainer of libgccjit.

The libgccjit C API is mature - and I’ve kept the API and ABI stable for many releases (merely adding new entrypoints).

However the libgccjit C++ API is not mature - I wrote it as I went along as I was working on that old patch for Octave, and I haven’t touched the API in years. It might be suitable for your needs; I’m not sure.

libgccjit is rather poorly named, in that the two projects using it successfully (Emacs and rustc_codegen_gcc) are both using it to do ahead-of-time compilation. The overhead of gcc’s code generator is so high that unless you’re going to reuse the result, it’s probably overkill for just-in-time compilation.

You might want to look at the emacs compilation project:
gccemacs
since it may share some of the characteristics of Octave (interpreted, but often running the same code over and over again, hence caching the generated code is a big win). It also gains a lot of speed by applying language-specific optimizations on an intermediate representation.

I’m not an expert on Octave, and most of my cycles these days are spent on the GCC static analyzer, so I’ll only be able to provide a little help with Octave.

I should also recommend looking at “Mir”, which a colleague of mine is working on, with a view to optimizing the Ruby interpreter; see his blog posts here:
Vladimir Makarov | Red Hat Developer
(Vladimir maintains GCC’s register allocator, and is far more skilled at code performance than I am)

Hope this is constructive; good luck!
Dave

2 Likes

Thank you @davidmalcolm for your perspective and for the links to alternatives. I did wonder if the C++ API was developed for Octave in your 2014 work, and you confirmed it.

I have some reading to do from the links you sent. I see that Vladimir Makarov describes MIR as language-agnostic JIT support for different dynamic languages, which certainly sounds promising.

I am not familiar with Octave interpreter but I can guess that vector/matrix operations are the most important for overall Octave performance and such high-level operations are the interpreter insns.

Using any JIT for such high-level insns will hardly improve Octave performance as insn operand decoding and interpreter dispatching are tiny parts of of the overall high level insn execution.

Probably you have low-level (scalar) interpreter insns too and JIT can improving their performance.

So imho if a typical Octave program mostly spends time in execution of high level insns, you probably don’t need JIT.

As for MIR, it is written on C and can be used as static or dynamic library. I don’t see obstacles to use this libraries in C++ code. MIR is light-weight JIT (small and having only basic optimizations). LLVM or GCC based JITs can generate 10-20% faster code than MIR based one JIT. For example, MIR JIT does not have and probably will never have autovectorization.

State of the art JIT still can improve vector/matrix insn performance, if you generate specific code for given size vector or matrix (by generating code with memory optimizations specifically tailored for given vector/matrix size). But in this case you should use GCC or LLVM based JIT, not MIR one. That is because memory optimization improvements will be eaten by generating worse code with MIR.

1 Like

@vmakarov : Thank you for your thoughts on MIR. Octave uses a mix of precompiled code (C++ and Fortran) and interpreted code written in the Octave language itself. There is often a 50x difference between them in performance hence the interest in speeding up the interpreted code, if we can.

I believe that @davidmalcolm recommended MIR based on the faster response time for an interpreted language than possible with libgccjit which he stated was used more for ahead-of-time compilation in spite of the JIT name.

It is too early to tell which approach Octave could adopt to accelerate its code but based on your response I’ll keep MIR in the mix because it’s actively developed by you and the API is largely compatible. There are also VM approaches being examined. It will become clear in the upcoming months.

Links for anyone studying MIR: