Buildbot having trouble building MXE Octave

@jwe’s buildbots seem to have trouble building some jobs for MXE Octave.

The mxe-native-all-on-debian builder fails on this command:

PKG_CONFIG_PATH='' LD_LIBRARY_PATH='/scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/usr/lib:/scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/usr/lib64' PATH='/var/lib/buildbot/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games' ../configure --prefix=/scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/usr && \

with this error:

checking whether the C compiler works... no
configure: error: in `/scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/octave-hg-repo/.build':
configure: error: C compiler cannot create executables
See `config.log' for more details
make: *** [/scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/hg-octave-dist.mk:34: hg-octave-dist] Error 77

IIUC, that builder should be using the compiler that was built as part of the build process. The PATH variable pointing to paths on the worker (instead of paths inside the MXE Octave build tree) looks fishy to me. Maybe the compiler installed on the build system is directed to use the standard libraries that were built for MXE’s compiler?
But I might be misunderstanding what is supposed to be happening.

The w*-release and w*-stable builders are failing when they try to compile qtbase. I’ve no idea why though. ISTR that we saw these kinds of errors with gcc 11 before. But we adopted a patch that is supposed to fix them (mxe-octave: 09d3533acacf src/qtbase-2-gcc11.patch).
Is that patch not used for some reason?

After looking a bit more closely, I realized that that patch was only on the default branch.
I grafted these changes to the release branch here:
mxe-octave: 6d67c9994aed
mxe-octave: 72f9d9dab23f

Were the workers updated recently and the installed gcc is now version 11?

Still not sure why native-all is failing.

I did upgrade recently. I’ll see about getting more info on the mxe-native-all failure.

1 Like

The config.log file for the failed build of Octave from the hg archive showed

configure:7510: checking whether the C compiler works
configure:7532: gcc    conftest.c  >&5
/usr/bin/ld: /scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/usr/lib/libctf.so.0: version `LIBCTF_1.1' not found (required by /usr/bin/ld)
collect2: error: ld returned 1 exit status
configure:7536: $? = 1
configure:7576: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "GNU Octave"
| #define PACKAGE_TARNAME "octave"
| #define PACKAGE_VERSION "7.0.0"
| #define PACKAGE_STRING "GNU Octave 7.0.0"
| #define PACKAGE_BUGREPORT "https://octave.org/bugs.html"
| #define PACKAGE_URL "https://www.gnu.org/software/octave/"
| #define PACKAGE "octave"
| #define VERSION "7.0.0"
| #define OCTAVE_SOURCE 1
| /* end confdefs.h.  */
| 
| int
| main (void)
| {
| 
|   ;
|   return 0;
| }
configure:7581: error: in `/scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/octave-hg-repo/.build':
configure:7583: error: C compiler cannot create executables
See `config.log' for more details

Info in the bug report

https://sourceware.org/bugzilla/show_bug.cgi?id=27250

gave me a clue that the problem might be fixed by a more recent version of binutils.

Updating the mxe-octave version of binutils to 2.37 seems to have fixed the problem.

I wonder why it is using /usr/bin/ld for linking. Shouldn’t it be using /scratch/buildbot/workers/jwe-debian-x86_64-5/mxe-native-all-on-debian/src/usr/bin/ld instead?

I am not sure if that is related, but I am trying to use lld linker (ld.lld) for clang builds.
So I set LD=ld.lld and LDFLAGS="-fuse-ld=lld -Wl,–threads=4". Configure passes OK,
yet during compile I see:

 CXXLD    libinterp/dldfcn/__glpk__.la
clang-13: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
clang-13: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]clang-13: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
clang-13: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]

clang-13: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
clang-13: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
config.status: creating build-aux/subst-script-vals.sh-tmp
/usr/bin/ld: unrecognized option '--threads=4'
/usr/bin/ld: use the --help option for usage information
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
/usr/bin/ld: unrecognized option '--threads=4'

Why /usr/bin/ld is invoked?

p.s. I can set /usr/bin/ld → /usr/bin/ld.lld and then it finishes compile

The log file http://buildbot.octave.org:8010/#/builders/20/builds/1045/steps/7/logs/stdio shows

PATH=/var/lib/buildbot/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games

That doesn’t include the directory where we are installing build tools, so the version of GCC built earlier in this compile step won’t be found.

It makes sense to me to at least begin building the build tools with the compiler that is installed on the system since we can’t do otherwise. But after we build our own GCC, we should be using that (at least) for the remaining packages, right? So I guess we need some changes to the build rules for the mxe-native-all configuration. Maybe even more changes if we want to bootstrap using the system compiler and then rebuild everything including the build tools with the new compiler. Either way, fixing this problem is a low priority for me.

The cross-builds for Windows are failing now.
The updated binutils version might be incompatible with the toolchain built by MXE Octave.

I guess the underlying issue is that LD_LIBRARY_PATH points to the MXE environment but PATH points to the tools on the build system.
We should probably get that sorted out (either by consistently using tools and libraries from the build system or those that MXE Octave just built).

For the time being, we should probably revert to the older version of binutils.

I reverted to the previous binutils version here:
mxe-octave: 768e44dcd00c

And followed up with what was probably fixing a typo here:
mxe-octave: 00a24f9b9cae

Let’s see if this will make the buildbots happy again.

There is already a comment in hg-octave-dist.mk that explains why it is difficult to build with our own toolchain.
When I tried to use our toolchain instead, the build failed on creating the images for the manual. We’d need to build (at least) Qt as part of the build toolchain to fix that IIUC.

Not sure if it is related either.
IIUC, libtool will use the ld it found during configure (by calling $CC -print-progname=ld?).
You could check the libtool script that is created by configure which that is. It’s set in line 310 of that script for me:

# The linker used to build libraries.
LD="/usr/bin/ld -m elf_x86_64"

I’m not sure if I correctly follow what LT_PATH_LD is doing. But it might be possible to override whatever it would select by setting lt_cv_path_LD=/usr/bin/ld.lld when configuring…

I started another topic

(with some extra details).