Merging branches for Octave 8

IIUC, we were planning to merge the default branch of the Octave repository to the stable branch early November.
When would be a good time for this?

In the past, this corresponded to the approximate feature freeze for the new version. Would anyone like to get a feature in before that will happen?

Usually, we bumped the gnulib version for a major release. Should we do that before the merge?

What about MXE Octave? Imho, merging the branches in that repository doesn’t need to coincide with the merge date of the Octave repository. It might even be a good idea to merge the release branch of MXE Octave with the default branch several times before the release.
Imho, it might be best to wait at least a few days until after the merge in the Octave repository. That way, we could check if anything broke on the default branch without having to guess whether it is in Octave or in a dependency.

What do you think?

I would like to polish and push the Matlab compatible mean function, related to the latest statistics package release. Sorry, time for Octave was short recently due to the server crash :sweat_smile: But I do not want to be a blocker, I will get on this project this weekend.

2 Likes

I bumped the gnulib revision here:
octave: e98fb9b4be86 (gnu.org)

It looks like they split the bootstrap script into multiple parts. I hope I added all of them correctly. The buildbots and other CI will probably tell us if something is still missing.

2 Likes

Just pushed a changeset for the mean compatibility: octave: 21962c678648 Nothing disruptive is waiting from my end before the Octave 8 code freeze.

In general @pr0m1th3as eventually updates implementations regarding missing all- and nan-flag handling and proper vector dimension handling, see GNU Octave - Bugs: bug #58089, [Feature Request] sum along... [Savannah]. Would it be too disruptive to apply those patches (if they get ready by the time) to Octave 8 already. Those Matlab incompatibilities are long outstanding issues and Matlab codes more and more use those features.

Seems like we are ready to merge default to stable now that gnulib is updated. I’d vote for that.

@jwe: Who usually does the merge of default to stable? Do we have it documented in case of the “bus problem”?

1 Like

To maybe help finding this again in the future. Some information about the new split bootstrap script on the gnulib mailing list:
Re: Instructions on new bootstrap? (gnu.org)

Paul Smith wrote:
Are there instructions on how the new model of bootstrap is supposed to
be used somewhere? I can’t quite figure it out.

I pulled in the latest gnulib/build-aux/bootstrap and ran it, then when
it was complete git shows:

Changes not staged for commit:
modified: bootstrap

Untracked files:
autogen.sh
autopull.sh
bootstrap-funclib.sh

I would recommend to

  • commit all these 4 files in your package’s repository,
  • document in the ‘HACKING’ or ‘README-hacking’ file that the developer
    has more control over the bootstrap process by running
    ./autopull.sh
    ./autogen.sh
    in two steps, rather than ‘./bootstrap’ all in one step.
    autopull.sh is a script for fetching auxiliary files that are omitted from
    the version control repository.
    autogen.sh is a script for regenerating all autogeneratable files. This
    script does not make network accesses nor destructive git operations.

Looking at bootstrap now it was replaced with gnulib/top/bootstrap and
these extra files were added. It would be nice if, at least, I could
have these files put somewhere besides my root directory.

autopull.sh and autogen.sh are intentionally in the root directory, because
the developer runs them every day.

bootstrap-funclib.sh could technically be moved to a subdirectory, but this
is a bit hairy to implement (because it requires parsing configure.ac in order
to find the $build_aux directory).

So I went and looked at the build-aux/bootstrap and it says:

  # Set this to true in bootstrap.conf to enable --bootstrap-sync by
  # default.
  bootstrap_sync=false
    ...
    --bootstrap-sync)
      bootstrap_sync=true;;
    ...

which seems to imply that unless I add that flag or set bootstrap_sync
in my bootstrap.conf file (which I did not), the sync won’t happen.

In your case, the sync already happened. So, either you must have had

  bootstrap_sync=true

in your bootstrap.conf, or you passed the option --bootstrap-sync manually.

Bruno

If I understand correctly, we could consider including autogen.sh (and a couple of other files) in the tarball to allow users to, e.g., update to a newer version of autotools. That is if we want them to do that.
Do we?

Hmm, seems awfully complex now. Most users don’t want to build from source. But if they do, then they just want to be able to run configure; make; make install.

If this is enabling special options for developers then maybe we should just point them at the Mercurial archive rather than the tarball.

I’m just trying to avoid work here, but am not opposed to making changes if there are some benefits.

For recent releases, I’ve merged default to stable.

The merge should be trivial because you just want to replace everything on stable with default. There shouldn’t be any conflicts. There are some fairly detailed instructions about what to do (including updating version numbers after the merge) in the etc/HACKING.md file.

I haven’t really followed any of the gnulib discussion, but if they are trying to solve the problem of updating to newer gnulib sources when building from a tarball distribution, then that would be a nice feature to have. There have been a few times when OS distributions change and released versions of Octave won’t build because of problems with gnulib source files. In those situations it is very difficult to offer a fix. Even an experienced user building from source will have a lot of trouble because not all the required bits of gnulib are in the Octave tarball. Until now there has been no good way to help someone work around the problem. So I would be in favor of adding those files to our repo and releases if that helps solve the problem I’m describing here.

It looks like some issues with merging ?!
See buildbots for details. E.g.:

make[2]: *** No rule to make target 'etc/NEWS.9.md', needed by 'NEWS'.  Stop.
make[2]: *** Waiting for unfinished jobs....
make[3]: Leaving directory '/home/buildbotu/fc25-x86_64/gcc-fedora/build'
Updating ../src/doc/liboctave/version-liboctave.texi
mv -f libgui/src/.deps/libgui_src_la-octave-qobject.Tpo libgui/src/.deps/libgui_src_la-octave-qobject.Plo
make[2]: Leaving directory '/home/buildbotu/fc25-x86_64/gcc-fedora/build'
make[1]: *** [Makefile:27647: all-recursive] Error 1
make[1]: Leaving directory '/home/buildbotu/fc25-x86_64/gcc-fedora/build'
make: *** [Makefile:11048: all] Error 2

P.S.: Looks like the change by @rik

maint: Bump version numbers for pre-release (see etc/HACKING.md)(an hour ago)
Author	Rik <rik@octave.org>
Date	November 15, 2022 4:18 PM (an hour ago)
Repository	https://hg.savannah.gnu.org/hgweb/octave
Branch	stable
Revision	f990f14d4b0c3bba0622beb4805cb6a509afd8d7
Properties	
Comment
maint: Bump version numbers for pre-release (see etc/HACKING.md)

* configure.ac (AC_INIT): Bump version.
(OCTAVE_PATCH_VERSION): Set to 1 for stabilization period before the
version 8.1.0 release.
(OCTAVE_RELEASE_DATE): Update.
Changed files
configure.ac

I created an empty file with touch etc/NEWS.9.md and then it built after that.

EDIT: looks like the file is in the repo now so no need to create an empty file.

Okay, I followed the instructions and it basically seems to have worked. Apologies to those who did hg pull in the midst of my series of changesets and got a code tree that wouldn’t build.

I did find that the documentation was a bit off so I updated HACKING.md in this changeset: octave: 38139b13192c.

@jwe: I started removing the features that were deprecated in Octave 7 and scheduled for removal in Octave 9. This was easy for m-files (see octave: c17155495497).

However, I’m not sure about these items mentioned in NEWS.7

- Operators

  Operator | Replacement | Description
  ---------|-------------|------------
  `**`     | `^`         | Matrix exponent
  `.**`    | `.^`        | Element-by-element exponent
  `.+`     | `+`         | Element-by-element addition
  `.-`     | `-`         | Element-by-element subtraction

- Interpreter

    * The use of `'...'` for line continuations *inside* double-quoted
    strings has been deprecated.  Use `'\'` for line continuations
    inside strings instead.

    * The use of `'\'` as a line continuation *outside* of double-quoted
    strings has been deprecated.  Use `'...'` for line continuations
    instead.

    * Any trailing whitespace after a `'\'` line continuation has been
    deprecated.  Delete unnecessary trailing whitespace.

Could you take a look at these since I think you will be able to do the removal quickly?

Also, using grep I see over 900 lines [depr2.list (85.0 KB)] of the form

OCTAVE_DEPRECATED (7, ...

example:
liboctave/util/lo-utils.h:218:OCTAVE_DEPRECATED (7, "use 'octave::fgetl' instead")

Is it correct that to remove these we need to go to every code segment where these are located and delete?

2 Likes

I bumped the version numbers in MXE Octave here:
mxe-octave: 77267b0c5bc8

1 Like

I was getting a big discrepancy with my grep results. It turns out that the lines in that file with builtin-defun-decls.h are all in the build directory, not the source directory. Could you confirm? Removing those, there are 401 lines in these 35 files:

     83 libinterp/corefcn/xpow.h
     57 libinterp/corefcn/xdiv.h
     52 libinterp/corefcn/graphics.in.h
     32 liboctave/numeric/oct-norm.h
     28 libinterp/corefcn/sparse-xdiv.h
     21 liboctave/util/lo-utils.h
     18 liboctave/numeric/oct-convn.h
     15 libinterp/octave-value/ov.h
     14 libinterp/corefcn/sparse-xpow.h
     12 liboctave/array/Range.h
     10 libinterp/corefcn/variables.h
      7 libinterp/corefcn/error.h
      6 libinterp/corefcn/c-file-ptr-stream.h
      5 libinterp/corefcn/hook-fcn.h
      5 libinterp/parse-tree/bp-table.h
      4 libinterp/corefcn/procstream.h
      4 libinterp/corefcn/xnorm.h
      3 libinterp/corefcn/ls-ascii-helper.h
      3 libinterp/corefcn/oct-iostrm.h
      3 libinterp/corefcn/oct-strstrm.h
      2 libinterp/corefcn/ls-utils.h
      2 libinterp/corefcn/oct-prcstrm.h
      2 libinterp/corefcn/oct-stdstrm.h
      2 liboctave/util/lo-ieee.h
      1 libinterp/corefcn/data.h
      1 libinterp/corefcn/load-save.h
      1 libinterp/corefcn/oct-fstrm.h
      1 libinterp/corefcn/oct-hdf5-types.h
      1 libinterp/corefcn/oct-procbuf.h
      1 libinterp/mk-builtins.pl
      1 libinterp/parse-tree/oct-lvalue.h
      1 liboctave/array/Array.h
      1 liboctave/array/DiagArray2.h
      1 liboctave/numeric/CollocWt.h
      1 liboctave/numeric/oct-spparms.h

EDIT: I made a first pass through the files above and removed as many instances as possible. Will wait for CI to find any errors.

(EDIT from @rik (11/16/22): Thank you @arungiridhar for taking the first pass at removing all the old code!)

The only ones still remaining as of hg id db8735ee84da are the following which cause build errors when removed.

libinterp/corefcn/error.h:339:    OCTAVE_DEPRECATED (7, "second argument is no longer accepted")

liboctave/array/Range.h:403:  OCTAVE_DEPRECATED (7, "use the 'octave::range<double>' class instead")
liboctave/array/Range.h:413:  OCTAVE_DEPRECATED (7, "use the 'octave::range<double>' class instead")
liboctave/array/Range.h:425:  OCTAVE_DEPRECATED (7, "use the 'octave::range<double>' class instead")
liboctave/array/Range.h:433:  OCTAVE_DEPRECATED (7, "use the 'octave::range<double>' class instead")
liboctave/array/Range.h:447:  OCTAVE_DEPRECATED (7, "use the 'octave::range<double>' class instead")
2 Likes

I’ll look at the Range class. It may need to continue to exist internally so that we can load Range objects from files saved with older versions of Octave.

Yes, you need to remove all the declarations and corresponding definitions if the functions are not defined inline in the header files.

I’ll look at the lexer and parser changes that are needed to remove the obsolete operators.

Don’t remove Range or octave_legacy_range yet. Some parts of that may need to continue to exist internally to support loading Range objects from files saved by older versions of Octave.

Should this part of octave: db8735ee84da (gnu.org) be reverted? Or should the corresponding definitions be removed?

--- a/liboctave/array/Range.h
+++ b/liboctave/array/Range.h
@@ -562,27 +562,4 @@
   { }
 };
 
-#if defined (OCTAVE_PROVIDE_DEPRECATED_SYMBOLS)
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator - (const Range& r);
-
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator + (double x, const Range& r);
-
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator + (const Range& r, double x);
-
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator - (double x, const Range& r);
-
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator - (const Range& r, double x);
-
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator * (double x, const Range& r);
-
-OCTAVE_DEPRECATED (7, "arithmetic operations on Range objects are unreliable")
-extern OCTAVE_API Range operator * (const Range& r, double x);
 #endif
-
-#endif

Finding deprecated code is kind of fun. In libinterp/module.mk I found

## FIXME: The following two variables are deprecated and should be removed
##        in Octave version 3.12.
DLL_CDEFS = @OCTINTERP_DLL_DEFS@
DLL_CXXDEFS = @OCTINTERP_DLL_DEFS@

Octave 3.12!! That is ancient.

1 Like

I pushed the following changeset to remove the unnecessary parts of the Range class:

http://hg.savannah.gnu.org/hgweb/octave/rev/0f968958d7d5

I will look at doing more to hide Range and octave_legacy_range objects in a future changeset.

I just finished compiling

(0f968958d7d5)

And got this during make install

GEN liboctave/external/mk-f77-def.sh
config.status: creating liboctave/external/mk-f77-def.sh-tmp
config.status: executing liboctave/external/mk-f77-def.sh commands
GEN libinterp/DOCSTRINGS
MAKEINFO doc/liboctave/liboctave.info
range.texi:25: unmatched `@end deftypefn’
make[2]: *** [Makefile:25898: doc/liboctave/liboctave.info] Error 1
make[2]: Leaving directory ‘/home/doug/octaves-clone’

But it passed all the check tests.