Octave Packages: the new Octave Forge

To make the publication of Octave packages less bureaucratic and more fun again, I propose the

Sources at GitHub - gnu-octave/packages: An index for GNU Octave packages.

Some highlights:

  • All Octave Forge packages in their latest release, and more, are indexed
    • No need to move away from Octave Forge with your package
  • The package maintainer is responsible for his package to work
  • Host your code / releases anywhere (SourceForge, GitHub, GitLab, own public server, …)
  • Easy, less bureaucratic addition of packages and releases
    • Manage your package metadata in a single file you can edit online
    • Let your users install the latest version of your package directly from the repository for testing (no release necessary, see the “dev” entry of the example package)
  • Easy web interface to find packages / code
  • Minimal automated quality checks
  • Optional integrity checks by sha256 checksums.
  • Fully community driven.
  • The index itself is a “static” website.
    • Automatically built by GitHub pages
    • The index pages can be hosted (mirrored, duplicated, etc.) anywhere

:arrow_right: If there are no objections, I would like to present this index alongside with Octave Forge on the main Octave website.

Octave Packages is just an index. No changes to the Octave package format are required, thus all packages listed there can be installed using the default pkg tool of Octave.

A further project is to make package installations fun again extending @apjanke’s GitHub - apjanke/octave-packajoozle: A just-for-fun reworking of GNU Octave's `pkg` tool

Comments are pretty welcome :point_down: :slightly_smiling_face: :point_down:

Could someone clarify the situation about package collections and how they are described in the documentation? The following seems to be a little contradictory or out of date:

  • The Octave wiki main page’s package section (Octave) describes Octave Forge as a “former collection of (packages)” implying that it is obsolete or no longer current, but the Forge page https://octave.sourceforge.io/ itself makes no reference to it not being current or superseded by something else.

  • The same Octave wiki main page points to the Github index of packages (GNU Octave - Packages), which calls itself a superset of Octave Forge (packages/README.md at main · gnu-octave/packages · GitHub), but there is nothing in "help pkg" that mentions the Github collection at all.

  • The "pkg install" command accepts "-forge" as an option but there is no equivalent for "-github" or similar, so currently one cannot do "pkg install -github coder" or "pkg install -forge coder" (for example) with the same ease as "pkg install -forge statistics". This requires a manual download from Github first before installation – is this a conscious design choice or should we make it one-step like the Forge packages?

What is a good way to proceed?

Thanks for picking up this topic again, I updated my first comment in this topic to the current situation.

In the last year the work on Octave Packages has proceeded a lot and I think this package repository / index is more than ready for production use. I maintain Octave Forge in parallel to keep Octave package development alive while the new pkg-tool is not ready.

However, Octave Forge is kind of dead now and cannot grow anywhere without more manpower. Most of the previous maintainers left the project and Octave Forge uses a web scraping approach for resolving packages :bomb: :scream: therefore I do not dare to make significant layout changes on Octave Forge, as each slightest changes might affect users of older Octave versions no longer being able to install packages :sweat: Thus all we can do today is maintaining what exists on Octave Forge, until somebody comes up with a solution to this dilemma.

What is missing to finally say goodbye to Octave Forge is the package manager side, a new pkg-tool, which is 90% :innocent: complete:

The package manager was/is a more difficult task than I initially thought, mostly because it must work with older Octave versions and other decisions made for Octave package development. I also made lots of progress here. In a near future the pkg usage will look on Octave 4.2.0 to 7.1.0 like this:

image

The installation of the new pkg itself will be as easy as:

pkg install https://github.com/gnu-octave/pkg/archive/refs/heads/main.tar.gz
pkg load pkg
pkg config -add-startup-hook

Then you can enjoy the new pkg that fully makes use of Octave Packages already today. To get rid of the pre-release:

pkg config -remove-startup-hooks
pkg unload pkg
pkg uninstall pkg

I write a better advertising article once the new pkg-tool is ready for more testing :slightly_smiling_face:

1 Like

That is excellent news, @siko1056! I saw the pkg tool in the Github list but hadn’t realized until now how that would fit in. Very happy to see your intended use for that command and looking forward to using it.

Is it ready to use alongside the existing pkg command in Octave now or will that cause breakage?

Sorry for the late reply. I still consider the new pkg-tool as unstable now. The major workflow ([un]installing, [un]loading packages) should work, but I still encounter edgy cases.

The Octave package format itself is left untouched and the new pkg-tool should seamlessly integrate into existing package installations once ready :slight_smile: Please report any bugs while testing.

1 Like

I can see in the www.octave.org page, in Octave Packages section, that there are two lings, one pointing to the Octave Forge page, and the other to a list of packages which is the same of the Octave Forge. As a maintainer of two packages, why the list is duplicated? Point the two pages to the same location or must I upload twice the new package versions?

@jgpallero The answer clarifying the difference between both platforms is in this thread, where I moved your new topic to.

  1. If you maintain a package on Octave Forge, please make your updates there as always (updates are very very welcome :slightly_smiling_face: )
    Octave Packages, will incorporate all changes done at Octave Forge. No need for double effort from your side.

  2. New packages are not accepted at Octave Forge. Please use Octave Packages.

For future consideration - if the -forge option no longer necessarily points to Octave Forge, would it make sense to introduce an alias -download or something similarly generic? As an alias backward compatibility would persist, and the option would make a bit more sense moving forward.

2 Likes

Handling -forge by Octave Packages in a fully backward compatible way (I tested until 4.2.0) could be achieved quite simple. A solution was really overdue, potential package maintainers got dragged away as Octave Forge was dead and Octave Packages not in charge for simple package installation. This gap is now closed :slightly_smiling_face:

As this flag has been introduced and has become the interface, it must be dealt with for years. Keeping this in mind, I think introducing another flag out of reflex is another potential design problem that future Octave developers have to deal with and it requires more and more tricks to deal with all those legacy options in a consistent way.

In my package tool under development, any of those flags have proven to give no additional information (despite checking the user’s typing skills and memory for parameter details :sweat_smile: ) and could easily be ignored if the input to the pkg tool is properly analyzed (local file, url, or package name) and processed:

Thus I vote against quick shots that have to be dealt with several years and hope someone or I find more time for a proper pkg redesign.

1 Like