Implementing __unimplemented__.m

bug #62362 popped up where a user noticed that several optim pkg functions were being incorrectly reported as unimplemented, or part of statistics, etc. It was mentioned that __unimplemented__.m is fairly out of date and relies on manual changes.

I had started a simple copy/paste of the Octave Forge function list into a spreadsheet and doing some braindead string and sort ops to turn it sometnig that could be cut/paste into __unimplemented__.m just to get it up to date. Noticed there are ~5000 functions and 60+ packages on the page. Many are duplicate/shadowed functions, but many aren’t.

Was wondering:

  • how the included package function lists were originally populated?
  • why just those packages?
  • how did the package lists get ‘belongs to this package but not yet implemented’ functions? is there another list somewhere? similar question for the missing_function function list at the end of the file.
  • are there any thoughts to a more automated way to keep that function up to date?
    • Can a function list be exported from octave forge or from Octave Packages?
    • Could an updated list just be made a part of the package release process, then put in a queue for __unimplemented.m__ inclusion?
2 Likes

The original function only listed Matlab core functions that had still not been implemented. It was later expanded to include the Matlab functions for the most common packages. Note that it does not list all the functions from those package, it only lists functions that are missing for Matlab compatibility.

I’m not sure how the first list was generated but when it was expanded to include the most common packages, that was done with a Perl script followed by some manual curation. With the ever changing internet, I’ll bet that script no longer works. The script also didn’t do well with classes and methods (there weren’t many back then).

The goal of this function is not to list all the stuff in Octave Forge and hint users where the function is. The goal is to catch users interested in a function not yet implemented and direct users on how to help us implement it. In the case of some packages, it reminds them they have to first load the package because that’s a tip for users coming from Matlab but also because we can’t properly check the list of functions until after is loaded.

3 Likes

Ok thanks. That’s informative. I think it should be fairly straightforward to keep a list of pkg functions to remind people to install or load them, since that is essentially already done by the code that generates the octave forge page.

The Matlab curation is quite a bit more cumbersome as you said, since you’re basically relying in the ability to scape a public facing list or set of pages. I think there is a functions list/ reference page, once upon a time I had looked at trying to do a sorted comparison to see what octaves compatibility coverage looked like, but that was 4-5 years ago. It would be good to at least scrub anything from that list that has been implemented, to keep things like in that bug report from confusing users.

Functions like this, which get used by almost every user at some point, getting stale just doesn’t look good for Octave. I would still be interesting in seeing about how to supply at least an up to date pkg function list so people are pointed to the right place for things. It almost seems like more of a burden to curate that list rather than just dump in a complete one. And dupes and shadows don’t matter since octave won’t call the function for anything it finds.

How big can lists like that get before it becomes noticeably slower to use in an m file? Does it matter if the cell array of names is a single column like the one at the end versus being filled to 80 cols like the package lists? @siko1056 , can Octave Packages generate a function list or is it still dependent on Octave forge for that?

2 Likes

That list is not reliable because it is common for maintainers to update the list when new functions are added to the package, and autoload complicates things. Also, the goal is not to hint where any function may be, the goal is to hint for functions that a user coming from Matlab may be expecting to find. If the goal changes to hint a user find where any function may be, then you also have to include all other packages and then packages that are not in Octave Forge too.

It is not that difficult but needs a bit of manual curation, particularly because of classes. Sometimes all methods are listed, other times only the class name, and it’s tricky to get the FQCN from their documentation. This curation is better done by someone that knows about the package. Also, not using one function/class per line also complicates matters because even if you automate printing the list, the manual curation will mess up the pretty print.

Anyway, the code to do it (works as of this morning), is at asorted-tools/list-matlab-package-functions.py at main · carandraug/asorted-tools · GitHub

To be honest, I don’t think we need to keep it that much out of date. I’d argue that listing the most commonly used function from the most commonly used packages would be enough for its goal. Note that listing all the functions in all the Octave Forge package is not the goal.

2 Likes

Interesting discussion :wink:

Currently, there is no such feature of listing package functions.

Looking at /scripts/help/__unimplemented__.m, I doubt that maintaining the list under

function rlist = missing_functions ()

Is really worth the effort. It did not motivate many contributors in the past 10 years to implement a function available in Matlab, but missing in Octave. The simple fact, that a function is not available in Octave does not need a lookup in this list. I vote to remove this list to facilitate the maintenance and improve the lookup speed.

More interesting is the matching with common Octave (Forge) Packages. Frequent Matlab users will appreciate the hint, that common statistic functions can be used by installing a loading the statistics package, for example.

But listing every package function there is a tedious maintenance work without much value in return and will fail (in the current implementation) once two packages implement the same function.

Summarizing:

  • Remove missing_functions list
  • Create a major release task: “Ask package maintainers already listed there to update this list”
  • Scripting possible, maybe not worth the effort.
2 Likes

This was discussed yesterday on the Octave developers meeting. Everyone agreed that the current state of things is not maintainable so we don’t want to be listing every function in Matlab and its package as we are currently trying to do. What was not decided was if the goal here is to direct users on how they can help implement a missing Matlab function (directing them to Missing function) which was the original goal, or if it’s to remind them they need to load or install an OF package.

But I guess now that the two things are not mutually exclusive and I guess we should do whatever is the most useful to the users. It was also suggested to drop the whole reference to it being a missing Matlab function (did we ever got someone to help implementing a missing function because of that error message?).

@jwe suggested to drop the list of functions from the code of __unimplemented__ and instead having one or multiple text files with one function name per line. The function could check those text files when handling the error. Because at that point we are handling an error it should be OK to be a bit slower. In this situation, Octave could be distributed with a base file, and more files could be added later, maybe even during a package installation.

1 Like