Goals for the next release

As discussed during the developer meeting on 2020/10/27, I created a wiki entry to list goals for the next release and added some of my own priority items.

You can find the wiki section here:

We can move it to a separate page or category if it would work better that way.

Also, as noted in that entry, the section is intended for active developers, not a place to dump wishlist items or feature requests. Please create a subsection for your ideas/prioirities.

I understand this shouldn’t be a wishlist and I hope I don’t open Pandora’s box by this: But a project I would like to see tackled is the cross-platform text widget as a replacement for the terminal widgets we currently use.
I wouldn’t know where to even start with this project. So my role would at best be to give some support once the major parts are done.

Yes, replacing the current terminal widget is something that I’d also like to see sometime soon, I just don’t know how much time I have to work on it, but maybe if I can sketch out how it could work with simple implementation it would provide a starting point for others?

+1 for the new terminal widget. I can assist with the gui part of the new implementation.

That would very probably help.
It might also help if it were possible to break this down into smaller sub-projects.
Like I already wrote, I don’t know where to even start looking. So I can’t estimate what these sub-projects could be.

I was interested in something slightly different when I proposed this. The Projects page on the wiki is good for just that, potential ideas to implement. Instead, I wanted to explore what major themes we might pursue for the Octave 7 release. After a theme (or two or three) was picked then projects would be selected to support that theme. Some possible organizing ideas that I had:

  • Improve graphics
    ** better graphic performance (plots of many polygons, such as through patch(), are much slower than OpenGl is capable of)
    ** hide the fact that OpenGl uses single precision variables (this has been on the wishlist forever)
    ** allow for very large datasets (right now we pass all points to OpenGl for ploting which can be slow for a million points)
    ** have an architecture review by an OpenGl expert (do we have enough in Octave’s fund to pay for a 4-hour review). I think we must be doing some things inneficiently.

  • Improve performance
    ** JIT compiler (this is a big one, but it has to happen sometime)
    ** overall performance has slowed considerably from earlier versions of Octave (3.4.X seems to have been the best). Use profiling and other tools to eliminate some of the slowdowns
    ** performance for certain operations like struct field lookups should be O(log2(N), but is not. Given that we are mapping on to C++ data structures which have this behavior it must be something that Octave is doing that prevents us from reaching full performance.

  • Code Base and technical debt
    ** complete BIST regression suite. It is easier to speed through the design/build/test cycle if we know we have a complete regression suite which will catch any bugs.
    ** create a performance regression test suite. Right now we can’t code with confidence that we aren’t solving a functional problem but also creating a performance one.
    ** reduce unnecessary functions in core so there is less to maintain. I’m thinking of a number of very UNIX-y sysadmin functions which we basically get for free, but are still cluttering and not really germane to numerical analysis. I would rather have people use a language like Perl if they want to do sysadmin. The functions could be shunted off to a new package.

  • Matlab compatibility
    ** strings class (bite the bullet and implement this and, probably, get rid of Octave’s very UNIX-y way of understanding single and double quotes)
    ** “argument” input validation blocks
    ** others? Seems like there are tall arrays, dataframes, etc.

  • Octave-only functionality
    ** built-in hash operator. I would like to see an O(1) search function incorporated in to the Octave language, similar to hashes in Perl or dictionaries in Python. This would be an extension over what Matlab provides, and could be used to implement containers.Map more easily as well as data frames.
    ** new efficient algorithm for eigenvectors. See https://www.quantamagazine.org/neutrinos-lead-to-unexpected-discovery-in-basic-math-20191113. It turns out the mathematics are sound and have been re-discovered from time to time. It probably beats what is in LAPACK.

These are a few themes, there could be more. For example, I haven’t even mentioned UI/UX because it isn’t a big deal for me, but I know others might want to make that a focus.

2 Likes

I concur.

It is no secret that Octave’s OpengGL rendering engine is written using the old school direct pipeline. This part of the API is at least discouraged and often deprecated. So yes, it would be great if we could hire someone to help us switch to modern OpenGL (glsl shaders, vertex buffer objects, …).

Fascinating stuff.
Reading up in the references of that article, I get the impression (but I may be wrong) that a performance gain is to be expected for (some types of) band matrices, not so much for general “random” matrices. But large “full” matrices are better avoided anyway. so yeah there’s some “perspective”.
Let’s wait until this is picked up more widely and actual implementations turn up that we can use for a start.

Dear All Octave Developers,
Will there be an image-acquisition package which also works for imaging devices connected to Windows OS? :upside_down_face:

I heard that the present package only works for imaging devices connected to a Linux OS.

@HKPhysicist: Generally, Octave “core” and the packages aren’t maintained by the same group of (voluntary) developers.
This thread is about the next release of core Octave. You’d have to ask the package developers of the image-acquisition package if they plan to support other platforms…

Another theme I forgot to mention

  • Documentation
    • Split Octave manual in to a Language Reference Manual which is strictly about documenting the language and its functions, and an Applications Manual which is a guide to using Octave that contains less formal language and many examples.
    • Ensure that every function which returns a value documents that in the function prototype. Lots of old Octave functions assumed that the user understood a value would be returned.
    • Enforce a common format with function prototypes, 1-line summary description, description of function, description of inputs, description of outputs, application notes (tips), references, and examples.
1 Like

I opened a new topic for the replacement command window and posted a (very!) preliminary patch here:

My Wish Item:
Please update Octave Coder for Octave v6.1.
https://github.com/shsajjadi/OctaveCoder

Thanks for your attention. :hugs:

AFAIK, that project isn’t managed or developed by the core Octave developers. You should probably contact the person responsible for it at their github project page.

I’m a big fan of everything @rik has laid out in the “Matlab compatibility” and “Documentation” items here.

As far as dataframes (called “table arrays” in Matlab) are concerned, I have a proof-of-concept/beta-quality implementation of tables in my Tablicious package for Octave (GitHub - apjanke/octave-tablicious: Table (relational, tabular data) implementation for GNU Octave) and as discussed on the Maintainers mailing list, would be happy to contribute that upstream to core Octave and do some work to get it integrated. I’ll work on improving the documentation for it so that it’s easier for y’all to assess its suitability.

FWIW, in my Matlab work lately, I’ve been making a fair amount of use of Matlab’s new “argument” input validation blocks, and have found them to be very useful and convenient. Would be nice to see Octave get that.

And as you can probably guess, I’m a fan of “biting the bullet” on string arrays/Matlab char compatibility/double-quoted-string-literals. But that’s from the perspective of someone who values compatibility with Matlab code much higher than back-compatibility with older Octave codebases.

@rik: For a hash operator, are you thinking it would be pass-by-value like a normal Octave array, or pass-by-reference like a handle object? Matlab’s containers.Map is handle/pass-by-reference, and in practice that turns out to make it not very useful, because you can’t mix-and-match it with other regular pass-by-value Matlab data structures. I’ve seen almost no use of containers.Map in my own code or in the wild, and I suspect it’s largely for this reason.

I don’t do it so much anymore, but I use to code a lot of Perl. And having scalars, lists, and hashes as fundamental data types built in to the language made it very easy to construct more complicated data structures like data frames.

I’m with you in that my code never uses containers.Map, and that is a shame because often a hash is what I’m looking for. I usually resort to strcmp or find (O(n) performance) which are fine only as long as the problem set is small.

Even the ostensible alternatives in Octave don’t work well. Octave’s struct data type is built on top of C++ std::map. This should be a good solution as the field names are the keys in std::map and the C++ STL keeps those keys in sorted order and guarantees search time is logarithmic (presumably, O(log2 (n)). But, because of inadequacies on our side, the performance of a lookup like isfield is linear in size (O(n)). See this bug report about that issue GNU Octave - Bugs: bug #58105, isfield needs time proportional to... [Savannah].

Perhaps this is just a long-winded way to say that we need to improve things in Octave, and once improved it needs to be adopted by both core Octave programmers and by regular users. If pass-by-value helps increase adoption then I’m all for it.

It’s been a while since we had a virtual Octave meeting on Jitsi. I propose that we have another one with this topic, “Goals for the 7.0 release”, as the primary theme.

@jwe: This post has subsequently received 2 Likes. Can you make it happen on Jitsi? I’m out next week, but otherwise ready to attend.

2 Likes

@rik: Oh I’m with you. I’d really like to have an efficient hash/dictionary/associative-array type in Octave, as long as it was pass-by-value instead of pass-by-reference. (Octave’s struct is better than Matlab’s in this regard, because it allows arbitrary strings as keys, not just valid identifiers/variable names, right? But you’re still limited to strings as the key type; not ideal.)

I’ve added some developer documentation for Tablicious:

Including stuff on the “proxy key” trick:

If anything in the doco isn’t clear, or you’d like to see more stuff in there, drop a request on the repo Issues tracker at:

1 Like