Automatic Differentiation

I have created an automatic differentiation class for Octave.

https://mathsfromnothing.epizy.com/scalar-functions-vector-matrix-and-tensor-functions/

It can handle the basic operators, most core functions, matrix factorizations, and functions. See the link for a complete list. The code is in a single file that can be downloaded from the ‘implementation and how to use it’ section of the page linked above. I also have another file, not on the website testing all (I think) of the code paths. It currently does not support complex numbers. They can be used but incorrect results may be returned. Is this something that you would like incorporated into Octave as a package?

It only finds the first derivative bug #59718 Broadcasting does not work with sparse matrices discouraged me from attempting the second derivative.

The automatic differentiation method I implemented is part of a large group of tutorials on numerical methods available on the same site

https://mathsfromnothing.epizy.com/numerical-mathematics/

As part of writing these tutorials, I have found several bugs and provided solutions (I’m sure I’ve missed a few):

  • #59705 repelem error with sparse matrix input
  • #60738 logm returning incorrect result with some real non-symmetric matrices
  • #60786 mpower: negative scalar to power of matrix containing non-integers returns nan
  • #60797 sqrtm: returns nan for matrix of ones with rows and columns >=4
  • #60817 log2 documentation incorporate X=F*2^E
  • #61191 eval can cause a crash to desktop

I have also implemented several new functions:

  • #59850 Add missing function “uniquetol”
  • #10019 [octave forge] (statistics) slicesample implementation
  • #10016 [octave forge] (statistics) mhsample implementation (over from bug #59924)
  • #10023 bvp4c and bvpinit

No decision has been made about which of the three different algorithms to use in bvp4c so I have not finished the implementation.

2 Likes

To me your contribution seems of wide enough interest that turning it into a package would be beneficial. packages/CONTRIBUTING.md at main · gnu-octave/packages · GitHub has the current guidelines for doing so.

2 Likes

I have tried to add it as a package. Link to the code on Github:

Let me know what further action to take.

IIUC, Octave packages must contain a DESCRIPTION file and a COPYING file in their root directory in order to be installable with the pkg command.
See, e.g., the documentation here: The DESCRIPTION File and Creating Packages

Those files have been added.

If I understand the documentation correctly, the src folder is for files that must be processed (e.g., compiled) before they can be installed. Since that folder only contains .m files, you should probably rename it to inst.
Not very important: Since you added a COPYING file, you could probably remove the (identical) LICENSE file.

Also, according to Creating Packages, as your package consists of m-files only, the directory containing the files should be named inst and not src.

Thanks for this very interesting contribution, looking forward to giving it a try!

I have made the above changes.

Now that this is done, you could try to install your package in Octave. The following command might do that:

pkg install "https://github.com/StevenWaldrip/Automatic-Differentiation/archive/refs/heads/main.zip"

If that works, you could add a tag to that revision and add a new .yaml file for Octave’s package index. That might help people finding your package.
See also here: packages/CONTRIBUTING.md at main · gnu-octave/packages (github.com)

Just a side note: I understand that naming the class ad makes it easy to type when working with your package. However, that name isn’t very specific. Maybe consider using a longer name or putting it into a namespace (+folder) to avoid clashes with user functions or variable names.

The above link works to install it. How do I remove it? It’s not in the package list or not on the path (that I saw). I tried to submit a .yaml file just before not sure that I did it correctly. I’m not keen to change the ad name to keep the typing to a minimum + Matlab has packages with the same name for automatic differentiation. I’m completely lost with the rest of the message. What is a ‘tag to that revision’ and what is a ’ namespace (+folder)'?

Ah. I didn’t know that this is a Matlab compatible implementation. In that case, keep that name!

I tried to install the package. But it failed with the following messages here:

>> pkg install "https://github.com/StevenWaldrip/Automatic-Differentiation/archive/refs/heads/main.zip"

warning: pkg: skipping invalid line in DESCRIPTION file
warning: called from
    get_description at line 54 column 9
    install at line 131 column 14
    pkg at line 603 column 9

error: description is missing needed field description
error: called from
    get_description at line 81 column 7
    install at line 131 column 14
    pkg at line 603 column 9

Are you sure it worked for you?

The Matlab package is a user-created one, not official. I also got the same error. It worked likely because of my local copy. I was not in the directory I created the file in so I assumed it would not work (and it’s not in a path). How do I fix the error?

I added the word description to the description file will that fix it?

With the latest version, I am seeing this:

>> pkg install "https://github.com/StevenWaldrip/Automatic-Differentiation/archive/refs/heads/main.zip"

warning: doc_cache_create: unusable help text found in file 'ad'
warning: called from
    doc_cache_create>handle_function at line 98 column 5
    doc_cache_create>create_cache at line 118 column 36
    gen_doc_cache_in_dir>@<anonymous> at line 150 column 16
    doc_cache_create>gen_doc_cache_in_dir at line 151 column 9
    doc_cache_create at line 62 column 12
    install>generate_lookfor_cache at line 840 column 5
    install at line 241 column 7
    pkg at line 603 column 9

>> pkg list
Package Name               | Version | Installation directory
---------------------------+---------+-----------------------
                    audio  |   2.0.4 | C:\Program Files\GNU Octave\Octave-7.1.0\mingw64\share\octave\packages\audio-2.0.4
automatic differentiation  |   1.0.0 | C:\Users\Markus\AppData\Roaming\octave\api-v57\packages\automatic differentiation-1.0.0
[...]

So this seems to be working (kind of).
I am not sure if a package name containing spaces is well supported though. Maybe consider replacing the space with a -.

I made that change. I’m not sure about the version it will start working with, it needs classdef. I have no documentation in the file Mainly because it has almost the same functionality as the inbuilt functions. I print to the screen instructions for its use and have a significant amount of documentation on my site. What should I change?

I tested the following:

pkg list
pkg load automatic-differentiation
adtest

The last command didn’t cause any errors. So, I think this is working.
However, it cleared my command window content which I find quite intrusive. Could you remove the clc from that function? Or better yet, the entire line clear all;clc; (so that it wouldn’t clear any break points either…).

The pkg function has a test facility that could be run with pkg test automatic-differentiation. Afaict, the package doesn’t contain any built-in self tests (BISTs) currently.
IIUC, those BISTs are run automatically on gnu-octave/packages whenever a package is added or updated. They help a lot in checking if something broke somewhere.
You could add the following line to the end of ad.m to have adtest be executed as a BIST:

%!test adtest

classdef is still pretty new in Octave. And some features of it have been added and improved over the last few versions. I don’t know how to tell which minimum version your package requires apart from manually testing it in different versions.
That would be the “nice option”. The “lazy option” would be to wait for users of older versions to complain…

After that, I’d say that the package is in a pretty good shape for a first release.

Regarding adding a tag: Are you using git to push from a local checkout of your repository? Or are you using GitHub’s web interface exclusively? (Either works.)

I removed the clc. The clear all is required for testing as the class is not updated when it is saved GNU Octave - Bugs: bug #45893, classdef properties are not... [Savannah]. I’ve added

%!test adtest

to the end of ad.m. I set the minimum version to 5.1.0 as it mostly works with that version (the end function causes it to fail in that version but it can be worked around) I got a syntax error with 4.2.1.

I’m currently using the web interface for github. I typically use bitbucket for version control and let the IDE I’m using manage everything. I’m not sure octave has version control inbuilt, it would be nice to have.

As the original links to my site are showing a security issue, although they still work (have not updated the SSL) they are repeated here:

https://mathsfromnothing.cf/scalar-functions-vector-matrix-and-tensor-functions/

https://mathsfromnothing.cf/numerical-mathematics/

And since I think my latest creation is really useful a link to my computer algebra system. Its intended purpose is to help high school students to learn algebra.

https://mathsfromnothing.cf/computer-algebra-system/

pkg test automatic-differentiation fails with the latest version.
If I replace clear all with clear functions in adtest, changes to the classdef are still picked up. But the test passes now locally.
Maybe, you could do something similar?

To add a version tag, the instructions here could be helpful: git - Create a tag in a GitHub repository - Stack Overflow

1 Like

I have changed clear all, to clear functions

Set a seed for rand to avoid known issues with complex numbers and some matrix factorizations.

Created a release or tag

Updated the .yaml file after changing links and creating an image and pressed commit.

Let me know if I need to make further changes.

1 Like