How to prevent that known issues get marked as regressions?

A recent commit by @jwe updated the bug status of some BISTs. Among those were some changes that look like this:

@@ -165,7 +165,7 @@
 %! v = [0, pi/2*i, pi*i, pi/2*i];
 %! assert (acosh (x), v, sqrt (eps));
 
-%!testif ; ismac ()   <52627>
+%!testif ; ismac ()   <*52627>
 %! ## Same test code as above, but intended only for test statistics on Mac.
 %! ## Mac trig/hyperbolic functions have huge tolerances.
 %! x = [1, 0, -1, 0];

I’m assuming that the scripts(?) that updated the bug status were run on Linux. On that platform, those tests should have been skipped. So technically, they didn’t fail.
However, we are using these constructs for duplicating tests that we know are failing on some platforms but are “fixed” on other platforms.

Which syntax should we use for those tests instead to avoid having their status be changed repeatedly?

I removed the * from the tags for the affected BISTs here:
octave: ff5ceee2de29 (gnu.org)

Would still be good to know how to set this up correctly so the tags won’t get updated inadvertently in the future.

there was a conversation i can’t find anymore talking about how we should use the bug number tagging instead of ?or in combination with? xfails, etc. Could that info and what you’re looking for get added over at the wiki under Tests - Octave

1 Like

Is it feasible to do something like this?

%!testif ; ismac ()   <52627>
%!testif ; islinux ()   <*52627>

Or can each bug number be mentioned only once?

That is basically the pattern that is currently used. Or more like:

%!testif ; ! ismac ()   <*52627>
%!testif ; ismac ()   <52627>

The bug numbers might be different for some tests. (One about the original issue. Another one about the platform specific issue.)
But it looks like the latter of the two gets updated automatically…

One possible solution that doesn’t require making the testing functions and syntax even more complicated is to open a bug report that can remain open for the OS-specific problem. Then there will be a separate bug number for the “ismac()” version of the test.

Thank you. ISTR that we talked about this before. But I forgot…

I re-opened the respective report:
GNU Octave - Bugs: bug #52627, Failing BIST tests on Mac OS X [Savannah]