Octave syntax highlighting incorrect for single-quoted strings

Site Feedback
#1

I think this post belongs to this category about the Octave Discourse site.

The issue is that the interpreter does not support line continuations in single-quoted strings. For example,

str = 'this is a string \
that runs on in the highlighter';

The entire string is highlighted, whereas it should end on a single line.

I don’t know where the syntax file for Discourse is located but it needs to be updated.

The same problem also exists with the Qscintilla syntax highlighter used in the GUI text editor. I have filed a bug report about that here: GNU Octave - Bugs: bug #59885, Qscintilla syntax file should not... [Savannah].

Support line continuations within single-quoted strings?
#2

Thanks for the pointer @rik. I’ll take a look how to apply these changes to this Discourse highlight.js instance :construction_worker_man:

Some test:

str = 'this is a string \
that runs on in the highlighter';
str2 = 'this is a string \ 
that runs on in the highlighter';
str3 = "this is a string \
that runs on in the highlighter";
str4 = "this is a string \ 
that runs on in the highlighter";
str5 = ['another ''string''' num2str(42) ' invalid, not terminated "]
str6 = ["another \"string\"" num2str(42) " invalid, not terminated ']

str7 = ['another \n\r ''string''' num2str(42) ' is valid and terminated ']
str8 = ["another \n\r \"string\"" num2str(42) " is valid and terminated "]
#3

@rik I think I played enough on this :exploding_head: Is the result satisfying? :innocent:

#4

The double quoted strings don’t appear with the “correct” coloring for me. (I tried to refresh the page. But that didn’t change anything here.)
The single quoted strings look correct to me.

image
image698×287 6.13 KB

#5

I agree. It looks like single-quoted strings are now highlighted correctly, but there are still a few changes needed for double-quoted strings.

Change #1 : Line continuations are allowed in double-quoted strings

str3 = "this is a string \
that runs on in the highlighter";

This should highlight the whole string as a string. Instead, only the first line up to the line continuation is highlighted.

Change #2 : Escape characters are allowed in double-quoted strings including \"

str6 = ["another \"string\"" num2str(42) " invalid, not terminated ']

The word “string” should be highlighted as a string in this instance.

#6

Oh, I reread the other thread Support line continuations within single-quoted strings? and hopefully now got a right understanding:

valid = "This string \
should work";
valid = "This string with whitespace \ 
should work";
invalid = "This string ...
should not work";
invalid = "This string
should not work';
invalid = "This string 
should not work';

invalid = 'This string \
should not work';
invalid = 'This string with whitespace after \ 
should not work';
invalid = 'This string ...
should not work';
invalid = 'This string
should not work';
invalid = 'This string 
should not work';
#7

I’m not sure if we should encourage using whitespace characters between backslash and line feed characters (your second example). That looks like an “Octave invention” that I didn’t see before…
Also, that syntax is deprecated on default Octave after Rik’s recent changes:
octave: 01679f865fc7 (gnu.org)

Otherwise, that looks very good now. :+1:

#8

Thanks @mmuetzel :slightly_smiling_face:

The whitespace after '\' is ignored, but valid syntax (the Octave 7 parser with @rik 's change does not complain here). But if it is not okay, I can switch to a “more pedantic” highlighting mode :smiling_imp: :wink:

#9

Is there a highlighting convention for errors? The second line of a bad line continuation just drops back in to black text on white background which isn’t very clear. The programmer has to notice that the color has changed. Here is an example of highlighting from Vim which indicates an error condition very forcefully.

error_highlighting

#10

To my understanding of highlight.js not. It tries to identify common programming language components (literals, operators, keywords, …) and highlights what is defined to be valid and does not highlight potential errors or unrecognized code.

#11

I took a look at octave.js. There is an illegal category to which a regular expression could be added. But, I don’t fully understand Highlight.js so I’m not sure how easy this would be. It’s probably fine for now.

I do see that octave.js was based on matlab.js and includes highlighting for functions which Octave does not have (in core) such as nanmax and nanmean. Perhaps @apjanke could update that Git repository.

#12

I can do this.

Remind me where I can find the list of unimplemented functions for Octave?

#13

I’ve started support for this on GitHub - apjanke/highlightjs-octave at support-missing-functions, but i don’t know highlight.js well enough to know how to map the new “missing” class to a style (like, “italicized pink”).

#14

@rik do you mean highlightjs-octave/octave.js at master · highlightjs/highlightjs-octave · GitHub? This Discourse forum has a copy of this in it’s layout customization, as described before. Thus changes there have to copied to this Discourse forum’s layout to become effective.

It is in scripts/help/__unimplemented__.m. I do not think that maintaining such a list in the highlight specifications is of great use. Octave is a very extendable language and missing features are not written in stone plates. Just imagine I have implemented a missing function myself and will always find when I write about it, that it is invalid syntax :thinking: The effort of maintaining such a list is doubled now for any release.