New command window widget

From first tests, I see the following issues:

  • No empty editor tab in visible editor at startup
  • Menu shortcuts in the main window not working
  • Running scripts from the editor
  • Executing commands from the history widget

Or is this working for others and it is my build?

I didn’t attempt to duplicate the signals/slots in the current QTerminal interface so I’m not surprised that these things aren’t working.

Another thing we have to work out is how to handle functions like input that perform user interaction in the terminal window. Previously that automatically worked because stdin/stdout/stderr were all filtered directly through the terminal widget. But I don’t think that will work if we have Octave running as a “server”, possibly across a network connection. So we need to work out some set of signals that can be passed through the event_manager object.

Probably that whole event_manager thing and communication between the GUI and the Octave interpreter needs to be simplified/generalized. See also this topic on the maintainers mailing list that includes some discussion about simplifying the event_manager link between the Octave interpreter and GUI applications:

Naturally, the signal/slot interface of the old terminal does not exist in the new proof of concept. But it is merely the changed startup sequence which, e.g., prevents the main window from executing some initialization code. Before duplicating portions of code, I am going to check how things can be rearranged/simplified in the existing code for being sort of more “compatible” to the new approach.

Qt and event handling aren’t really my area of expertise. So at the moment, I’ll probably not be able to give more support than simple testing (mostly anectodal, but hopefully somewhat useful).
I tried jwe’s patch on Ubuntu 20.10 on top of hg id 4363b185d31f. (It no longer applies cleanly. But easy enough to rebase.)
I started Octave with ./run-octave in CLI mode. After that, I started the GUI with __event_manager_start_gui__. I had to resize the GUI slightly to make it appear properly. Before that it was mostly a black canvas. Maybe a side effect of the missing initialization Torsten was writing about.
I tried to check the persistent figures by executing demo light. I probably shouldn’t have done that because that left the command line in the GUI irresponsive to any further input. That is probably the related to the incomplete user interaction that jwe was hinting at with the input function.
I then closed the GUI hoping to be able to recover from that state on the CLI. And the CLI indeed showed “Press <enter> to continue:”. But the CLI was stuck as well and I had to kill the Octave process.

I also tried by cross-compiling with the patch for Windows: Entering non-ASCII characters (bug #47571) seems to be working with that approach. :+1:
I’m attaching a patch for MXE Octave that adds a new configure option --enable-experimental-terminal-widget (MXE-Octave-commandline-widget.patch (2.6 KB)). The Octave source ball used in MXE Octave needs to be created from an Octave repository with jwe’s patch for that configure option to have any effect.

Please find “attached” a version of the base patch, that applies to the current default branch without conflicts:

new-terminal-diffs-v01.patch (41.8 KB)

Attached a patch on top of the base patch, that uses terminal font and colors from the preferences:

new-terminal-diffs__addon_01.patch (11.0 KB)

After changeset you can start the new terminal widget by using the command line options --experimental-terminal-widget --gui. Please note the following (from the commit message):

This changeset provides a new experimental proof-of-concept cross-platform command window. It is intended to demonstrate how communication between the GUI command window and the Octave interpreter can work when the GUI is completely responsible for user input instead of having the interpreter wait for input inside readline.

This initial implementation uses a simple text box for input and a separate text edit area for output. This design is not intended to be the final arrangement, but was easy to implement for demonstration purposes. These changes also make it possible to run the command-line version of Octave in a similar client-server mode with a function gathering input in one thread and the Octave intepreter running in another.

The new command window is not intended to provide a general-purpose terminal window. So running something like “system (‘emacs’)” will not bring up an instance of Emacs running in the command window. This also means that it will no longer be possible to use an external output pager such as “less” in the command window. OTOH, it is also no longer necessary to fork and exec a separate process on some Unixy systems when Octave starts solely to give up the controlling terminal so that “less” will function properly.

With the new terminal window, it is now possible to start Octave in command line mode and later open the GUI desktop window and then return to the command line when the GUI window is closed.

Some things that need to be finished:

  • Merge the input and output windows so that command input and output are interleaved as they are in a normal command window.

  • Provide real readline command editing in the GUI command window and at the client-server CLI prompt. We can use the readline callback interface for the GUI and the normal readline interface for the CLI. With this design, the command widget and the CLI front end hold a copy of the command history. They do not need to communicate with the interpreter to navigate the command history.

  • Create interpreter_event messages for handling user input so that the interpreter can ask the GUI to gather user input for functions like “input”.

  • Consider passing results back to the command widget as octave_value objects and allowing the command widget to format and display them instead of passing formatted text from the interpreter to the command widget.

  • Provide an output pager feature for the GUI command window? Maybe this feature is not necessary since we have scroll bars in the GUI and we can also have the GUI ask the user whether to display output if it is large.

is there something short of a full pager that would optionally just provide a ‘press Enter to continue’ when the output spans more than the current number of display lines?

I’m sure we could add simple pager-like features to the command window widget that would allow partial output to be displayed and remaining output to be discarded.

This works for me on Ubuntu 20.10 and Windows 10.
For some reason, the command history is empty when I start with that option.

Being able to enter any characters on my keyboard without the fear of crashing the command window (on Windows) is a huge improvement imho.
I wouldn’t even mind if input and output were separated (like they currently are with the experimental terminal widget) if implementing a combined input/output widget turns out to be difficult.
I’d find it more convenient if the input line was below the output text box though.

It would also be nice if the font was used that was set for the command widget in the preferences.

The only major drawback (and possibly the only point why I wouldn’t start using it on a daily basis just now) is this point you mentioned:

When that point is resolved, we could set the new terminal widget as the default imho (at least on Windows).

Would it be possible to have an option in the preferences to select with which terminal widget the GUI should start by default?

Support for readline in the input line would also be nice to have.

I am going to merge the addon-patch from my comment on Dec 27 to the new command-widget. It implements the usage of some terminal preferences, moves the input line to the button, and adds some style polishing.

While testing the new pause feature, which works perfectly when running a script, I noticed two issues:

  1. There is no way to know whether Octave is waiting for input or already running some command: Currently the [in] prompt only appears at the beginning of lines that have already been executed. I understand that having a read-only prompt at the left-hand side of the input line is not easy, but having an indication that Octave is actually waiting for input would help. At least the input line should not be editable while the interpreter is already running.
  2. Octave may fail understanding some partial command:

I tried to run the following infinite loop at the command line, one line after the other:

while (true)
a=rand (1);

And here is what I obtain in the terminal:

[in]: while (true)
a=rand (1);
[in]: end
error: parse error:

  syntax error

>>> end

For some reason, the third line is actually interpreted as a single command, out of the context of the two previous ones. Note that adding another line, e.g. "b = 2;" before the end keyword makes it work.

I have pushed the patch for reading terminal settings and moving input box to the bottom with cset octave: d6b2d9f9e1e0

I’m fairly sure that feature worked properly at one point, so maybe I broke it with some other change. I’ll try to check it out and fix.

The [in] marker in the output window is just there to show that the line is input, not output.

If you enter something like for i = 1:10 in the input text box, you should see the prompt change from >> to > to indicate that more input is expected. It should stay that way until the statement is complete, but as you noticed, if you enter another line like disp(i) then the prompt goes back to >> indicating that the interpreter is confused about the current input state.

Also, having separate input and output windows is only temporary. I have no intention of leaving it that way. It was just easy to implement for this initial experiment.

I pushed the following change:

I’m now able to enter multi-line input and it appears to work correctly.

If I execute something like for i = 1:100, i, pause (1), end at the command prompt, then I am able to use the [stop] button to interrupt the execution and jump back to the prompt, though that action is not immediately obvious because there is no indication in the input or output window that the interrupt occurred.

For that same loop input at the command line, I’m also able to use the [pause] button to interrupt the interpreter and enter the debugger. But again, there is no clue in the input or output windows about that. From there, the [continue] button resumes execution, as will dbcont at the command prompt.

If those same commands are place in a script file and the script is executed, then the [pause] button again enters the debugger and the script file is opened in the editor window and the usual debugger interactions work there with an arrow indicating where the evaluator is currently paused. In this case a “stopped at” debugger message is displayed in the output window but the GUI doesn’t switch to the editor window.

I’ll see whether I can at least fix the prompt so that it switches to “debug>” or similar and provide some feedback about interrupts happening.

make check fails for me at hg id b65824235c7f during the first test in
Possibly related to that change.

Oops. You’d think by now I would know better than to push a change like that without at least attempting to run some tests…

I pushed a more limited change:

I am attaching a patch that merges the input widget of the new experimental terminal widget (previously a separate line edit) and the output into one single widget. There is still no readline functionality or history browsing.

If you think, that this goes into the right direction, I would push this patch to the default branch.

new-terminal-in-single-widget.patch (12.3 KB)


@ttl: I compiled with your patch on Windows and started Octave with ./run-octave --gui --experimental-terminal-widget.
The update to the new command window widget looks great. :+1::tada:
With your changes, it almost feels like the old one. But the widget no longer freezes on entering non-ASCII characters (which is a big plus!).

Imho, it is a step in the right direction. I can’t comment much on the implementation. But afaict, it looks reasonable.

Just some remarks (that could also be addressed in later changesets imho):

  • When clicking somewhere in the command window and starting to type, the text is inserted at that position. Hitting the “enter” key doesn’t execute that command, but displays a new prompt line. Imho, it would be nice if the cursor jumped to the command line when starting to type something.
  • When executing error('test'), the >> indicating a new command prompt isn’t displayed.
  • When executing some command that requires an input at the prompt (e.g. demo light), the command window is no longer usable. Every new command seems to execute demo light again. But ISTR something similar already happened before your changes. So, this isn’t a regression. (Probably some missing signal-event-slots?)
  • Only aesthetics: The scroll bar on the side of the command widget looks different from the native scroll bar of the OS.
  • Double-clicking a command from the history doesn’t execute it. (Also probably independent of your latest patch.)