Parsing command-style function call syntax

I’m trying to improve Octave’s compatibility with Matlab for parsing command-style function call syntax. See the following bug reports about trouble caused after changing Octave to be more like (modern) Matlab and not check the status of symbols as variables when deciding whether to parse something like foo +bar as an expression or a command-style function call:

https://savannah.gnu.org/bugs/?60882
https://savannah.gnu.org/bugs/?60941

To better understand what Matlab behavior actually is, I wrote some test scripts. Could someone please help by downloading and unzipping the attached file and running the included main_test_script in a current version of Matlab and then uploading the results here?

I decided to open a new topic instead of posting this file under the “Need help understanding Matlab behavior” topic because I anticipate additional discussion about how best to handle the conflicts that arise due to Octave’s extended set of operators.

command_syntax_tests.zip (5.9 KB)

The script fails early on with a syntax error. This is because the bash operator ! means something different in Matlab and Octave. It sends a command to the system shell in Matlab.
After changing the ! to ~, I see the following in Matlab R2021a:

>> main_test_script
    @ : FAILED 3/4 tests
cmd_fcn @ arg1;: evaluation error: 'arg1' is not a valid base class.
cmd_fcn@arg1;: evaluation error: 'arg1' is not a valid base class.
cmd_fcn@ arg1;: evaluation error: 'arg1' is not a valid base class.
   && : OK
    & : OK
    * : OK
    + : OK
    - : OK
   .* : OK
   ./ : OK
   .\ : OK
   .^ : OK
    / : OK
    : : OK
    < : OK
   <= : OK
   == : OK
    > : OK
   >= : OK
    \ : OK
    ^ : OK
    | : OK
   || : OK
   ~= : OK
    ' : FAILED 2/4 tests
cmd_fcn ' arg1;: evaluation error: Error: Character vector is not terminated properly.
cmd_fcn 'arg1;: evaluation error: Error: Character vector is not terminated properly.
   .' : FAILED 2/4 tests
cmd_fcn .' arg1;: evaluation error: Invalid expression. Check for missing multiplication operator, missing or unbalanced delimiters, or other syntax error. To construct matrices, use brackets instead of parentheses.
cmd_fcn .'arg1;: evaluation error: Error: Character vector is not terminated properly.
    ~ : OK
   .+ : FAILED 2/4 tests
cmd_fcn.+arg1;: evaluation error: Invalid use of operator.
cmd_fcn.+ arg1;: evaluation error: Invalid use of operator.
   .- : FAILED 2/4 tests
cmd_fcn.-arg1;: evaluation error: Invalid use of operator.
cmd_fcn.- arg1;: evaluation error: Invalid use of operator.
  .** : FAILED 2/4 tests
cmd_fcn.**arg1;: evaluation error: Invalid use of operator.
cmd_fcn.** arg1;: evaluation error: Invalid use of operator.
   != : FAILED 3/4 tests
cmd_fcn != arg1;: evaluation error: Invalid use of operator.
cmd_fcn!=arg1;: evaluation error: Invalid use of operator.
cmd_fcn!= arg1;: evaluation error: Invalid use of operator.
   ** : FAILED 2/4 tests
cmd_fcn**arg1;: evaluation error: Invalid use of operator.
cmd_fcn** arg1;: evaluation error: Invalid use of operator.
    ! : OK
   ++ : OK
   -- : OK
    $ : OK
    ? : OK
    ` : OK
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   &= : FAILED 2/4 tests
cmd_fcn&=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn&= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  **= : FAILED 2/4 tests
cmd_fcn**=arg1;: evaluation error: Invalid use of operator.
cmd_fcn**= arg1;: evaluation error: Invalid use of operator.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   *= : FAILED 2/4 tests
cmd_fcn*=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn*= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   += : FAILED 2/4 tests
cmd_fcn+=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn+= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   -= : FAILED 2/4 tests
cmd_fcn-=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn-= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
 .**= : FAILED 2/4 tests
cmd_fcn.**=arg1;: evaluation error: Invalid use of operator.
cmd_fcn.**= arg1;: evaluation error: Invalid use of operator.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  .*= : FAILED 2/4 tests
cmd_fcn.*=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn.*= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  .+= : FAILED 2/4 tests
cmd_fcn.+=arg1;: evaluation error: Invalid use of operator.
cmd_fcn.+= arg1;: evaluation error: Invalid use of operator.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  .-= : FAILED 2/4 tests
cmd_fcn.-=arg1;: evaluation error: Invalid use of operator.
cmd_fcn.-= arg1;: evaluation error: Invalid use of operator.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  ./= : FAILED 2/4 tests
cmd_fcn./=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn./= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  .\= : FAILED 2/4 tests
cmd_fcn.\=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn.\= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
  .^= : FAILED 2/4 tests
cmd_fcn.^=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn.^= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   /= : FAILED 2/4 tests
cmd_fcn/=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn/= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   \= : FAILED 2/4 tests
cmd_fcn\=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn\= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   ^= : FAILED 2/4 tests
cmd_fcn^=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn^= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 14)
In main_test_script (line 116) 
Warning: Identifier 'cmd_fcn' in the evaluated statement does not refer to the variable. In a future release, using an identifier different to how it is used in the file will not be supported. 
> In test_computed_assign_op>test_code (line 77)
In test_computed_assign_op (line 37)
In main_test_script (line 116) 
   |= : FAILED 2/4 tests
cmd_fcn|=arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.
cmd_fcn|= arg1;: evaluation error: Incorrect use of '=' operator. Assign a value to a variable using '=' and compare values for equality using '=='.

Changed files:
command_syntax_tests.zip (5.8 KB)

Oops. I guess I would have expected a syntax error for those uses but not a shell escape unless ! was at the beginning of a statement. Just out of curiosity, what is picked up as the shell escape arguments? Everything until the end of the line?

No. There were still open brackets that haven’t been closed before the bash operator. So it failed because of a syntax error.

OK, here is an updated test function. It’s all in one file now. I hope with this version we can at least capture the current behavior for both Octave and Matlab. I think I have all the “expected” (i.e., current behavior) results for Octave correct now.

main_test_fun.m (6.3 KB)

Some endif made it into the test function. After replacing them by end, I see the following in Matlab R2021a:

>> main_test_fun
   && : OK
    & : OK
    * : OK
    + : OK
    - : OK
   .* : OK
   ./ : OK
   .\ : OK
   .^ : OK
    / : OK
    : : OK
    < : OK
   <= : OK
   == : OK
    > : OK
   >= : OK
    ^ : OK
    | : OK
   || : OK
   ~= : OK
    ' : OK
   .' : OK
    ! : OK
    $ : OK
    ? : OK
    ` : OK
    @ : OK
    \ : OK
    ~ : OK
   .+ : OK
   .- : OK
  .** : OK
   != : OK
   ** : OK
   ++ : OK
   -- : OK
   &= : OK
  **= : OK
   *= : OK
   += : OK
   -= : OK
 .**= : OK
  .*= : OK
  .+= : OK
  .-= : OK
  ./= : OK
  .\= : OK
  .^= : OK
   /= : OK
   \= : OK
   ^= : OK
   |= : OK

Thanks.