Conversion to int8 yields false result in subtraction

System: Linux pinot 5.19.7-1-default
Octave: >> version
ans = 7.2.0

What is going on here:

x=1.35
x = 1.3500

x-fix(x)
ans = 0.3500 <---- OK

x-int8(fix(x))
ans = 0 ???

Can anyone explain why such a weird result is displayed?

When you add / subtract an int8 to/from a double, the double gets converted to int8 first. So you’re doing implicitly this operation:

int8(1.35) - int8(1)

Which becomes 1-1 giving 0, of type int8.

See this for reference: Automatic Conversion of Data Types (GNU Octave (version 7.2.0))

2 Likes

I suspected this already but found no docu for that. Thank for your explanation and for the link.

The example is misleading in the link:
uint8 (1) + 1
** ⇒ 2**

In my opinion it would be better
uint8 (1) + 1.88
** ⇒ 2**

When doubles are converted to integer, they are rounded not floored. This too is for Matlab compatibility. So your second example would result in 3, not 2.

octave:1> uint8(1) + 1.9
ans = 3

Thank you for teaching me this details.
You see that my example is much better for learning this topics.
Thanx