PDA

View Full Version : Strange behaviour of math.bxor function



VSDomotics
October 3rd, 2008, 11:25 AM
The LUA code


print(math.bxor(2147483648,2147483647))


should give 4294967295 as the right answer but I get -1.

How is this possible.
What I try to do is toggle a certain bit in a 4 byte word (32 bits)

So
mask= 10000000000000000000000000000000 (2147483648)
data= 01111111111111111111111111111111 (2147483647)

by XOR I should get
result= 11111111111111111111111111111111 (4294967295)

How is this possible? I think it perhaps has to do with a limitation of LUA (first bit used for sign somewhere?).

Is this intended behaviour or not? If so, how do I solve this in a simple way (toggling a certain bit in a 32 bits word)?

Rob H
October 3rd, 2008, 11:33 AM
Hmm... something is using signed arithmetic - I suspect that it may be the output routines

Rob H
October 3rd, 2008, 11:39 AM
Yes, if you try adding a math.decimaltohex() around the math.bxor then you'll see that you actually get ffffffff from it.

And, indeed, if you try


print(math.decimaltohex(-1))

you also get ffffffff.

VSDomotics
October 3rd, 2008, 04:00 PM
Rob, does this mean it is 'by design'? If so, how to make a workaround? Is it possible to not use a certain function (the signed part) and still get the wanted output?

Rob H
October 3rd, 2008, 04:48 PM
I'm not sure if it is by design, but you do actually get the desired output, it's just that when you print the result you see -1. The bit pattern is actually correct though.

VSDomotics
October 4th, 2008, 12:43 PM
Ahhhh, I see. I only can't compare the decimal values but have to compare the hex values.

Strange when the same variable does not equal when decimal and do equal when the same variables are converted to hex. But it works.
Thanks Rob