**
**
## Binary comparison operation returns a wrong value
Print

Modified on: Mon, 14 Mar, 2022 at 10:12 AM

When using calculations in a macro you sometimes need to check certain values. Most of the time this will be done by an 'If x==y' line.

Although this will work fine when working with numbers like 2, 400 or 29898...things can go wrong if you start to work with numbers like 2.4304, 400.85426 etc. these numbers are considered to be a float number. This refers to how they are represented internally by the computer.

A good explanation can be found here:

In the reply of 'Daniel Scott' the solution is very nicely described as follows:

*"In practice, this problem of precision means you need to use rounding functions to round your floating point numbers off to however many decimal places you're interested in before you display them.*

*You also need to replace equality tests with comparisons that allow some amount of tolerance, which means:*

*Do not do *`if (x == y) { ... }`

*Instead, do *`if (abs(x - y) < myToleranceValue) { ... }`

.

*where *`abs`

is the absolute value. `myToleranceValue`

needs to be chosen for your particular application - and it will have a lot to do with how much "wiggle room" you are prepared to allow, and what the largest number you are going to be comparing may be (due to loss of precision issues)."

Did you find it helpful?
Yes
No

Send feedback Sorry we couldn't be helpful. Help us improve this article with your feedback.