float/double calculation bug ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

float/double calculation bug ?

Tx. T
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
Mac OS X 10.12.3


groovy:000>     def calc = { amount, ttl ->
groovy:001>         double rtn
groovy:002>         if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003>         else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
groovy:004> 
groovy:004>         rtn
groovy:005>     }
===> groovysh_evaluate$_run_closure1@39fcbef6
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: float/double calculation bug ?

Derek Visch
Looks like floating point to me, what are you expecting?

On Mar 19, 2017 10:04 AM, "Tx. T" <[hidden email]> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
Mac OS X 10.12.3


groovy:000>     def calc = { amount, ttl ->
groovy:001>         double rtn
groovy:002>         if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003>         else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
groovy:004> 
groovy:004>         rtn
groovy:005>     }
===> groovysh_evaluate$_run_closure1@39fcbef6
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: float/double calculation bug ?

Steve Byrne
What happens if you use .toDouble() instead of .toFloat, and an explicitly double 100.0 literal?  Right now, your computation is taking an integer, dividing by a float, then multiplying by another float.  I suspect that going double precision won't fix this in all cases (floating point math is, after all, just an approximation), but at least you're processing the values as precisely as you can.

On Sun, Mar 19, 2017 at 8:28 AM, Derek Visch <[hidden email]> wrote:
Looks like floating point to me, what are you expecting?

On Mar 19, 2017 10:04 AM, "Tx. T" <[hidden email]> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
Mac OS X 10.12.3


groovy:000>     def calc = { amount, ttl ->
groovy:001>         double rtn
groovy:002>         if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003>         else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
groovy:004> 
groovy:004>         rtn
groovy:005>     }
===> groovysh_evaluate$_run_closure1@39fcbef6
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: float/double calculation bug ?

Edmond Kemokai-2
In reply to this post by Derek Visch
Seems like floating point approximation at runtime. Interestingly enough your code works with exact result when I try it but any minor reformulation ends up with the approximate result. Also when I replace your computation with literal integers I get the exact result...floating point arithmetic can get messy so you'll have to think about what you're trying to accomplish and perhaps consider rounding up/down as needed...I would avoid words like mantissa or significand but basically you could choose a tiny  number and set it as the limit for rounding up in terms of the difference between your floating point result and the next integral/integer value...I don't want to get back into numerical analysis so I'll stop :)

I am the developer of HiveMind, it is a JSR223 based web app platform for Groovy and other JVM languages. You can access:

url: http://demo.crudzilla.com:7000/
login: developer/developer

I have created a test file in: /com/crudzilla/cloudTest/web/txt8888/test.ste

You can test the code by right clicking on the test.ste file and selecting "Open In Browser".

I have put your code in there, feel free to play with it and try out the platform as a whole.

On Sun, Mar 19, 2017 at 11:28 AM, Derek Visch <[hidden email]> wrote:
Looks like floating point to me, what are you expecting?

On Mar 19, 2017 10:04 AM, "Tx. T" <[hidden email]> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
Mac OS X 10.12.3


groovy:000>     def calc = { amount, ttl ->
groovy:001>         double rtn
groovy:002>         if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003>         else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
groovy:004> 
groovy:004>         rtn
groovy:005>     }
===> groovysh_evaluate$_run_closure1@39fcbef6
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: float/double calculation bug ?

sbglasius
In reply to this post by Steve Byrne
Hi,

This discussion does not belong on the dev mailing list, but the user mailing list ([hidden email]). Please continue your very interesting discussion there :-) Thanks.

/Søren
On Sun, 19 Mar 2017 at 16:47 James Bond <[hidden email]> wrote:
What happens if you use .toDouble() instead of .toFloat, and an explicitly double 100.0 literal?  Right now, your computation is taking an integer, dividing by a float, then multiplying by another float.  I suspect that going double precision won't fix this in all cases (floating point math is, after all, just an approximation), but at least you're processing the values as precisely as you can.

On Sun, Mar 19, 2017 at 8:28 AM, Derek Visch <[hidden email]> wrote:
Looks like floating point to me, what are you expecting?

On Mar 19, 2017 10:04 AM, "Tx. T" <[hidden email]> wrote:
Any idea why the follow code "calc" returns the "70%" of the 330000 incorrectly?
testing on: Groovy Version: 2.4.9 JVM: 1.8.0_112 Vendor: Oracle Corporation OS: Mac OS X
Mac OS X 10.12.3


groovy:000>     def calc = { amount, ttl ->
groovy:001>         double rtn
groovy:002>         if (amount[-1] != '%') rtn = amount.toDouble()
groovy:003>         else rtn = ttl / 100.0 * amount.replaceAll(/%\Z/, '').toFloat()
groovy:004> 
groovy:004>         rtn
groovy:005>     }
===> groovysh_evaluate$_run_closure1@39fcbef6
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("10%", 330000)
===> 33000.0
groovy:000> calc("20%", 330000)
===> 66000.0
groovy:000> calc("30%", 330000)
===> 99000.0
groovy:000> calc("40%", 330000)
===> 132000.0
groovy:000> calc("50%", 330000)
===> 165000.0
groovy:000> calc("60%", 330000)
===> 198000.0
groovy:000> calc("70%", 330000)
===> 230999.99999999997
groovy:000> calc("80%", 330000)
===> 264000.0
groovy:000> calc("90%", 330000)
===> 297000.0
groovy:000> calc("100%", 330000)
===> 330000.0

--
Best regards / Med venlig hilsen,
Søren Berg Glasius

Hedevej 1, Gl. Rye, 8680 Ry, Denmark
Mobile: +45 40 44 91 88, Skype: sbglasius
--- Press ESC once to quit - twice to save the changes.
Loading...