Weird Math (aka “IEEE-754 double-precision floating-point number sucks”)

actionscript 3, bugs, Math

Open your favorite actionscript editor and type:

var n:Number=123456789012345672;
trace(n);
// outputs: 123456789012345660

or simply:

trace(123456789012345672);
// outputs: 123456789012345660

now try incrementing the number by one:

trace(123456789012345673);
// outputs: 123456789012345680

oh dear… IEEE-754 is the specification on which the Number class of actionsctipt/javascript (all ecma…) and many others language is based.

IEEE-754 sucks!

Reading some documentation this format should support 53 digits numbers… but seems kinda different.

Ok ok those were high numbers… but try this:

trace(0.1*3);
// outputs: 0.30000000000000004
trace(0.1*6);
// outputs: 0.6000000000000001
trace(0.1*9);
// outputs: 0.9
trace(0.1*12);
// outputs: 1.2000000000000002
trace(0.1*15);
// outputs: 1.5

I was thinking to use actionscript to collaboratively solve some of Project Euler problems… just for fun, but I don’t think I will waste just a single line of code on this from now on.
This really disappoints me, I mean, if computers cannot either do math, why the hell am I still sitting here?

2 Comments

2 Comments

  1. Vyper  •  Mar 22, 2008 @9:53 am

    The low float value behaviour can be explained here:
    http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=tn_13989

    However the high number issue makes no sense since the highest number Flash should be able to handle is 1.79769313486231e+308.

  2. EECOLOR  •  Mar 22, 2008 @12:01 pm

    Try this:

    trace(“123456789012345672:”, 123456789012345672, “, 123456789012345672 – 72:”, 123456789012345672 – 72);

    As you can see, Flash can handle bigger numbers. Apparently something goes wrong while displaying them.

    Greetz Erik

Leave a Reply

Allowed tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>