It's all in the code

The nought point threeness of 0.3

with 5 comments

I seem to be programming a lot in javascript these days. Since I work on and one thing that crops up a lot is the need to represent numbers on screen. A simple requirement would be that 0.3 shows up as 0.3 rather than something like 0.29999999999987. For this, javascript provides a couple of useful functions. One rounds your number to a fixed number of decimal places, and the other returns as many significant figures as you want.

So far so good, but there’s a mildly surprising aspect to these two functions. They don’t return numbers. Instead they return character strings – text to you and me. Think about it some more and you realise that yes you really do want the character string “0.3” rather than some number 0.3. After all, the computer program does not usually manipulate a data structure that represents three tenths precisely. Instead, it uses a truncated binary representation that fits the machine architecture better. That’s why, given a chance, it might print out 0.29999999999987 instead of 0.3.

This gets one thinking about the huge difference between the machine’s internal representation of a number – the working model which can be manipulated arithmetically – and the external representation – the set of characters to be sent to a display device. 

The machine usually represents 0.3 in an approximate binary form simply because that’s the fastest thing to manipulate if all you want is answers to 13 significant figures or so. But if you really wanted to do rational arithmetic precisely, then fine, the machine can use a better data structure – one that stores the integers 3 and 10 separately – and achieve perfect precision. We’ve all played with calculators that can add fractions perfectly after all.

The interesting thing about this is that internally, the number we know as 0.3 has to be represented in different ways for different purposes. It’s an object that contains both data – the binary floating point string, the 3, the 10 – and algorithms. Provided we can capture the essence of 0.3 in these objects in a finite amount of computer memory and processing time we have a precise internal model of the number.

Now what about a transcendental number?  Can a computer hold a perfect model of pi? How big an object would the representation be? Would it be finite? Can we ask for ‘perfect’ or must we make do with ‘practical’? Tricky one…


About these ads

Written by gmp26

January 27, 2012 at 10:18 am

5 Responses

Subscribe to comments with RSS.

  1. the solution to the problem of inaccurate fractions is to develop a new data type of the form x*6000.

    I.e. the number 0.3 would be stored as 0.3*6000 = 1800
    the number 1/3 would be stored as 2000
    if users of the system knew of this they could limit their use of fractions to ensure that the internal representation was a whole number in the way indicated and that would eliminate such problems as
    10*0.3 != 3
    linked issue 1
    A problem of the “feedback” facility in number plumber is that it doesn’t have a “don’t store the working”.

    At present, if you are using some recursive algorithm ( such as a repeated fraction for calculating the golden ratio ) then the shown working gets ridiculously long and eventually the interpreter gets it wrong. or gets impossibly slow.
    linked issue 2
    if there was another special data type of the form which preserved the last few digits of an operation while allowing the overall value to be expressed in exponential form such as 1e20 ends 986 +1 = 1e20…987
    (this occurs in Becky’s number plumber reported earlier)
    or just an alternative data form that throws away the most significant part of the number ( not the least significant part), when integer overflow is reached so it would look like ?12335235986

    linked issue 3
    it would be nice to have an input type “counting integer” rather than “random integer”. this would enable you to show convergence of an infinite series against the term number e.g. plot and tabulate the sum(0.9^n) against n for n=1,2,3,4,5,6 etc.

    David Seed

    February 2, 2012 at 12:44 am

    • I might want to work out 1/3 + 1/7 though?

      Will look at these useful NumberPlumber comments the next time the project is reopened on It will be recycled and updated at some point.


      February 2, 2012 at 10:24 am

      • 1/3 is already covered in the method described above.
        I’ve just realized that fractions of 2 are already covered as binary fractions.
        so we would only need to multiply our number by 9*5*7*11 =3465 to cover all fractions 1/n n=2..12

        If you do reopen the project another simple enhancement would be to be able to declare one of the several function inputs as being the one against which the graph is to be plotted, this would be helpful because the feedback input is not useful as a parameter of the function.

        also clearing the plot area does not clear plots for functions which have been deleted.


        David Seed

        February 2, 2012 at 10:29 pm

      • @David – yes, and so on ad infinitum – but that’s not the point I’m arguing here.

        With a little engineering it’s not difficult to make an algorithm that can do basic arithmetic with any fraction p/q where the integers p and q are only limited by performance and memory considerations rather than any hard limits. Sometimes (usually?) the floating point representation is the best. Occasionally, any representation designed for arithmetic purposes misses the point.

        However, the point I’m trying to make is that we must construct internal representations of numbers that are appropriate to the problems in hand, and that each internal implementation may capture some but certainly not all of the essence of 0.3. I think the same is true for our brains. We don’t ever hold the totality of mathematical knowledge about a number, so in a sense we don’t ever capture the concept entirely. This isn’t about decimal places and accuracy, it’s more about properties.


        Mike Pearson

        February 3, 2012 at 9:10 am

      • Brain states are bit philosophical for me.
        If you want an exact representation of any calculation such that the answer is of the form PI/2, or (1+sqrt(5))/2 or 6/257 or whatever then I think you might look at Mathematica which can do algebraic calculations.
        This could be adapted so that every prime number, surd and transcendental number is treated as an independent value until the final presentation of the number. But I think the computational cost of implementing something like this in an interpreted language such as JavaScript would be impracticable.

        Even limiting the objectives to fractions still leaves us with the task of frequently finding the LCM which requires efficient integer calculations, searching for integer (prime number) divisibility. Javascript only has a number datatype, ( I think this is a double length float) which is not efficient for the purpose.

        My primary aim is to have a tool which can make the sort of manipulations expected of students including hand calculations, calculator work and graphing, without presenting results that are clearly wrong.
        I think scaled double in the form I describe will be efficient and relatively easy to implement.

        David Seed

        February 3, 2012 at 8:12 pm

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

Join 110 other followers

%d bloggers like this: