Math Bug?

Results 1 to 3 of 3

Thread: Math Bug?

  1. #1
    Join Date
    Dec 1969

    Default Math Bug?

    &#062;Could you tel me why when I execute this calcul, I get &#062;3.99999999999636E-02 insted of getting 0.04???<BR>&#062;<BR>&#062;Tmp = 2254.27 + -2254.23<BR>&#062;Response.write Tmp<BR>&#062;<BR>&#062;The output is: 3.99999999999636E-02<BR><BR>This question was posted on 4/20 by Oli ... The problem is easily fixed by rounding the number ... but Why do you have to round?!<BR><BR> 2254.27 - 2254.23 IS .04 not matter how you look at it...<BR><BR>I did a little testing and it seems to only behave this way when the integer part of the difference is less than 100 AND the integer parts of the two numbers being subtracted have more than 2 digits ...<BR><BR>ASP Running on IIS 4.0/NT4.0 sp 6<BR><BR>Weird ...

  2. #2
    Mark Guest

    Default RE: Math Bug?

    Welcome to the real world of binary numbers on binary machines....<BR><BR>Keep in mind that all numbers in a digital computer are stored in a binary format (0&#039s and 1&#039s). When mathamatical computations are performed with such binary numbers, especially with non-integer type numbers (variants), the resultant of the mathamatical computation yields a very close "approximate" value, not necessarily a true or absolute value. This is especially true with floating-point type numbers (such as your problem).<BR><BR>From a Mathamaticians perspective 0.03999999999... is equivalent to 0.04.<BR><BR>For example:<BR>1/9 = 0.11111111111...<BR>2/9 = 0.22222222222...<BR>3/9 = 0.33333333333...<BR>4/9 = 0.44444444444...<BR>5/9 = 0.55555555555...<BR>6/9 = 0.66666666666...<BR>7/9 = 0.77777777777...<BR>8/9 = 0.88888888888...<BR>9/9 = 0.99999999999... <BR><BR>Common sense tells us that 9/9 equals unity or 1.<BR><BR>Hence, 0.9999999999... = 1 (mathamatical definition)<BR><BR>The same thinking applies to mathamatical computations of real non-integer numbers on a binary computer.<BR><BR>As a programmer, you make remember the above.<BR><BR>I know...I spent many hours trying to debug code on a "Conditional If" statement. After pulling my hair out, I soon realized that when a binary machine (computer) performs mathamatical computations on non-integer numbers, the results are often not what you would expect.<BR><BR>If you ever make "Conditional Tests" on non-integer computations, then you must remember that computers are 100% perfect. They sometimes give you a very close answer, not a totally accurate absolute answer. Hence, your Conditional Tests must consider this fact.<BR><BR>From an Engineering perspective, your two numbers (2254.27 and 2254.23) are only significant to 0.01. So, your final answer should also be no more significant than 0.01. Hence, your final answer of 3.99999999999636E-02 is technically 3.99 +/- 0.01. The correct or absolute answer of 0.04 is within this range. No Engineering problem!<BR><BR>This really isn&#039t a just the nature of the beast (the binary computer).<BR><BR>I hope this helps.....whew<BR><BR>

  3. #3
    Join Date
    Dec 1969

    Default Thank you Mark , very informative.

    Thank you Mark , very informative.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts