
Math Bug?
>Could you tel me why when I execute this calcul, I get >3.99999999999636E02 insted of getting 0.04???<BR>><BR>>Tmp = 2254.27 + 2254.23<BR>>Response.write Tmp<BR>><BR>>The output is: 3.99999999999636E02<BR><BR>This question was posted on 4/20 by Oli ... The problem is easily fixed by rounding the number ... but Why do you have to round?!<BR><BR> 2254.27  2254.23 IS .04 not matter how you look at it...<BR><BR>I did a little testing and it seems to only behave this way when the integer part of the difference is less than 100 AND the integer parts of the two numbers being subtracted have more than 2 digits ...<BR><BR>ASP Running on IIS 4.0/NT4.0 sp 6<BR><BR>Weird ...

RE: Math Bug?
Welcome to the real world of binary numbers on binary machines....<BR><BR>Keep in mind that all numbers in a digital computer are stored in a binary format (0's and 1's). When mathamatical computations are performed with such binary numbers, especially with noninteger type numbers (variants), the resultant of the mathamatical computation yields a very close "approximate" value, not necessarily a true or absolute value. This is especially true with floatingpoint type numbers (such as your problem).<BR><BR>From a Mathamaticians perspective 0.03999999999... is equivalent to 0.04.<BR><BR>For example:<BR>1/9 = 0.11111111111...<BR>2/9 = 0.22222222222...<BR>3/9 = 0.33333333333...<BR>4/9 = 0.44444444444...<BR>5/9 = 0.55555555555...<BR>6/9 = 0.66666666666...<BR>7/9 = 0.77777777777...<BR>8/9 = 0.88888888888...<BR>9/9 = 0.99999999999... <BR><BR>Common sense tells us that 9/9 equals unity or 1.<BR><BR>Hence, 0.9999999999... = 1 (mathamatical definition)<BR><BR>The same thinking applies to mathamatical computations of real noninteger numbers on a binary computer.<BR><BR>As a programmer, you make remember the above.<BR><BR>I know...I spent many hours trying to debug code on a "Conditional If" statement. After pulling my hair out, I soon realized that when a binary machine (computer) performs mathamatical computations on noninteger numbers, the results are often not what you would expect.<BR><BR>If you ever make "Conditional Tests" on noninteger computations, then you must remember that computers are 100% perfect. They sometimes give you a very close answer, not a totally accurate absolute answer. Hence, your Conditional Tests must consider this fact.<BR><BR>From an Engineering perspective, your two numbers (2254.27 and 2254.23) are only significant to 0.01. So, your final answer should also be no more significant than 0.01. Hence, your final answer of 3.99999999999636E02 is technically 3.99 +/ 0.01. The correct or absolute answer of 0.04 is within this range. No Engineering problem!<BR><BR>This really isn't a bug....it's just the nature of the beast (the binary computer).<BR><BR>I hope this helps.....whew<BR><BR>

Thank you Mark , very informative.
Thank you Mark , very informative.
Posting Permissions
 You may not post new threads
 You may not post replies
 You may not post attachments
 You may not edit your posts

Forum Rules

