More trouble rounding to 2dp

Results 1 to 2 of 2

Thread: More trouble rounding to 2dp

  1. #1
    Join Date
    Dec 1969

    Default More trouble rounding to 2dp

    Sorry for starting a new post on a topic discussed already bit further down but anyway using Bill&#039;s function - a enhanced version of mine<BR><BR>RoundToLarger(69.475, 2) gives the unexpected 69.47 (or same as normal Round 69.47) when wanted 69.48<BR><BR>This effectively means that Int(69.475*100+0.5)/100 gives 69.47<BR><BR>or rather 69.475*100+0.5 is fractionally less than 6948<BR><BR>Yes indeed a Response.Write (69.475*100+0.5) - 6948<BR>yields -9.09...E-13 not zero as one might hope!<BR><BR>Maybe then I need to use<BR>something like Int(69.475*100+0.500001)/100 ???<BR><BR>Umm FORTRAN never seemed this bad doing sums!!!<BR><BR>Cheers<BR>Tim

  2. #2
    Join Date
    Dec 1969

    Default Fortran had same problem...

    ...but you didn&#039;t see it becuz it always displayed *all* digits of a result, by default.<BR><BR>VBS *automatically* rounds numbers you see to about 12 to 14 digits. So your number is *REALLY*<BR> 69.4749999999998<BR>or something like that. And when you multiply it by 100, you still get<BR> 6947.49999999998<BR>and add 0.5 and you get<BR> 6947.99999999998<BR><BR>It&#039;s all because most decimal fractions can&#039;t be represented exactly in binary floating point.<BR><BR>Yeah, your hack is probably as good as you&#039;ll get.<BR><BR>

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts