Here's a fun one...

# Thread: Here's a fun one...

1. Senior Member
Join Date
Dec 1969
Posts
96,118

## Here's a fun one...

Akhilesh started this, but maybe others would be interested:<BR><BR>http://www.aspmessageboard.com/forum/asp.asp?M=325048&P=1&F=20

2. Senior Member
Join Date
Dec 1969
Posts
11,334

## You cheated

You copied this from a previous post several months ago ;)<BR><BR>http://www.aspmessageboard.com/forum/showMessage.asp?M=278406&F=27&P=1

3. Senior Member
Join Date
Dec 1969
Location
Los Angeles, CA
Posts
21,192

## Very possible

and you never said WHY<BR><BR>it is cause the precision of VB (VBS also) is a bit off.<BR><BR>try the same in SQL<BR><BR>declare @s decimal<BR>declare @i int<BR>set @s = 0.0<BR>set @i = 1<BR>while @i &#060; 10<BR>begin<BR>set @s = @s + 0.1<BR>set @i = @i+1<BR>end<BR>select 1 - @s

4. Senior Member
Join Date
Dec 1969
Location
Los Angeles, CA
Posts
21,192

## Oh jeez that does not work

have to see what i did wrong

5. Senior Member
Join Date
Dec 1969
Location
Los Angeles, CA
Posts
21,192

## DOH!

ok had decimal there<BR><BR>try this<BR><BR>declare @s float<BR>declare @i int<BR>set @s = 0.0<BR>set @i = 0<BR>while @i &#060; 10<BR>begin<BR> set @s = @s + 0.1<BR> set @i = @i+1<BR>end<BR>select @s<BR><BR><BR>compare the answer with this<BR><BR>select (.1+.1+.1+.1+.1+.1+.1+.1+.1+.1)<BR><BR><BR>QED my precisioin thingie<BR><BR>

6. Senior Member
Join Date
Dec 1969
Posts
96,118

## So I guess that means...

...that I may or may not be clever, but I&#039;m sure as hell senile?<BR><BR>&#060;sigh /&#062;<BR><BR>

7. Senior Member
Join Date
Dec 1969
Posts
96,118

## DOH, indeed!

while @i &#060; 10<BR>begin<BR>set @s = @s + 0.1<BR>set @i = @i+1<BR>end<BR><BR>compare the answer [@s] with this<BR>select (.1+.1+.1+.1+.1+.1+.1+.1+.1+.1)<BR><BR>*********** ************<BR><BR>Huh? Those are exactly the same thing! 0.1 added to itself 10 times. Whoopee. Of course they give the same answer!<BR><BR>Do *THIS* and give me what you see!<BR><BR>while @i &#060; 10<BR>begin<BR>set @s = @s + 0.1<BR>set @i = @i+1<BR>end<BR>select (@s-1.0)<BR><BR><BR>

8. Senior Member
Join Date
Dec 1969
Location
Los Angeles, CA
Posts
21,192

## No...NOT really

&#062;Those are exactly the same thing! 0.1 added to itself 10 times. <BR>&#062;Whoopee. Of course they give the same answer!<BR>They give 2 DIFF answers<BR><BR><BR>VB does NOT do the same...in vb they act diff cause that pricition thingie kicks in.<BR><BR>try these 2 in SQL...they give DIFF answers<BR><BR>declare @s float<BR>declare @i int<BR>set @s = 0.0<BR>set @i = 0<BR>while @i &#060; 10<BR>begin<BR>set @s = @s + 0.1<BR>set @i = @i+1<BR>end<BR>select 1-@s<BR><BR>select 1-(.1+.1+.1+.1+.1+.1+.1+.1+.1+.1)<BR><BR>the first method uses the precision of the float datatype and the second one does not...notice the diff answers??<BR><BR>now try this in vb with your example<BR><BR>a = 1-(0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1)<BR><BR>THOSE 2 give the same answers....care to explain why?<BR><BR><BR>

9. Senior Member
Join Date
Dec 1969
Posts
96,118

## I have neither VB nor SQL Server...

...up and running here.<BR><BR>SO...<BR><BR>&#062; notice the diff answers??<BR><BR>Nope. My mental SQL Server processor isn&#039;t working that well.<BR><BR>&#062; now try this in vb with your example<BR><BR>Can I use your machine?<BR><BR>So, please, tell us what you see for all those results?<BR><BR>

10. Senior Member
Join Date
Dec 1969
Posts
96,118

## Compile time vs. run-time

I&#039;m not a VB afficianado or authority, but in *other* languages, this kind of difference can easily occur because of the difference between compile-time evaluation of *CONSTANT* expressions and run-time evaluation.<BR><BR>I could easily imagine that <BR><BR>a = 1-(0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1 + 0.1)<BR><BR>would be entirely evaluated at compile time and you&#039;d end up with a zero answer.<BR><BR>After you show me what SQL Server and VB are doing to those, maybe I can comment further. But maybe not. I&#039;d be guessing, no matter what, without having the software here to test with.<BR><BR><BR>

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•