Math question

1. Senior Member
Join Date
Dec 1969
Posts
259

Math question

Hello,<BR><BR>I have the following function:<BR><BR>Sub testb()<BR>Dim numa As Integer<BR>Dim resulta As String<BR> numa = 100<BR> resulta = ((2 ^ numa) - 2) / numa<BR> Debug.Print resulta<BR> If Int(resulta) = resulta Then<BR> Debug.Print "Prime: " & numa<BR> End If<BR>End Sub<BR><BR>How do I make resulta show as the full number instead of<BR>1.26765060022823E+28<BR>?<BR><BR>Thanks!<BR> Andy

2. Senior Member
Join Date
Dec 1969
Posts
96,118

You can't

VBScript has only 4 kinds of numbers that it understands:<BR> Short integers: -32768 to +32767<BR> Long integers: +/- 2 billion or so<BR> Float: floating point numbers with about 6 decimal digits of accuracy<BR> Double: floating point numbers with about 14 decimal digits of accuracy<BR><BR>So when you calculate a number like that, one that needs 29 digits of accuracy, you *can* force it to print all 29 digits, but only the first 14 or 15 will be accurate and the rest will either be all zeroes or pure nonsense (depending on how you force the printing).<BR><BR>This limitation is not unique to VBScript. Most computer languages use those same 4 basic types (some add a "long long integer which will give about 18 digits of accuracy). To go beyond that you need a software library that is capable of more.<BR><BR>The .NET framework has such a library, but I don&#039;t recall what it&#039;s limits of accuracy are, offhand. I think it will go to 28 digits, but I don&#039;t remember how much beyond that.<BR><BR>

3. JLM
Junior Member
Join Date
Dec 1969
Posts
1

RE: you need to constuct the right algorithm

While the language itself may not be able to calculate and display more decimal places I am sure that someone can construct an algoithim to calculate more decimal places, while you may have to construct the answer as a string I suspect the limitation is only down to the inbuilt structures. Ie if you want to just calculate the 29 and 30th places of the answer and display them then I would be surprised if there is not a numerical technique for doing so. I rem from numerical analysis calculations of errors and series approximations, unfortunately my expertise is in flash and I my maths is also not up to it but there is probably a way if you need it enough, desk top computers these days have huge computational abilities which only games progammers probably truely tax. <BR><BR>JLM

4. Senior Member
Join Date
Dec 1969
Posts
96,118

Oh, of course you are right

In fact, if you really wanted to, you could write a set of "string arithmetic" functions. I&#039;ve done that a long time ago.<BR><BR>But I figured if this person didn&#039;t know the fundamental data types of the language, then he likely isn&#039;t up to creating a string arithmetic package. Doing so and getting reasonable results is not a trivial process. Albeit, since he only needs to work with integers, that makes it a lot simpler. But even with integers only, a division operation using string arithmetic will take a while to get working correctly, I&#039;d bet.<BR><BR>Anyway, I just think that trying to use VBScript for this purpose is a mistake. Especially since the .NET framework Decimal data type will do what he wants:<BR>http://msdn.microsoft.com/library/en-us/cpref/html/frlrfSystemDecimalClassTopic.asp?frame=true<BR><BR >Granted, it *ONLY* goes up to 10^28, but still...<BR><BR>

Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•