If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Data Access, Manipulation & Batch Languages > Delphi, C etc > Visual Baisc Rounding, Precision, Scale

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 11-03-03, 11:23
aleeali aleeali is offline
Registered User
 
Join Date: Sep 2003
Location: New Jersey
Posts: 4
Red face Visual Baisc Rounding, Precision, Scale

I am retrieving a number from a database field. The number I retrieve is 0.002222222222222. I need to multiply this number by 100. When I multiply this number by 100 the result becomes 0.2222222222222. There are 13 digits to the right of the decimal.

I need to see 15 digits to the right of the decimal i.e. 0.222222222222222. Any ideas how I can solve this issue, and see 15 places to the right of the decimal point. I need to see 0.222222222222222 and not 0.222222222222200

Any ideas.

Thanks.

Alee
Reply With Quote
  #2 (permalink)  
Old 11-05-03, 09:22
vududoc vududoc is offline
Registered User
 
Join Date: Jul 2001
Location: NC
Posts: 102
If I remember my precision rules correctly, and its been years since I've needed to use them, 100 X .0022222etc equals 0.2222, since you don't know the precision of the number 100....it could be rounded from 100.1 or 99.9. Now 100.0 X 0.00222etc would give you 0.22222 Just because a computer gives you upteen decimal places doesn't mean that the answer has the correct precision.

You didn't indicate processor, operating system or software you were using when you got the error, so its hard to help you. But, did you remember to dimension both numbers as double? Try multiplication with both numbers having the same number of digits.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On