Results 1 to 2 of 2
  1. #1
    Join Date
    Sep 2003
    New Jersey

    Red face Unanswered: Visual Baisc Rounding, Precision, Scale

    I am retrieving a number from a database field. The number I retrieve is 0.002222222222222. I need to multiply this number by 100. When I multiply this number by 100 the result becomes 0.2222222222222. There are 13 digits to the right of the decimal.

    I need to see 15 digits to the right of the decimal i.e. 0.222222222222222. Any ideas how I can solve this issue, and see 15 places to the right of the decimal point. I need to see 0.222222222222222 and not 0.222222222222200

    Any ideas.



  2. #2
    Join Date
    Jul 2001
    If I remember my precision rules correctly, and its been years since I've needed to use them, 100 X .0022222etc equals 0.2222, since you don't know the precision of the number could be rounded from 100.1 or 99.9. Now 100.0 X 0.00222etc would give you 0.22222 Just because a computer gives you upteen decimal places doesn't mean that the answer has the correct precision.

    You didn't indicate processor, operating system or software you were using when you got the error, so its hard to help you. But, did you remember to dimension both numbers as double? Try multiplication with both numbers having the same number of digits.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts