Results 1 to 4 of 4
  1. #1
    Join Date
    May 2003
    Posts
    26

    Unanswered: HELP - Calculation Error

    PLEASE HELP ...................I have a simple table with two fields both defined as Field Size = Double and Decimal Places = Auto for input of numeric values.

    When using an expression in a query to calculate the variance between these two fields I receive strange results on some, but not all records. A snapshot of the results are listed below. The expression is [FieldA]-[FieldB] .... has anyone run accross this before?

    Field A - Field B = Result
    123345.87 - 123345.87 = 1.45519152283669E-11
    1677.72 - 1677.72 = 2.27373675443232E-13
    737.99 - 737.99 = 0

  2. #2
    Join Date
    Feb 2004
    Posts
    8
    try to include format in your expression like this:

    format(field1-field2,"fixed")

  3. #3
    Join Date
    May 2003
    Posts
    26
    I've tried changing to various formats both in the expression and in field size type with the same results. Thanks anyway.

  4. #4
    Join Date
    Sep 2003
    Location
    MI
    Posts
    3,713

    Re: HELP - Calculation Error

    Originally posted by ramonas
    PLEASE HELP ...................I have a simple table with two fields both defined as Field Size = Double and Decimal Places = Auto for input of numeric values.

    When using an expression in a query to calculate the variance between these two fields I receive strange results on some, but not all records. A snapshot of the results are listed below. The expression is [FieldA]-[FieldB] .... has anyone run accross this before?

    Field A - Field B = Result
    123345.87 - 123345.87 = 1.45519152283669E-11
    1677.72 - 1677.72 = 2.27373675443232E-13
    737.99 - 737.99 = 0
    Ever hear of floating point error???? Here's an option for you:

    dim NewDouble As Double

    NewDouble= CDbl((CLng(FieldA * 100)-CLng(FieldB *100))/100)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •