Results 1 to 6 of 6
  1. #1
    Join Date
    Dec 2015
    Posts
    3

    Unanswered: Arithmetic overflow while calculating distance based on Latitude and longitude

    Hi,

    I am using following formula to calculate distance based on lat and long of two points.

    ACOS(COS(RADIANS(90-lat1)) *COS(RADIANS(90-lat2)) +SIN(RADIANS(90-lat1)) *SIN(RADIANS(90-lat2)) *COS(RADIANS(long1-long2))) *6371

    However, getting arithmetic overflow error in DB2.

    Please help.

    Thanks

  2. #2
    Join Date
    Apr 2014
    Location
    Kentucky
    Posts
    495
    Provided Answers: 24
    Then the target variable it too small for the result....
    Integer should be long.
    Single should be Double.
    Etc.

  3. #3
    Join Date
    Jul 2013
    Location
    Moscow, Russia
    Posts
    666
    Provided Answers: 55
    Hi,

    Can you provide the values which it happens on?
    Code:
    select ACOS(COS(RADIANS(90-lat1)) *COS(RADIANS(90-lat2)) +SIN(RADIANS(90-lat1)) *SIN(RADIANS(90-lat2)) *COS(RADIANS(long1-long2))) *6371
    from table(values 
    (?, ?, ?, ?)
    ) t (long1, long2, lat1, lat2)
    Insert them in place of the question marks.
    Regards,
    Mark.

  4. #4
    Join Date
    Dec 2015
    Posts
    3
    Thanks for the immediate response. I have changed the data types of lat1, lat2, long1,long2 to double. However, still facing same issue.

  5. #5
    Join Date
    Dec 2015
    Posts
    3
    I am new to db2, request your help

  6. #6
    Join Date
    Jan 2003
    Posts
    4,292
    Provided Answers: 5
    What DB2 version and OS are you using? For what values of lat1, lat2, long1,long2 do you get the eror?

    Andy

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •