Results 1 to 2 of 2
  1. #1
    Join Date
    Mar 2012

    Unanswered: Sybase can't handle large math calcul

    declare @size float
    select @totalsize=(894720*4096/1048576)
    Result in Arithmetic overflow occured.

    Having any idea as to how to fix this, my simple tabletop calculator can handle such calcul but not a powerfull db like Sybase ... !?

  2. #2
    Join Date
    Mar 2012
    I have to be honest, this situation was so bugging me that I crossed posted this question. Fortunately for me I got the right answer and thought it would be a goog idea to share the answer with you.

    Even though I declared the holding variable (@size) as float, the RDBMS was not using it until the end of the math calculus.

    So, if I take every step of the calculus it would look like that : (894720*4096/1048576) = (int*int/int)

    The fact is, the first part, int*int = 894720*4096 is already higher that what an interger can contain thus the "Arithmetic overflow occured"....

    Two methods to solve this issue :

    First, imply float by adding .0, like this : (894720.0*4096/1048576) this way, RDBMS starts the calculus using float instead in integer (float*int = float / int = float).

    Second, use convert : (convert(float, 894720)*4096/1048576) same behavior from the RDBMS, starts the calculus using float, not integer.

    I use the second method as I don't alway know what will be the value.

    Hope this will help someone someday, it sure as hell helped me.


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts