-- if @fl_total != @int_total
-- select @error_message =
-- "@pt must be a multiple of @ts"
-- return 1
Sybase gets a little confused because floats are an approximate data type and errors are generated if, for example, @pt is set as 0.001 and @ts is set as 0.000001 but is then stored as 0.000009999999999, on the server. When the operation is done, with these values, Sybase complains that @pt isn't an exact multiple of @ts (i.e. 0.001 is not an exact multiple of 0.000009999999999).
Short of changing to a numeric data type for both values (this is being looked into but would involved an extremely large amount of work), does anyone have any ideas on how the operation could be done to eliminate the error?
I have looked into converting the values from floats, into numerics and then doing the calculation but this doesn't always work.
I have also looked into using the "round()" arithmetic operation and this works well until you start using larger values of @pt (arithmetic overflows and other errors occur).