If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > exponential value

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 11-29-11, 21:53
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
exponential value

Could you please explain why the value is not stored/displayed correctly?

Last edited by db2girl; 11-30-11 at 16:56.
Reply With Quote
  #2 (permalink)  
Old 11-29-11, 22:46
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Floating point numbers (FLOAT and DOUBLE) are approximate. In the second example you convert the BIGINT literal twice - from BIGINT to DOUBLE, then from DOUBLE to DECIMAL, so I'd think some loss of precision might be expected.

It would be good to know what the customer think are "correct" and "incorrect" results.
Reply With Quote
  #3 (permalink)  
Old 11-29-11, 23:14
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
In this example, customer thinks scale !=0 is incorrect:

"Correct" result:
444444444444444.00

"Incorrect" results:
444444444444444.50
444444444444444.10


In their case, the column is defined as dec (31,2) - 1st example.


I'm attaching another doc with some additional info provided by the customer.

Do you think what they're seeing is expected (but zOS results are different from LUW), db2 bug, jdbc bug...?

Last edited by db2girl; 11-30-11 at 16:55.
Reply With Quote
  #4 (permalink)  
Old 11-29-11, 23:51
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by db2girl View Post
In this example, customer thinks scale !=0 is incorrect:

"Correct" result:
444444444444444.00

"Incorrect" results:
444444444444444.50
444444444444444.10


In their case, the column is defined as dec (31,2) - 1st example.

I might be wrong as I've had a couple of beers now, but the FLOAT literal 4.4444444444444444E14 seems to be the same as 444444444444444 plus something around 0.44 (floating point numbers are approximate), which nicely fits into DECIMAL(31,2), so 444444444444444.00 would definitely be incorrect, z/OS or not.

Now, JDBC Double supports only 15 digits for the mantissa, which does not mean that it would truncate the extra digits - rather, if you supply more than 15 digits for the mantissa (which they do) only the first 15 will be defined.

Can your customer explain why they think 444444444444444.00 is the correct decimal representation of 444444444444444.44?

Normally you would never use floating point data type for currency (money) variables - only decimals.
Reply With Quote
  #5 (permalink)  
Old 11-30-11, 09:39
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
You're correct - 4.4444444444444444E14 is 444444444444444.44 so I'm not sure why they think 444444444444444.00 is correct.


Using their second example of inserting
444444444444444.00


Why does select return?
444444444444444.10 - LUW
444444444444444.00 - zOS


I'm not sure I completely understand about "JDBC Double supports only 15 digits for the mantissa..."
Reply With Quote
  #6 (permalink)  
Old 11-30-11, 10:22
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Mantissa holds the significant digits of a number, so if you try to store in a Double something that needs more than 15 digits, it will be approximated to a number that has 15 significant digits. "4.4444444444444444E14" has 17 significant digits, and if you were to assign it to a Double, it would turn into ".444444444444444E15" (15 digits).

As for the difference between the LUW and z/OS representation of the DOUBLE 444444444444444.00, I guess it might be attributed to the different floating point implementation that depends on the hardware; both numbers are approximately the same, and that is correct by the definition of the DOUBLE: "A double-precision floating-point number is a 64-bit approximation of a real number. "
Reply With Quote
  #7 (permalink)  
Old 11-30-11, 11:26
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Thanks a lot, Nick. Very helpful.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On