If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > BIGINT to DECIMAL and Math

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 05-10-07, 13:57
djschmitt djschmitt is offline
Registered User
 
Join Date: Nov 2002
Location: Omaha, NE
Posts: 23
BIGINT to DECIMAL and Math

I have a few BIGINT columns I would like to add and then divide producing a percentage. When I do this in SQL, I receive a result of 0. When I do it with a calculator, it comes out to .97 or something. I am certain I am selecting only one record.

This returns 0

select
a / (a + b) as foo
from
bar

This returns 0.000

select
decimal(a / (a + b), 4, 3) as foo
from
bar

Thank you for your assistance.
Reply With Quote
  #2 (permalink)  
Old 05-10-07, 15:31
djschmitt djschmitt is offline
Registered User
 
Join Date: Nov 2002
Location: Omaha, NE
Posts: 23
Smile

I knew this was something a little "Infected Mushroom" could help clear up

select
decimal(float(a) / (float(a) + float(b)), 4,3)
from
foo

returns .997
Reply With Quote
  #3 (permalink)  
Old 05-11-07, 04:19
Wim Wim is offline
Registered User
 
Join Date: Nov 2004
Posts: 1,279
Bigints do integer calculus:

3/3 = 1
6/3 = 2
5/3 = 2 the rest is lost
a bigint can't represent 0.99, only discrete values like 1 or 0
__________________
With kind regards . . . . . SQL Server 2000/2005/2008/2008 R2 Earned beers: 16
Wim
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald Knuth
Grabel's Law: 2 is not equal to 3 -- not even for very large values of 2.
Pat Phelan's Law: 2 very definitely CAN equal 3 -- in at least two programming languages
Reply With Quote
  #4 (permalink)  
Old 05-11-07, 07:45
umayer umayer is offline
Registered User
 
Join Date: Dec 2005
Posts: 273
simply convert one of the arguments ( not the result ! ) to DECIMAL:

SELECT
DECIMAL(a) / (a+b)
FROM
bar
Reply With Quote
  #5 (permalink)  
Old 05-11-07, 16:39
stolze stolze is offline
Registered User
 
Join Date: Jan 2007
Location: Jena, Germany
Posts: 2,662
Quote:
Originally Posted by Wim
Bigints do integer calculus:

3/3 = 1
6/3 = 2
5/3 = 2 the rest is lost
a bigint can't represent 0.99, only discrete values like 1 or 0
Actuall, integer calculus means that 5/3 = 1. First you divide and then truncate fractions. There is no rounding involved.
__________________
Knut Stolze
IBM DB2 Analytics Accelerator
IBM Germany Research & Development
Reply With Quote
  #6 (permalink)  
Old 05-13-07, 06:04
Wim Wim is offline
Registered User
 
Join Date: Nov 2004
Posts: 1,279
Quote:
Actuall, integer calculus means that 5/3 = 1.
Indeed, my error.
__________________
With kind regards . . . . . SQL Server 2000/2005/2008/2008 R2 Earned beers: 16
Wim
Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald Knuth
Grabel's Law: 2 is not equal to 3 -- not even for very large values of 2.
Pat Phelan's Law: 2 very definitely CAN equal 3 -- in at least two programming languages
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On