simple mathematic calculation

shatishr · #1 (**permalink**) 01-21-06, 09:07

hie all

i created a test table and inserted '54' into nm1 column and '62' into nm2 column

i wanted to do a calculation which is 'nm2-nm1/nm1' in other words (62-54)/62 which should return 0.12

but its returning some other result...

any help ?

andrewst · #2 (**permalink**) 01-21-06, 10:59

What result did you get? Not 61 by any chance?

If you look carefully at your calculation you will see it is actually 62-54/54, which is evaluated as 62-(54/54) which is 61.

WilliamR · #3 (**permalink**) 01-21-06, 11:07

nm2-nm1/nm1 is not the same as (62-54)/62! Check the rules for arithmetic precedence.

This will be the same in other languages, for example (just for fun) in Korn shell:

Code:

/Users/williamr: float nm1=54
/Users/williamr: float nm2=62
/Users/williamr: echo $(( nm2-nm1/nm1 ))
61
/Users/williamr: echo $(( (nm2-nm1)/nm2 ))
0.129032258065

Update: Tony beat me to it.

I've just noticed that your formula varies:

Quote:

Originally Posted by shatishr

i created a test table and inserted '54' into nm1 column and '62' into nm2 column.

i wanted to do a calculation which is 'nm2-nm1/nm1' in other words (62-54)/62

If nm1=54 and nm2=62, shouldn't that be (62-54)/54?

Code:

/Users/williamr: echo $(( (nm2-nm1)/nm1 ))
0.148148148148

Internet Services

Web Development

Digital Marketing

Consumer Tech