IMPORTANT ANNOUNCEMENT

These forums were permanently set to read-only mode on July 20, 2022. From that day onwards, no new posting or comment is allowed on the site, but the historical content remains intact and searchable.

A new location for posting questions about PlanetPress Suite is now available:

OL Learn - PlanetPress Classic (opens in new tab)

Topic Options
#37758 - 11/10/11 03:45 PM adding a float numbers
dwal Offline
OL Newbie

Registered: 11/10/11
Posts: 19
I am adding up a several numbers that have 3 places after the decimal. It is postage like 0.640 and 0.515 and 0.440. I am using the command strtofloat to convert to a "measure". The results that I am getting are like 4.3499999. The amount seems to be correct, but how can I be getting so many decimal places when I am adding numbers up that only have 3 decimal places. Any help in understanding this would be greatly appreciated.

Thanks

Top
#37760 - 11/10/11 04:33 PM Re: adding a float numbers [Re: dwal]
Raphael Lalonde Lefebvre Offline
OL Expert

Registered: 10/14/05
Posts: 4956
Loc: Objectif Lune Montreal
dwal,

When I add up those three numbers you gave me, I get 1.595. How are you getting 4.3499999? Are you adding more than those three values?

If you are, have you checked all of those values, and made sure one of them did not end up having a lot of decimals? Are you just adding, or you're also multiplying?

I think we need a bit more details about what numbers you're using, and how you're adding them up, to better understand this issue.

Regards,
Raphaël Lalonde Lefebvre

Top
#37768 - 11/10/11 10:12 PM Re: adding a float numbers [Re: Raphael Lalonde Lefebvre]
dwal Offline
OL Newbie

Registered: 11/10/11
Posts: 19
Raphael,

I have a variable defined as a measure.
Here is some of the code-
&field6:=trim(regionline(1.63,9.29,1.97,9.43))
If(&conditionA)
&dollar1:=&dollar1+(strtofloat(&field6))
else
&dollar2:=&dollar2+(strtofloat(&field6))
endif

&dollar2 is at 7.835 on the next record &field6 is 0.640.
The value of should be 8.475, but instead &dollar2 is 8.47999.
I am doing no multiplication.

Any ideas?

Thanks

Top
#37770 - 11/11/11 12:39 AM Re: adding a float numbers [Re: dwal]
Yannick Fortin Offline
OL Expert

Registered: 08/25/00
Posts: 354
Loc: Objectif Lune Montréal
Are you trying to use the measure type to make currency calculations? That's a VERY bad idea. There is another data type called Currency that can be used for, well, dealing with currency or any fixed-point value. I very strongly suggest that you change your code to use the Currency type instead, or, even better, do currency calculations at the source if possible.

As you experienced, the measure type (really, a single-precision floating point number) does not round up as humans would expect. Furthermore, it only has 7-8 significan digits. That means that, for a measure, 700000000 is the same as 700000001. There is a reason why it is called a "measure", after all.

There are tons of web sites that explain the why's, do's and don'ts of floating point number, so I won't bother. But if you really want to dig, the Wikipedia article on floating point is a good start.
_________________________
Yannick Fortin, Team OL

Top
#37771 - 11/11/11 08:15 AM Re: adding a float numbers [Re: Yannick Fortin]
dwal Offline
OL Newbie

Registered: 11/10/11
Posts: 19
I had started off using the currency format. However, my totals were low. I believed this to be the case as some of the amounts include a 1/2 a penny. For example, I was adding numbers like 0.515

Does the currency format allow me to add up 3 decimal places instead of just two?

Top
#37772 - 11/11/11 09:25 AM Re: adding a float numbers [Re: dwal]
Raphael Lalonde Lefebvre Offline
OL Expert

Registered: 10/14/05
Posts: 4956
Loc: Objectif Lune Montreal
dwal,

If I just use this code:

Code:
define(&dollar2, measure, 0.0)
define(&field6, measure, 0.0)

&dollar2 := 7.835
&field6 := 0.640

&dollar2:=&dollar2+&field6

show(floattostr(&dollar2))


And force the values at 7.835 and 0.640, I get the proper value, 8.475. I am thinking that the value you see in your database field "field6" might have a lot of digits after the decimal point, and it's having troubles rounding it as a result, for the reasons that Yannick explained. What's the actual value of the field?

And currencies only allow two digits to be displayed. It's meant to display dollar amounts, which only have two digits.

Regards,
Raphaël Lalonde Lefebvre

Top
#37775 - 11/11/11 04:06 PM Re: adding a float numbers [Re: Raphael Lalonde Lefebvre]
dwal Offline
OL Newbie

Registered: 11/10/11
Posts: 19
I am printing out the variable &field6 which is a string variable and I only have 3 digits after the decimal point. When print out the string variable &field6 the value is 0.640.

Top