PrintNW Posted February 15, 2013 Share Posted February 15, 2013 I have a calculation script that is meant to subtract one dollar amount from another and then format the result into either ##¢ (for cents results) and $##.## (for dollar results). I've worked out the subtraction, but the formatting is getting thrown off because the the result of the subtraction id not correct. For example, when I subtract 6.49-5.89 the result is 0.6000000000000005 and when I subtract 7.00-5.89 the result is 1.1100000000000003. This all occurs even before my currency formatting script begins. Can someone please look at the code below and help me determine how to correct it? Or, if possible, provide me with an alternative script to accomplish what I want? Thanks in advance. Lee Francis, Print NW var prc1 = Field("LRG Cheese Pizza Only Price"); var prc2 = Field("Base Price - Pepperoni"); var totalAmount = StringToNumber(prc1 - prc2); // Used only to see how FPD was calculating // return totalAmount if (totalAmount.length < 3) return FormatNumber("##", totalAmount)+'¢'; else return '$'+ FormatNumber("#,###.##", totalAmount); Quote Link to comment Share on other sites More sharing options...
Dan Korn Posted February 15, 2013 Share Posted February 15, 2013 Computerized floating-point math operations are always inaccurate. Most banking software uses integral numbers of cents in such computations to avoid this. So basically you multiply by 100 to get the value in cents, as an integer, and base all your internal storage and calculations on that integer value, and then format values back to fractional dollars only for display. Your code which reports the results as cents should be multiplying by 100 anyway, so that you report, say, "50¢" instead of ".50¢". (When I see signs with prices like ".50¢", I like to give the cashier a penny and say, "Keep the change.") At any rate, the calculation to determine whether to present the price in cents shouldn't be based on the length of the string representation of the number; it should be based on the numeric value itself. That will still work right, regardless of any floating-point math inaccuracies. Then the FormatNumber function will clean everything up at the end. So I think you want to do this: if (totalAmount < 1) return FormatNumber("##", totalAmount*100)+'¢'; else return '$'+ FormatNumber("#,###.##", totalAmount); Quote Link to comment Share on other sites More sharing options...
PrintNW Posted February 15, 2013 Author Share Posted February 15, 2013 Thanks Dan! That did the trick. Quote Link to comment Share on other sites More sharing options...
Dan Korn Posted February 15, 2013 Share Posted February 15, 2013 Thanks Dan! That did the trick. Glad I could help. But now I'm hungry for a pepperoni pizza. Quote Link to comment Share on other sites More sharing options...
PrintNW Posted February 15, 2013 Author Share Posted February 15, 2013 Okay, new issue. My number formatting is not working correctly. If I use the code as you supplied it, the price cuts off if the price ends in zeros (so that $1.10 appears as $1.1). I tried correcting this by changing the format to look like this: return '$'+ FormatNumber("#,###.00##", totalPrice); But that causes the amount to read as $1.10000 Any ideas on what I can do? Quote Link to comment Share on other sites More sharing options...
Dan Korn Posted February 15, 2013 Share Posted February 15, 2013 Okay, new issue. My number formatting is not working correctly. If I use the code as you supplied it, the price cuts off if the price ends in zeros (so that $1.10 appears as $1.1). I tried correcting this by changing the format to look like this: return '$'+ FormatNumber("#,###.00##", totalPrice); But that causes the amount to read as $1.10000 Any ideas on what I can do? Change it to this: return '$'+ FormatNumber("#,###.00", totalPrice);I still want a pizza! Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.