|
Home > Archive > BizTalk Server General > December 2005 > multiplication bug??
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
multiplication bug??
|
|
|
| first off, i've been, ahem, handed biztalk admin role with zero
experience and am really just fixing bugs as i can so i apologize if
i'm missing something entirely. secondly, i'm running bt2002 with
hipaa accelerator. i need to get some dollar amounts into a 9(5)v99
format. so, i'm just running it through the multiplication functoid,
multiplying by 100. this works fine. however, the value of 629.55
should become 62955. however, it becomes 62954.9999. all other
numbers have gone through fine. but, i started testing numbers and did
find that 628.55 gives the same results. but these are the ONLY
numbers that seem to round. i can't for the life of me figure out how
rounding and multiplication even fit together. so, someone please let
me know if they've seen this, if they can duplicate this, or if i'm
just crazy. thanks.
heath
| |
| Greg Forsythe 2005-11-30, 8:48 pm |
| This is a limitation or performing floating point arithmetic using a
computer.
http://docs.python.org/tut/node16.html
You need to explicitly round the result of the multiplication functoid using
the round functoid.
Greg.
"heath" <guyemup@yahoo.com> wrote in message
news:1133388203.312643.317210@f14g2000cwb.googlegroups.com...
> first off, i've been, ahem, handed biztalk admin role with zero
> experience and am really just fixing bugs as i can so i apologize if
> i'm missing something entirely. secondly, i'm running bt2002 with
> hipaa accelerator. i need to get some dollar amounts into a 9(5)v99
> format. so, i'm just running it through the multiplication functoid,
> multiplying by 100. this works fine. however, the value of 629.55
> should become 62955. however, it becomes 62954.9999. all other
> numbers have gone through fine. but, i started testing numbers and did
> find that 628.55 gives the same results. but these are the ONLY
> numbers that seem to round. i can't for the life of me figure out how
> rounding and multiplication even fit together. so, someone please let
> me know if they've seen this, if they can duplicate this, or if i'm
> just crazy. thanks.
>
> heath
>
| |
|
| thanks, greg. great point and great article -- makes sense and makes
me question any floating math programming i've ever done. however, if
i copy the actual vbscript code from the multiplication functoid into a
different vbscript application, i get the correct answer. can you tell
me why other apps (i used both a vbs file and an activex task in a dts
package for my testing) can round this correctly but biztalk cannot?
|
|
|
|
|