|
Home > Archive > BizTalk Server General > June 2004 > FIFO Message processing
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
FIFO Message processing
|
|
| Denis 2004-06-20, 11:07 pm |
| We have a file receive function that reads message from a folder. It seems that Biztalk doesn't guarantee FIFO processing. How can we guarantee that 100%?
We are using Biztalk Server 2002 Partner Edition.
The server that we have currently has 4 cpu.
| |
| Jan Eliasen 2004-06-20, 11:07 pm |
| Hi
As you state yourself, BizTalk doe snot guarantee sequential
processing.
There are a couple of workarounds that almost work:
First of all, you can set the "Maximum number of worker threads per
processor allowed" to one. This will not guarantee sequential
processing, but it will come almost close :-) If you switch off three
of your CPU's you will come even closer to a solution.
Another solution that almost works is to have a preprocessor that
wirtes a sequence-number in the incoming document somewhere and then
deliver the document to an AIC, that will hold the document until all
sequence-numbers smaller than the current have passed through. This
will work, but only guarantee delivering of dicuments in the order the
preprocessor received them. Setting the "Maximum number of receive
function threads allowed" to one will let you only take one document
at a time. But the order the receive functions look for documents in
might not be the order you want the sorting to be based upon... so
this might not be what you want.
If the documents themself contain a sequence-number, then you are in
luck :-) Build an AIC that will only deliver documents when all
previous sequence-numbers have been delivered.
Hope this helps.
Jan Eliasen
On Mon, 14 Jun 2004 09:03:02 -0700, "Denis"
<Denis@discussions.microsoft.com> wrote:
>We have a file receive function that reads message from a folder. It seems that Biztalk doesn't guarantee FIFO processing. How can we guarantee that 100%?
>
>We are using Biztalk Server 2002 Partner Edition.
>The server that we have currently has 4 cpu.
--
Jan Eliasen, representing himself and not the company he works for.
| |
| Denis 2004-06-20, 11:07 pm |
| Thanks for your answer.
Here is the scenario that I'm using right now :
I have a File receive Function that reads HL7 messages from a folder, parse it as XML and sends it to a queue. I have a second receive function that map the HL7/XML message to a custom XML (My schema). And lastly a third MSMQ Receive function that calls a
Xlang orchestration to put it in SQL Server.
As I stated before, I want this whole process to work as a FIFO process.
I find it pretty odd that Biztalk doesn't provide FIFO functionality.
"Jan Eliasen" wrote:
> Hi
>
> As you state yourself, BizTalk doe snot guarantee sequential
> processing.
>
> There are a couple of workarounds that almost work:
>
> First of all, you can set the "Maximum number of worker threads per
> processor allowed" to one. This will not guarantee sequential
> processing, but it will come almost close :-) If you switch off three
> of your CPU's you will come even closer to a solution.
>
> Another solution that almost works is to have a preprocessor that
> wirtes a sequence-number in the incoming document somewhere and then
> deliver the document to an AIC, that will hold the document until all
> sequence-numbers smaller than the current have passed through. This
> will work, but only guarantee delivering of dicuments in the order the
> preprocessor received them. Setting the "Maximum number of receive
> function threads allowed" to one will let you only take one document
> at a time. But the order the receive functions look for documents in
> might not be the order you want the sorting to be based upon... so
> this might not be what you want.
>
> If the documents themself contain a sequence-number, then you are in
> luck :-) Build an AIC that will only deliver documents when all
> previous sequence-numbers have been delivered.
>
> Hope this helps.
>
> Jan Eliasen
>
> On Mon, 14 Jun 2004 09:03:02 -0700, "Denis"
> <Denis@discussions.microsoft.com> wrote:
>
>
> --
> Jan Eliasen, representing himself and not the company he works for.
>
| |
| Jan Eliasen 2004-06-20, 11:07 pm |
| Hi
To me, it sounds like you haven't quite understood BizTalk.
If you basically want to take one document, transform it into another
format and then send it to a message queue, then you can do that with
one receive-function, one channel and one port. You do not need the
three steps you have.
You can map directly from your inbound document specification to the
last outbound document specification. The transformation to and from
XML is done automatically by BizTalk for you.
So create a document specification for the inbound and use that in
your map. Use the specification for your final output as the outbound
schema in the map and use the map in the channel. Don't use the
middle-steps.
Besides that, there is a BizTalk Adapter for SQL Server. Maybe that
would be better than letting an orchestration deliver the document to
a SQL Server - depending on the structure of your output.
Lastly, if you follow my guidelines, you will have only one
receive-function - a file receive function. You can then use the
method described below:
Write a preprocessor that adds a sequence-number to the document (if
there isn't allready one?), and deliver the document to a AIC instead
of an orchestration. Program the AIC to not process a document until
all smaller sequence numbers have been processed and then write your
own code to put the document into SQL Server.
Yes, it is a hassle, but BizTalk does not provide what you ar elooking
for. Primarily because the underlying layer does not support it. I
mean: What if you are receiving two documents in the folder at the
same time? Which one should BizTalk process first? There is no way you
can get the filesystem to be FIFO.
Jan Eliasen
On Tue, 15 Jun 2004 07:14:01 -0700, "Denis"
<Denis@discussions.microsoft.com> wrote:
>Thanks for your answer.
>
>Here is the scenario that I'm using right now :
>
>I have a File receive Function that reads HL7 messages from a folder, parse it as XML and sends it to a queue. I have a second receive function that map the HL7/XML message to a custom XML (My schema). And lastly a third MSMQ Receive function that calls
a Xlang orchestration to put it in SQL Server.[vbcol=seagreen]
>
>As I stated before, I want this whole process to work as a FIFO process.
>
>I find it pretty odd that Biztalk doesn't provide FIFO functionality.
>
>"Jan Eliasen" wrote:
>
--
Jan Eliasen, representing himself and not the company he works for.
| |
| Denis 2004-06-20, 11:07 pm |
| Hi,
I use Itemfield ContentMaster to parse the the HL7 message to XML. I don't know if can map it to my XML schema in the same Channel/Port. I did not want to mix all the processing in the same channel/port.
The reason I would use an Orchestration is to provide decision capability before calling my COM component to update the server. I will check for the AIC. I think I can have a sequence number directly from the file.
Thanks for your help.
"Jan Eliasen" wrote:
> Hi
>
> To me, it sounds like you haven't quite understood BizTalk.
>
> If you basically want to take one document, transform it into another
> format and then send it to a message queue, then you can do that with
> one receive-function, one channel and one port. You do not need the
> three steps you have.
>
> You can map directly from your inbound document specification to the
> last outbound document specification. The transformation to and from
> XML is done automatically by BizTalk for you.
>
> So create a document specification for the inbound and use that in
> your map. Use the specification for your final output as the outbound
> schema in the map and use the map in the channel. Don't use the
> middle-steps.
>
> Besides that, there is a BizTalk Adapter for SQL Server. Maybe that
> would be better than letting an orchestration deliver the document to
> a SQL Server - depending on the structure of your output.
>
> Lastly, if you follow my guidelines, you will have only one
> receive-function - a file receive function. You can then use the
> method described below:
>
> Write a preprocessor that adds a sequence-number to the document (if
> there isn't allready one?), and deliver the document to a AIC instead
> of an orchestration. Program the AIC to not process a document until
> all smaller sequence numbers have been processed and then write your
> own code to put the document into SQL Server.
>
> Yes, it is a hassle, but BizTalk does not provide what you ar elooking
> for. Primarily because the underlying layer does not support it. I
> mean: What if you are receiving two documents in the folder at the
> same time? Which one should BizTalk process first? There is no way you
> can get the filesystem to be FIFO.
>
> Jan Eliasen
>
>
> On Tue, 15 Jun 2004 07:14:01 -0700, "Denis"
> <Denis@discussions.microsoft.com> wrote:
>
s a Xlang orchestration to put it in SQL Server.[vbcol=seagreen]
>
> --
> Jan Eliasen, representing himself and not the company he works for.
>
|
|
|
|
|