|
Home > Archive > BizTalk Server General > March 2006 > Single - multi message into multi single messages
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Single - multi message into multi single messages
|
|
| SteveM 2006-03-16, 5:51 pm |
| Now that should confuse you.
I have an input schema where I have multiple parent records, each with child
records.
I want to be able to pass the messages to my orchestration as one parent per
message, rather than all the parents at once.
So as an example, the way I have it now, my stored procedure returns
parent1(+children), parent2(+children) and one orchestration gets fired to
handle the lot.
I want to have two orchestrations fired - one for each parent. I am sure
there is a setting somewhere that encourages the original message to be
broken into mini messages.
Obviously, what I want is the output to be one parent per file.
Any suggestions on how to do this?
Thanks
Steve
| |
| Jeff Lynch 2006-03-16, 5:51 pm |
| Steve,
I'm not sure if this is exactly your scenario but I wrote a short article
describing how to logically split an XML document. I hope this helps.
http://codebetter.com/blogs/jeff.ly...les/128664.aspx
--
Jeff Lynch
"Ramblings From A [Microsoft] Connected Universe"
http://codebetter.com/blogs/jeff.lynch
"SteveM" <SteveM@discussions.microsoft.com> wrote in message
news:97C44148-6E85-4240-B292-3860E81CC7BA@microsoft.com...
> Now that should confuse you.
>
> I have an input schema where I have multiple parent records, each with
> child
> records.
>
> I want to be able to pass the messages to my orchestration as one parent
> per
> message, rather than all the parents at once.
>
> So as an example, the way I have it now, my stored procedure returns
> parent1(+children), parent2(+children) and one orchestration gets fired to
> handle the lot.
>
> I want to have two orchestrations fired - one for each parent. I am sure
> there is a setting somewhere that encourages the original message to be
> broken into mini messages.
>
> Obviously, what I want is the output to be one parent per file.
>
> Any suggestions on how to do this?
>
> Thanks
> Steve
| |
| SteveM 2006-03-16, 5:51 pm |
| Thanks Jeff, this looks pretty close to what I want and I will play with it
over the weekend to bend it to me needs.
The basic difference appears to be that I actually have mutiple headers,
rather than needing to repeat a single header. (If I read your blog
correctly).
Cheers
Steve
"Jeff Lynch" wrote:
> Steve,
>
> I'm not sure if this is exactly your scenario but I wrote a short article
> describing how to logically split an XML document. I hope this helps.
>
> http://codebetter.com/blogs/jeff.ly...les/128664.aspx
>
> --
> Jeff Lynch
> "Ramblings From A [Microsoft] Connected Universe"
> http://codebetter.com/blogs/jeff.lynch
>
>
| |
| Jeff Lynch 2006-03-17, 2:53 am |
| Steve,
Yes I did have a single header. If you actually have different instance
documents within a single interchange and just need to split it up, there
are much simpler methods and Stephen Thomas [MVP - BizTalk] has done a lot
of work on this.
http://www.biztalkgurus.com/Labs/Me...h-Envelope.html
--
Jeff Lynch
"Ramblings From A [Microsoft] Connected Universe"
http://codebetter.com/blogs/jeff.lynch
"SteveM" <SteveM@discussions.microsoft.com> wrote in message
news:5A91B576-BB58-4BB3-B3EC-4FA26F9563F7@microsoft.com...
> Thanks Jeff, this looks pretty close to what I want and I will play with
> it
> over the weekend to bend it to me needs.
>
> The basic difference appears to be that I actually have mutiple headers,
> rather than needing to repeat a single header. (If I read your blog
> correctly).
>
> Cheers
> Steve
>
> "Jeff Lynch" wrote:
>
>
| |
| maltistefan@yahoo.co.nz 2006-03-17, 7:49 am |
| Hi Guys,
I think you need some best practice in this case.
First Jeff :
I saw your site, it is interesting how you deal with the orchestration
and all the bla bla, however what you could do, is instead of doing
your mapping inside an orchestration you could have done it straight in
the port. In the inbound maps, it is much quicker! like this the
orchestration receives directely the correct mapping.
Second Steve :
There is a fantastic tric, I do not know if it applies for your case
but here is the case:
- In the schema you must declare in the part that repeats its self that
the max occurs is "1"
- The pipeline will interpret the message it receives and split it into
as many parts as it is necessary.
Example:
if you have a message that is something like
<Root>
<PurchaseOrder>
<bla/>
<bli/>
<blou/>
</PurchaseOrder>
<PurchaseOrder>
<bla/>
<bli/>
<blou/>
</PurchaseOrder>
</Root>
Then in your Schema you declare the PurchaseOrder as having max occurs
equal to 1
The pipeline will do the splitting, and you'll have two orchestration
started.
Hope this will help you guys
Stefan
| |
| maltistefan@yahoo.co.nz 2006-03-17, 7:49 am |
| Hi Guys,
I think you need some best practice in this case.
First Jeff :
I saw your site, it is interesting how you deal with the orchestration
and all the bla bla, however what you could do, is instead of doing
your mapping inside an orchestration you could have done it straight in
the port. In the inbound maps, it is much quicker! like this the
orchestration receives directely the correct mapping.
Second Steve :
There is a fantastic tric, I do not know if it applies for your case
but here is the case:
- In the schema you must declare in the part that repeats its self that
the max occurs is "1"
- The pipeline will interpret the message it receives and split it into
as many parts as it is necessary.
Example:
if you have a message that is something like
<Root>
<PurchaseOrder>
<bla/>
<bli/>
<blou/>
</PurchaseOrder>
<PurchaseOrder>
<bla/>
<bli/>
<blou/>
</PurchaseOrder>
</Root>
Then in your Schema you declare the PurchaseOrder as having max occurs
equal to 1
The pipeline will do the splitting, and you'll have two orchestration
started.
Hope this will help you guys
Stefan
| |
| Greg Forsythe 2006-03-17, 5:52 pm |
| There are a number of ways of splitting an incoming batch into multiple
messages.
The choice of which one you use will vary depending on your requirements.
Splitting in the pipeline using the disassembler is the most efficient but
will not always be the best choice.
A good example:
On Biztalk 2004 has no recoverable batch processing (new feature in 2006).
If one of the child messages fails processing in the pipeline (e.g.
Validation error) Then then entire batch of messages will fail. By moving
this into an orchestration you can have more control, catch exceptions and
only reject the offending message.
There are a number of good reasons for using an orchestration - I would
suggest reading Stephan Thomas' paper that Jeff mentioned.
Both the FFDisassembler and XmlDisassembler will split incoming batches into
multiple messages, however the mechanism is different for each.
If you are receiving a Flat File then you can use the maxOccurs=1 technique
as mentioned.
If you are receiving an Xml file then you should use an Envelope and message
schema..
Greg
<maltistefan@yahoo.co.nz> wrote in message
news:1142591136.582338.134560@e56g2000cwe.googlegroups.com...
> Hi Guys,
>
> I think you need some best practice in this case.
>
> First Jeff :
>
> I saw your site, it is interesting how you deal with the orchestration
> and all the bla bla, however what you could do, is instead of doing
> your mapping inside an orchestration you could have done it straight in
> the port. In the inbound maps, it is much quicker! like this the
> orchestration receives directely the correct mapping.
>
>
> Second Steve :
>
> There is a fantastic tric, I do not know if it applies for your case
> but here is the case:
> - In the schema you must declare in the part that repeats its self that
> the max occurs is "1"
> - The pipeline will interpret the message it receives and split it into
> as many parts as it is necessary.
> Example:
> if you have a message that is something like
> <Root>
> <PurchaseOrder>
> <bla/>
> <bli/>
> <blou/>
> </PurchaseOrder>
> <PurchaseOrder>
> <bla/>
> <bli/>
> <blou/>
> </PurchaseOrder>
> </Root>
> Then in your Schema you declare the PurchaseOrder as having max occurs
> equal to 1
> The pipeline will do the splitting, and you'll have two orchestration
> started.
>
> Hope this will help you guys
> Stefan
>
| |
| SteveM 2006-03-19, 11:59 am |
| Thanks Greg,
> If you are receiving an Xml file then you should use an Envelope and message
> schema..
This is what I will be doing. Do you have any exmaples you can point me at?
BTW - using BTS 2006
Basically, I have a XML schema generated from SQL Server
<Calls>
<Job>
... elements/attributes ...
</Job>
</Calls>
Used this as my Envelope schema (And set the Envelope property of the Schema
node to Yes)
Took
<Job>
... elements/attributes ...
</Job>
and used it as my document
Created a pipeline which specifed the element and document schemas
Created a new map to use the document instead of the envelope.
However, despite not making any changes to the receive pipeline (only the
underlying schema), apparently the schema of the selected data no longer
matches any of the given schemas.
Will try different combinations in the pipeline, but would appreciate any
pointers to this approach.
Otherwise I may have to try the approach suggested by Jeff
Thanks
Steve
"Greg Forsythe" wrote:
> There are a number of ways of splitting an incoming batch into multiple
> messages.
> The choice of which one you use will vary depending on your requirements.
> Splitting in the pipeline using the disassembler is the most efficient but
> will not always be the best choice.
> A good example:
> On Biztalk 2004 has no recoverable batch processing (new feature in 2006).
> If one of the child messages fails processing in the pipeline (e.g.
> Validation error) Then then entire batch of messages will fail. By moving
> this into an orchestration you can have more control, catch exceptions and
> only reject the offending message.
> There are a number of good reasons for using an orchestration - I would
> suggest reading Stephan Thomas' paper that Jeff mentioned.
>
> Both the FFDisassembler and XmlDisassembler will split incoming batches into
> multiple messages, however the mechanism is different for each.
> If you are receiving a Flat File then you can use the maxOccurs=1 technique
> as mentioned.
> If you are receiving an Xml file then you should use an Envelope and message
> schema..
>
>
> Greg
>
| |
| Greg Forsythe 2006-03-21, 3:09 am |
| With an Envelope schema you also need to specify the node that contains the
documents.
You need to set the Body Xpath property on the root node.
In your case this should be set to the root node itself -> Calls.
This will produce a new message for each child of the <Calls> node.
Greg
"SteveM" <SteveM@discussions.microsoft.com> wrote in message
news:DB6BD260-674C-407C-918B-6549F2933AC8@microsoft.com...
> Thanks Greg,
>
>
> This is what I will be doing. Do you have any exmaples you can point me
> at?
>
> BTW - using BTS 2006
>
> Basically, I have a XML schema generated from SQL Server
>
> <Calls>
> <Job>
> ... elements/attributes ...
> </Job>
> </Calls>
>
> Used this as my Envelope schema (And set the Envelope property of the
> Schema
> node to Yes)
>
> Took
> <Job>
> ... elements/attributes ...
> </Job>
> and used it as my document
>
> Created a pipeline which specifed the element and document schemas
>
> Created a new map to use the document instead of the envelope.
>
> However, despite not making any changes to the receive pipeline (only the
> underlying schema), apparently the schema of the selected data no longer
> matches any of the given schemas.
>
> Will try different combinations in the pipeline, but would appreciate any
> pointers to this approach.
>
> Otherwise I may have to try the approach suggested by Jeff
>
> Thanks
> Steve
>
> "Greg Forsythe" wrote:
>
>
| |
| SteveM 2006-03-21, 3:09 am |
| Thanks Greg, still getting the error message though.
When I google for the message, the first two posts tell me that this is not
possible due to the schema not having a fully qualified assembly name to
search for (or some such).
Apparently this was a problem in BTS 2004 and appears to continue to be so
in BTS 2006
When tested just loading my data and dumping with XML Receive, no trouble.
As soon as I put in a custom pipeline for the XML document, it all falls over.
I believe Jeff came across this problem at some stage also.
"Greg Forsythe" wrote:
> With an Envelope schema you also need to specify the node that contains the
> documents.
> You need to set the Body Xpath property on the root node.
> In your case this should be set to the root node itself -> Calls.
> This will produce a new message for each child of the <Calls> node.
>
> Greg
>
| |
| Greg Forsythe 2006-03-21, 3:09 am |
| In a previous mail you said:
"Took
<Job>
... elements/attributes ...
</Job>
and used it as my document"
Did you create a separate schema with a root node of <Job> or move the
<Job> element to be a root node within the SQL generated schema?
The XmlDisassembler must be able to find the schema that matches the split
message so the MessageType can be set on the message when it is published to
the messagebox
So you end up with two effective schemas.
1. Envelope:
<Calls>
<xs:any> or <xs:element ref="Job>
</Calls>
2. Message
<Job>
... elements/attributes ...
</Job>
Greg
"SteveM" <SteveM@discussions.microsoft.com> wrote in message
news:C78F1DB3-97B8-4F4E-9F67-327C36AF28BC@microsoft.com...[vbcol=seagreen]
> Thanks Greg, still getting the error message though.
>
> When I google for the message, the first two posts tell me that this is
> not
> possible due to the schema not having a fully qualified assembly name to
> search for (or some such).
>
> Apparently this was a problem in BTS 2004 and appears to continue to be so
> in BTS 2006
>
> When tested just loading my data and dumping with XML Receive, no trouble.
> As soon as I put in a custom pipeline for the XML document, it all falls
> over.
>
> I believe Jeff came across this problem at some stage also.
>
>
> "Greg Forsythe" wrote:
>
| |
| SteveM 2006-03-21, 3:09 am |
| OK, so rather than two separate .xsd files, both definitions are in one, with
the first definition referencing the second, rather than containing a replica
of the second.
Hope I've got what you're saying here - will give this a try as it may
explain the error messages I have been getting this afternoon while fiddling
around with various combinations.
Thanks again.
"Greg Forsythe" wrote:
> In a previous mail you said:
> "Took
> <Job>
> ... elements/attributes ...
> </Job>
> and used it as my document"
>
> Did you create a separate schema with a root node of <Job> or move the
> <Job> element to be a root node within the SQL generated schema?
> The XmlDisassembler must be able to find the schema that matches the split
> message so the MessageType can be set on the message when it is published to
> the messagebox
> So you end up with two effective schemas.
> 1. Envelope:
> <Calls>
> <xs:any> or <xs:element ref="Job>
> </Calls>
> 2. Message
> <Job>
> ... elements/attributes ...
> </Job>
>
> Greg
>
| |
| SteveM 2006-03-21, 3:10 am |
| Greg, I am afraid that this is fast turning to mud for me. I have tried to
use the <xs:any> notation but when I try to set up my receive location the
xml schema is deemed not to have been generated by the SQL adapter.
When I try to use the <xs:element ref="Job> notation, I get a message saying
the element 'Job' is undefined.
Obviously, this comes down to my xml editing skills.
If at all possible, could you provide a more detailed example of the xml
schema? I have probably moved the Job element incorrectly and all this
complexType syntax etc is starting to look like mush.
"Greg Forsythe" wrote:
> In a previous mail you said:
> "Took
> <Job>
> ... elements/attributes ...
> </Job>
> and used it as my document"
>
> Did you create a separate schema with a root node of <Job> or move the
> <Job> element to be a root node within the SQL generated schema?
> The XmlDisassembler must be able to find the schema that matches the split
> message so the MessageType can be set on the message when it is published to
> the messagebox
> So you end up with two effective schemas.
> 1. Envelope:
> <Calls>
> <xs:any> or <xs:element ref="Job>
> </Calls>
> 2. Message
> <Job>
> ... elements/attributes ...
> </Job>
>
> Greg
>
> "SteveM" <SteveM@discussions.microsoft.com> wrote in message
> news:C78F1DB3-97B8-4F4E-9F67-327C36AF28BC@microsoft.com...
>
>
>
| |
| Greg Forsythe 2006-03-21, 3:10 am |
| Steve,
Can you email me the schema as generated by the SQL Adapter.
Greg
"SteveM" <SteveM@discussions.microsoft.com> wrote in message
news:60019B5D-EF63-4E28-84BB-D37A9786069D@microsoft.com...[vbcol=seagreen]
> Greg, I am afraid that this is fast turning to mud for me. I have tried to
> use the <xs:any> notation but when I try to set up my receive location the
> xml schema is deemed not to have been generated by the SQL adapter.
>
> When I try to use the <xs:element ref="Job> notation, I get a message
> saying
> the element 'Job' is undefined.
>
> Obviously, this comes down to my xml editing skills.
>
> If at all possible, could you provide a more detailed example of the xml
> schema? I have probably moved the Job element incorrectly and all this
> complexType syntax etc is starting to look like mush.
>
>
>
> "Greg Forsythe" wrote:
>
|
|
|
|
|