BizTalk Server Orchestration - Batching large files in an Orchestration: Is it a good approach?

Author

Batching large files in an Orchestration: Is it a good approach?

Pandit

2006-01-27, 9:27 pm

Hi,

I have a design problem. For one of our clients, we designed a BizTalk 2004
solution. The input to the BizTalk solution is a large file with some 2,000
to 10,000 records in it. What we do once it reaches BizTalk is, we break up
this huge file into individual records and process them for obvious
performance. The clients requirement is, we need to batch all these split
messages back [once all the necessary processing is done], into one single
file and send the file out. We had used a Sequential convoy pattern to do
the batching in a Orchestration. But, this has become a huge performance
issue, because once it hits the Sequential Convoy it becomes pretty much
single threaded. Are there any better design principles to improve the
performance?
--
Pandit