|
Home > Archive > BizTalk Server General > April 2006 > Pipeline InstanceID...
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Pipeline InstanceID...
|
|
|
| Hi,
How to get Receive Pipeline InstanceID in a custom pipeline component?
Is there a method available in BizTalk API that I can make use of?
Thanks in advance.
Regards,
Kiran Pabba
| |
| Lucas R. Vogel 2006-04-27, 7:26 am |
| What version of BizTalk are you using, and what are you trying to do with the
Instance ID?
--
===========
Lucas R. Vogel
"Pabba" wrote:
> Hi,
>
> How to get Receive Pipeline InstanceID in a custom pipeline component?
> Is there a method available in BizTalk API that I can make use of?
>
> Thanks in advance.
>
> Regards,
> Kiran Pabba
>
>
| |
|
| Hi Vogel,
My requirement is to track all incoming flat-files and log the
information at all stages(Receive Pipeline, Orchestration, Send
Pipeline) into BAM so that operations can easily identify the failure
reason and information to resolve it. The information logged in BAM
includes ActivityID, File Name, FileCreationTime, RecordCount,
ReceivedFrom, MessageSize, Orchestration InstanceID,
PipelineInstanceID(?) etc., that helps operations to re-run the file
based on resolution.
I'm using BAM API to track this information at Rcv Pipeline,
Orachestration & Send Pipeline levels and trying to interlink these
into a single BAM definition using the BAM Continuity. Finally I
provide a Report to operations by retrieving information from this BAM
view and BizTalkDTADb dtav_ServiceFacts view(since BizTalk platform
level message details[Error Info, StartTime, EndTime, Duration etc.]
can be found in this view). I'm able to link these two views at the
Orchestration level by logging Orchestration InstanceID into BAM
entries and currently trying to identify how to link these at the
Pipeline record/entry level. I'm googling to find how to get the
Pipeline InstanceID so that I can BAM that in my pipeline and finally
build a join query that fetches me EndToEnd record to report to
operations making use of BAM ActivityID, Pipeline InstanceID &
Orchestration InstanceID.
Appreciate your inputs in this regard.
Thanks & Regards,
Kiran Pabba
| |
|
| I missed it, I'm using BTS 2006.
Regards,
Kiran Pabba
| |
| Lucas R. Vogel 2006-04-27, 7:26 am |
| It seems like what you are looking to do could be done through either Health
and Activity Tracking configuration, a Tracking Profile, or some combination
of the two, without having to use the BAM API directly...
You can access a PipelineId (guid) property within the IPipelineContext, and
a MessageId (guid) property within the IBaseMessage objects passed in as
parameters in a custom disassembler component, if you need them for API
access. To the best of my knowledge, however, you'll have to create a custom
pipeline component to write these to the API...see the documentation
(http://tinyurl.com/hbvy6 and http://tinyurl.com/jvlgb, respectively) for
more details.
Hope this helps!
Lucas
--
===========
Lucas R. Vogel
"Pabba" wrote:
> Hi Vogel,
>
> My requirement is to track all incoming flat-files and log the
> information at all stages(Receive Pipeline, Orchestration, Send
> Pipeline) into BAM so that operations can easily identify the failure
> reason and information to resolve it. The information logged in BAM
> includes ActivityID, File Name, FileCreationTime, RecordCount,
> ReceivedFrom, MessageSize, Orchestration InstanceID,
> PipelineInstanceID(?) etc., that helps operations to re-run the file
> based on resolution.
>
> I'm using BAM API to track this information at Rcv Pipeline,
> Orachestration & Send Pipeline levels and trying to interlink these
> into a single BAM definition using the BAM Continuity. Finally I
> provide a Report to operations by retrieving information from this BAM
> view and BizTalkDTADb dtav_ServiceFacts view(since BizTalk platform
> level message details[Error Info, StartTime, EndTime, Duration etc.]
> can be found in this view). I'm able to link these two views at the
> Orchestration level by logging Orchestration InstanceID into BAM
> entries and currently trying to identify how to link these at the
> Pipeline record/entry level. I'm googling to find how to get the
> Pipeline InstanceID so that I can BAM that in my pipeline and finally
> build a join query that fetches me EndToEnd record to report to
> operations making use of BAM ActivityID, Pipeline InstanceID &
> Orchestration InstanceID.
>
> Appreciate your inputs in this regard.
>
> Thanks & Regards,
> Kiran Pabba
>
>
| |
| Tomas Restrepo \(MVP\) 2006-04-27, 7:26 am |
| Pabba,
> My requirement is to track all incoming flat-files and log the
> information at all stages(Receive Pipeline, Orchestration, Send
> Pipeline) into BAM so that operations can easily identify the failure
> reason and information to resolve it. The information logged in BAM
> includes ActivityID, File Name, FileCreationTime, RecordCount,
> ReceivedFrom, MessageSize, Orchestration InstanceID,
> PipelineInstanceID(?) etc., that helps operations to re-run the file
> based on resolution.
>
> I'm using BAM API to track this information at Rcv Pipeline,
> Orachestration & Send Pipeline levels and trying to interlink these
> into a single BAM definition using the BAM Continuity. Finally I
> provide a Report to operations by retrieving information from this BAM
> view and BizTalkDTADb dtav_ServiceFacts view(since BizTalk platform
> level message details[Error Info, StartTime, EndTime, Duration etc.]
> can be found in this view). I'm able to link these two views at the
> Orchestration level by logging Orchestration InstanceID into BAM
> entries and currently trying to identify how to link these at the
> Pipeline record/entry level. I'm googling to find how to get the
> Pipeline InstanceID so that I can BAM that in my pipeline and finally
> build a join query that fetches me EndToEnd record to report to
> operations making use of BAM ActivityID, Pipeline InstanceID &
> Orchestration InstanceID.
The pipeline instance ID is also just a service instance id (pipeline
execution is a service instance, as well). That said, I looked around and
didn't find a way to directly access the id for the pipeline instance
through the API. You can get the Pipeline ID (through
IPipelineContext.PipelineID) but not the instance id, which is what you're
asking for. That said, given the PipelineID and the MessageID (as Lucas
suggested) there might be something you can do to get the instance ID out of
HAT.
--
Tomas Restrepo
tomasr@mvps.org
http://www.winterdom.com/
| |
|
| Thanks Lucas & Tomas,
The reason for not using HAT to achieve this functionality is that, my
client wants to have a Reporting tool to track the messages similar to
HAT (but not HAT) with BAM capabilities so that in future he can make
use of BAM OLAP features (which may get deviated if I choose HAT, I may
be wrong in interpreting things?). As mentioned I'm not familiar with
how to configure HAT or TPE or combination to suite to this
requirement. It would be great if you can throw some light on this by
explaining how to use them.
And coming to getting Pipeline InstanceID from HAT using PipelineID &
MessageID. How to get that, any pointers? And when I see BizTalkDTADb
dtav_ServiceFacts
view there is no way to get a unique record from this based on
PipelineID and there is no MessageID in this view so the only option is
some how I should get Pipeline InstanceID to further proceed with my
development.
Appreciate your responses in resolving this issue.
Thanks again,
Kiran Pabba
| |
| Lucas R. Vogel 2006-04-27, 7:26 am |
| Kiran,
In an earlier post you stated your requirements were "to track all incoming
flat-files and log the information at all stages(Receive Pipeline,
Orchestration, Send
Pipeline) into BAM so that operations can easily identify the failure
reason and information to resolve it". I hope the following
sugggestions/guidelines will convince you that you don't need to go down the
path of writing your own custom BAM code that attempts to do what is already
built into the BizTalk monitoring capabilities...
To log information at all stages, I suggest reviewing the documentation
regarding BizTalk Monitoring
(http://msdn.microsoft.com/library/e...dabc41eb144.asp
). In order to track and log information at all stages, I suggest turning on
Tracking, which as you will read, is performed at the Admin console and can
be fine-tuned to track inbound/outbound message data, message properties,
bodies and parts, and a whole list of options for orchestrations, ports,
pipelines, etc. -- all without writing any custom code. The role of HAT has
changed, if only slightly, from BizTalk 2004 - its emphasis now is on Health
and Activity Tracking; it's a tool that lets you view data you've captured in
the BizTalkDtaDb, amongst other things (a review of its documentation
wouldn't hurt either).
In order to handle failed messages, I would suggest building that kind of
handling into your orchestration so that messages that fail are routed to an
error message "capture" location (such as MSMQ), all of which can also be
monitored using the above technique. I would also review the ErrorHandling
sample found in the SDK and build on the example they provided to create a
more robust error-handling solution for your project, with tracking
configured to monitor and provide data on these processes as well - all
without having to write any custom BAM code.
Lastly, I would also implement a MOM monitoring solution to watch over your
BizTalk systems, and provide alerts and managment metadata so your operations
team can help pinpoint the time and place errors occur.
Please let me know if this helps...
Lucas
--
===========
Lucas R. Vogel
"Pabba" wrote:
> Thanks Lucas & Tomas,
>
> The reason for not using HAT to achieve this functionality is that, my
> client wants to have a Reporting tool to track the messages similar to
> HAT (but not HAT) with BAM capabilities so that in future he can make
> use of BAM OLAP features (which may get deviated if I choose HAT, I may
> be wrong in interpreting things?). As mentioned I'm not familiar with
> how to configure HAT or TPE or combination to suite to this
> requirement. It would be great if you can throw some light on this by
> explaining how to use them.
>
> And coming to getting Pipeline InstanceID from HAT using PipelineID &
> MessageID. How to get that, any pointers? And when I see BizTalkDTADb
> dtav_ServiceFacts
> view there is no way to get a unique record from this based on
> PipelineID and there is no MessageID in this view so the only option is
> some how I should get Pipeline InstanceID to further proceed with my
> development.
>
> Appreciate your responses in resolving this issue.
>
> Thanks again,
> Kiran Pabba
>
>
| |
|
| Thanks Vogel for your valuable inputs.
I would like to highlight more on my requirements,
A. I need to log following information to be viewed in the Reporting
Website:
Custom data demanded by Application:
1. FileName
2. PartyName (File Received from)
3. File/Message Size
4. RecordCount
5. ApplicationName
6. FileCreationTime
7. Status (Application Level)
8. ErrorXml (Application Level)
9. RcvPipelineInstanceID(?)
10. OrchestrationInstnaceID
11. SendPipelineInstanceID(?)
Standard data provided by BTS Tracking tools:
12. StartTime
13. EndTime
14. Duration
15. ErrorInfo
16. ServiceInstance/State
17. AssemblyName
18. AssemblyVersion
19. AssemblyDeploymentTime
Since I need a combination of Custom & Standard data I am using BAM
that helps me define my attributes. And finally I create a [SELECT +
JOIN + UNION] on the BAM Definition view and BizTalkDTADb
dtav_ServiceFacts view using the Orchestration InstnaceID & Pipeline
InstanceID that results in record(s) displayed per Message(Flat-file
received from a particular party) on a custom Website/Reporting page
that provides different views to Administrator, Parties, Developes
based on roles.
B. The discussed website (in previous section) is similar to BAM Portal
but not exactly since in BAM Portal there is no provision to view the
"Standard Tracking data" as can be viewed in the Reporting page.
Another reason to use BAM is, if in case Adminstrator or Parties want
to aggregate the data they can use BAM OLAP features.
C. The reason for approaching Reporting tool is to eliminate multiple
places to locate/track and find where error has occurred and how to
resolve it.
The business requirement demands operations Reporting to be a webpage
accessable to:
1. Different roles at client site where the inbound flat-files are
processed as well as to
2. The parties(large in numbers & remotely located) who send these
flat-files so that both are questionable/answerable to the missing
flat-files and unprocessed flat-files.
I am still awaiting a response on how to get Pipeline InstanceID in my
custom code.
Appreciate your responses in enhancing the design. I am open to an
alternate design that suites to my requirement if in case I'm unable to
resolve the Pipeline InstnaceID issue.
Thanks & Regards,
Kiran Pabba
| |
| Lucas R. Vogel 2006-04-27, 7:26 am |
| Kiran,
There is no Pipeline Instance Id available for reporting purposes. The
architecture of pipelines is such that messages are meant to be streamed
through continuously; therefore, no instance (per se) is generated by the
system. Orchestrations, on the other hand, can have a unique instance per
received message, and will have an InstanceId associated with them.
Lucas
--
===========
Lucas R. Vogel
"Pabba" wrote:
> Thanks Vogel for your valuable inputs.
>
> I would like to highlight more on my requirements,
>
> A. I need to log following information to be viewed in the Reporting
> Website:
>
> Custom data demanded by Application:
>
> 1. FileName
> 2. PartyName (File Received from)
> 3. File/Message Size
> 4. RecordCount
> 5. ApplicationName
> 6. FileCreationTime
> 7. Status (Application Level)
> 8. ErrorXml (Application Level)
> 9. RcvPipelineInstanceID(?)
> 10. OrchestrationInstnaceID
> 11. SendPipelineInstanceID(?)
>
> Standard data provided by BTS Tracking tools:
>
> 12. StartTime
> 13. EndTime
> 14. Duration
> 15. ErrorInfo
> 16. ServiceInstance/State
> 17. AssemblyName
> 18. AssemblyVersion
> 19. AssemblyDeploymentTime
>
> Since I need a combination of Custom & Standard data I am using BAM
> that helps me define my attributes. And finally I create a [SELECT +
> JOIN + UNION] on the BAM Definition view and BizTalkDTADb
> dtav_ServiceFacts view using the Orchestration InstnaceID & Pipeline
> InstanceID that results in record(s) displayed per Message(Flat-file
> received from a particular party) on a custom Website/Reporting page
> that provides different views to Administrator, Parties, Developes
> based on roles.
>
> B. The discussed website (in previous section) is similar to BAM Portal
> but not exactly since in BAM Portal there is no provision to view the
> "Standard Tracking data" as can be viewed in the Reporting page.
>
> Another reason to use BAM is, if in case Adminstrator or Parties want
> to aggregate the data they can use BAM OLAP features.
>
> C. The reason for approaching Reporting tool is to eliminate multiple
> places to locate/track and find where error has occurred and how to
> resolve it.
>
> The business requirement demands operations Reporting to be a webpage
> accessable to:
> 1. Different roles at client site where the inbound flat-files are
> processed as well as to
> 2. The parties(large in numbers & remotely located) who send these
> flat-files so that both are questionable/answerable to the missing
> flat-files and unprocessed flat-files.
>
> I am still awaiting a response on how to get Pipeline InstanceID in my
> custom code.
>
> Appreciate your responses in enhancing the design. I am open to an
> alternate design that suites to my requirement if in case I'm unable to
> resolve the Pipeline InstnaceID issue.
>
> Thanks & Regards,
> Kiran Pabba
>
>
| |
|
| Thank you Vogel,
I was curious to find what are ServiceInstance/InstanceID &
ServiceInstance/ActivityID entries for Send and Receive Pipelines in
dtav_ServiceFacts view of BizTalkDTADb database? According to your
posting: "no instance is generated by the system" for pipelines...could
you help me understand what are these entries and is there a way to
access them in cusotm code since they look to be unique and can be used
for my purpose.
I started my design looking at these entries and I got pointer to
ServiceInstance/InstanceID value in orchestrations by using
"Microsoft.XLANGs.Core.Service.RootService.InstanceId" property and I
was expecting something similar to this in Pipline too.
Appreciate your response.
Regards,
Kiran Pabba
| |
| Lucas R. Vogel 2006-04-27, 7:26 am |
| After studying the dtav_ServiceFacts view in the BizTalkDtaDb database, I
found a correlation between the ActivityId in the pipeline entries and the
messages received in the dta_MessageInOutEvents table. As a result, it
appears that the Activity ID listed for the pipeline is the same Activity ID
listed for the Message received by the pipeline. What I believe is happening
is that BizTalk is creating its own internal Activity information for all of
its own internal monitoring purposes. The dtav_ServicesFacts view is in fact
a Health and Activity Tracking (HAT) view used to query and troubleshoot what
you configure BizTalk, as a server, to monitor.
If you want to create some kind of "unique instance" id to associate with
every time a message is received in a pipeline, you will have to create a
custom pipeline component for the activity you are trying to monitor.
Depending on what you want to record, you may want to create a custom
component for every service you use in the pipelines - for example, a custom
disassembly component to record when a message is received by the
disassembler. In your custom component, you will set it up so that the only
"custom" step that it does is generate BAM information for submitting to your
custom Activity, which can easily be done with the BAM API and Activity
objects. The real work, however, lies in the creation of the custom
components - and custom pipelines - where you want to generate information
indicating messages have passed through.
Another possible solution might be to try to create a custom query that will
take the HAT data you are looking at and inject it into your custom
activities. Perhaps you can create a solution where you export the DTA data
out of the view and into your reports, linking them through MessageId or some
other correlation.
In summary, the data you are looking at in the BizTalkDtaDb is BAM data
automatically generated by BizTalk as a result of having Tracking turned on.
If you want to want to use the Instance ID information generated by BizTalk,
you can try and build something based on what BizTalk will give you
automatically through the tracking configuration settings found in the
Administration Console (for hosts) and in HAT (for message and message
context, pipeline, orchestration, etc.), where all of the data is ultimately
collected, inject some custom code into your pipelines, or perform some kind
of data extract on the BizTalkDtaDb tables and views in order to get that
specific data.
If there is some other, better way possible than the ones I have listed, I
too would be very interested to know how to do it 
Regards,
Lucas
--
===========
Lucas R. Vogel
"Pabba" wrote:
> Thank you Vogel,
>
> I was curious to find what are ServiceInstance/InstanceID &
> ServiceInstance/ActivityID entries for Send and Receive Pipelines in
> dtav_ServiceFacts view of BizTalkDTADb database? According to your
> posting: "no instance is generated by the system" for pipelines...could
> you help me understand what are these entries and is there a way to
> access them in cusotm code since they look to be unique and can be used
> for my purpose.
>
> I started my design looking at these entries and I got pointer to
> ServiceInstance/InstanceID value in orchestrations by using
> "Microsoft.XLANGs.Core.Service.RootService.InstanceId" property and I
> was expecting something similar to this in Pipline too.
>
> Appreciate your response.
>
> Regards,
> Kiran Pabba
>
>
| |
|
| Thank you Vogel,
I would like to try the possibilities insisted in your posting and will
update you.
Regards,
Kiran Pabba
|
|
|
|
|