Very Strange behavior of XML Disassembler- Property Promotion not happening properly

While working on a pipeline component I encountered a very strange behavior of XML disassembler. I had few promoted properties which I was trying to access in a pipeline component in Validate stage after the execution of XML Disassembler. As we all know that the Pipelines are executed in sequential manner and the output of one pipeline component is used as input to the next one in the sequence(except in Disassemble stage). ( Read more- Understanding Pipeline Execution). So, ideally after execution of XML Disassembler, Promoted Properties should be accessible in later stages. But shockingly there value was NULL. Now let’s have a look into my sample BizTalk application for better understanding. I have a simple schema in which both the properties are promoted- Emp & Property Schema Image I created a simple pipeline component where I tried to read values of ID and Name from message context using below code and later promoted “ReceiveFileName” Fetching Promoted Properties in Pipeline Component Image Created a simple Custom Receive Pipeline with XML Disassembler and later used my pipeline component in Validate Stage. Receive Pipeline Image But strangely when I tried to fetch the promoted properties found out that it’s returning NULL as shown below. Promoted Properties as NULL in Pipeline component Image Cause:- After some research got to know that it’s a known behavior. Basically, the XmlDisassembler does not read through the incoming message immediately.  It waits until some later component or the MessageBox reads it.  This is done for various performance reasons like memory consumption, etc. So, since my custom pipeline component is next in line, when it gets the Message, data has not yet been read by the XmlDisassembler even though it passed that Stage in the Pipeline, so it hasn’t had a chance to Read and Promote any Context Properties.  This is by design, and what you are seeing is a known side-effect. Its accomplished by wrapping the data in a custom Stream class.  That is why it only gets read and processed when some other component reads it. Solution/Workaround:- The easiest option would be to just do you work with the Properties in an Orchestration. If you absolutely have to use a Pipeline Component, then you will have to somehow force a Read by the XmlDisassembler. For example in our pipeline component we have read the values using below code and forced a Read/Promote by the XmlDisassembler. //Reading message data using XML Document. System.Xml.XmlDocument xmlDoc = new System.Xml.XmlDocument(); xmlDoc.Load(originalStream); Using XMLDoc can have performance implication as the complete message will be loaded in memory, so you can use the below code as well. //Reading message data using Stream, this will not have any performance implication. System.IO.MemoryStream ms = new MemoryStream(); originalStream.CopyTo(ms); ms.Position = 0; inmsg.BodyPart.Data = ms; As shown in below image.    Promoted Properties containing Value After performing Forced Read by XML Disassembler Image   Check the discussion over the same issue- XML Diassembler component is not doing property promotion   Hope it helps.   Download the sample application from here. Word version of this blog is here.  

Contact Me:- 

@Gmail, @Facebook , @Twitter, @LinkedIn , @MSDNTechnet, @My Personal Blog 

Advertisement

Implementing Correlation (Property Promotion) without any Common Element

I recently encountered a scenario where I had to implement correlation without any common element between the outgoing and incoming messages. Generally we implement correlation by Initializing and following correlation on the same element.

Let’s come to our sample for better understanding.

We have two schemas “Candidate” with fields as CandidateID, Name City and Status and “Employee” schema with fields as EmployeeID, Name, ReportingCity and ReportingManager.

We need to initialize correlation for CandidateID and follow correlation on EmployeeID. But as these are two different elements so they can’t become one CorrelationType. However they hold the same value.

 Candidate, Employee and Property Schema Image

Solution: – I created a PropertySchema with one element as “UniqueID” and promoted CandidateID and EmployeeID against it.

How to achieve it:-

• Create on property schema with one element as “UniqueID”
• In Candidate schema, Right click on CandidateID-> show promotion ->Property Fields -> Click on top right corner a folder like icon (Add new Property Schema) and select the property schema created in first step.
• Now click on the CandidateID and “Add”. This will add the property for CandidateID field as shown below.

Property Promotion Image

• Perform the same steps to promote EmployeeID field as well present in Employee Schema.

Let’s have a look into the final schemas. If you notice in the Properties section you will find PropertyName = “UniqueID”. It means EmployeeID or CandidateID will be promoted against UniqueID field present in Property Schema.

Employee Schema

Employee Schema Image

Candidate Schema

Candidate Schema Image

Let’s create our Orchestration to meet our requirement. Only important thing to notice is how the correlation set is created and later used.

Creating Correlation Set Image

Later in Orchestration I have initialized Correlation while sending the Candidate information for approval and followed the same while receiving the Employee Info (Approved Candidates)

 Orchestration Image

Some interesting things:-

Created a test file with CandidateID = 1 and processed the same.

Promoted Properties of Candidate Schema Image

Subscription Filter on Correlated Receive shape:-

https://CorrelationWithoutCommonElement.PropertySchema.UniqueID == 1  And

http://schemas.microsoft.com/BizTalk/2003/system-properties.ReceivePortID == {6D17881D-4468-4EFD-A4EB-A05327E8DAE7}  And

http://schemas.microsoft.com/BizTalk/2003/system-properties.MessageType == http://CorrelationWithoutCommonElement.Employee#Employee

Correlated Receive shape has an instance subscription i.e. an instance of Orchestration is waiting for a particular type of message.

The main motive of this sample is to demonstrate how to do property promotion if you don’t have any common element.

Hope it helps.

Download the sample application from here. Word version of this blog is here.

 

Contact Me:- 

@Gmail, @Facebook , @Twitter, @LinkedIn , @MSDNTechnet, @My Personal Blog 

Accessing Orchestration Variables in Maps

Many times we feel the need of using Orchestration Variables in Maps. But there is no straight forward way to achieve it.

Below are the two ways through which we can achieve the same.

1. Is to create a dummy schema with Elements which needs to be used in Maps. In below example I have created a dummy schema with Element as “Age” because I wanted to access intAge variable in Map.
2. Is to assign some dummy value in map and later assign the actual value in Message Assignment shape. In below sample I have assigned “Null” (from String Concatenate functoid) and later within the Message Assignment shape assigned the value from Orchestration Variable (intAge).

Which method should be chosen depends on below points.

• 1^st technique should be used when you need this variable extensively like to perform some DBLookup etc. In below example “intAge” value is used as a lookup value and required in making decisions.
• 2^nd technique is preferable when you want to populate few fields without much processing.

Now let’s come to our Example.

In this post I will use this sample BizTalk solution and try to portray both the techniques.

Here we receive a simple XML having information of a person like- “Name”, “DateOfBirth” and “City”. In Orchestration will calculate the Age of the person and in Map on the basis of it’s value we will decide whether person in eligible for Voting, Marriage and Drinking or not.

Source & Destination Schema Image

As I needed value of Age variable to make decisions in Map, so I created another schema with just one element “Age” as shown below-

Age Schema Image

Created msgAge by below code-

xmlDoc.LoadXml(“<ns0:Person xmlns:ns0=’http://UseOrchestrationVariableInMaps.Age’><Age></Age></ns0:Person>&#8221;);

msgAge= xmlDoc;

msgAge.Age= intAge;

Now next step is to create a map which takes two schemas as input- Input schema and Age Schema and generates one Output Schema.

• Drag and drop the Transform shape
• Double click -> New Map-> Select two in Source Message(msgAge & msg In in our case)
• Select output schema as destination.
• Check the box to launch BizTalk Mapper.

Mapper Image

This will generate a map with two parts (InputMessagePart_0 & InputMessagePart_1), under “Root” record. It actually generates a multipart message with parts referring to each input/output message.

Now in map you can play around with the value of Age element. In our sample application I am performing below checks on its value.

1. Age >=18 -> Eligible for Voting
2. Age >=22-> Eligible for Marriage
3. Age >= 25-> Eligible for Drinking Alcohol

Don’t miss to see the irony here- In India being few months younger to 25 years allows you to marry a girl or choose your Prime Minister of country (& other representatives) but makes you criminal if you drink alcohol.

Read more about how to develop and test maps with multiple source and destination schemas here.

Map Image

Now let’s demonstrate the second technique to assign values in Orchestration.

For this you need Message Assignment shape along with Transform shape in Construct shape as shown below.

Message Assignment Shape Image

In Map assign Null(or any other dummy value) to all those elements for which Orchestration variables are required. For example in our sample Age element is assigned as NULL using string concatenate functoid.

Later in orchestration using Message Assignment shape assign the required values as shown above.

Hope it was helpful.

Download the sample application from here. Word version of this blog is here.

 

Contact Me:- 

@Gmail, @Facebook , @Twitter, @LinkedIn , @MSDNTechnet, @My Personal Blog 

How to Subscribe the Response Message of 2-Way (Request- Response) Send Port

Many times we have face a situation when we want to see the response received from a 2-way send port (request- response) after calling a web-service, performing some database operation etc. using two way send port. Normally such a scenario arises during testing phase.

There are multiple ways to subscribe to the response message received from the 2-way send port but the easiest option is explained below.

Benefits of below implementation:-

1. It doesn’t requires any code change
2. It doesn’t affects the original flow of the application

How to do it :-

Just create a Send Port with Filter Condition as

BTS.SPName = <Name of the 2-way Send Port>

Filter On Send Port Image

How does it works:-

Whenever a two way send port publishes the response back to message box, it promotes two properties

• SPName: – This context property holds the name of the Send Port which has published this message to MsgBox. We are exploiting this property only.

Context Properties of Response Message Image

• Correlation Token: – For a two way send port End Point Manager (EPM) creates Instance subscription for receive shape with subscription for Correlation Token.

In BizTalk we have two subscriptions – Activation and Instance.

Activation Subscription: – A new instance of the subscriber is created when a message is received. For example first receive shape in Orchestration with “Activate = True”.

Instance Subscription: – Incoming message is routed to an already-running instance of the subscriber. Unique instance ID is stored in the subscription table in the master Message Box.

For example correlated receive in Sequential or Parallel Convoys or in two way send port.

I have created a sample application which consumes a simple “Hello World WCF Service”.

Download the sample application here.

Don’t forget to edit the binding files before running the test.

Get the MSWord version of this blog from here.

Contact Me:- 

@Gmail, @Facebook , @Twitter, @LinkedIn , @MSDNTechnet, @My Personal Blog 

Implementing Scatter Gather Pattern in BizTalk using Self Correlation

This is one of the most widely used integration patterns and also the most asked interview question.

In this pattern a huge message (also called batch file) is debatched and sent to multiple or single recipient which process these messages and sends a response or acknowledgement back. Later all these responses are gathered and collated into a single message in Parent Orchestration. So basically this complete process can be divided into two sub processes- Splitter Pattern and Gatherer pattern.

You can read more about the Scatter and Gather Pattern here.

There can be many ways to implement this pattern in BizTalk but below implementation is asynchronous and loosely coupled.

Basics:-

a) For debatching the huge message (batch file) I have called/executed Receive Pipeline with XML Disassembler component from within the Parent Orchestration. This acts as Splitter.

b) Sent out the debatched messages using Start Orchestration shape, this will allow asynchronous processing of all the child messages(debatched messages)

c) For gathering the responses in parent orchestration I have used Self Correlation, acts as Gatherer

Detailed Steps:-

Parent Orchestration

1) Create a parent orchestration to receive input message (batch file) with “Transaction Type = Long Running”. This scope should be long running because it will contain an Atomic Scope for debatching.

2) Do some processing/transformation if required.

3) Create an Atomic Scope for debatching the big message. Here the scope should be Atomic because we will be calling/executing Pipeline within it and the pipeline is of non- serializable type.

4) Execute Receive Pipeline (having XML Disassembler component) in Expression Shape for debatching.

    varReceivePipelineOutput= Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline(typeof(TestDebatchedProcessing.ReceivePipeline_DebatchInput),msgFormatted);

Here “varReceivePipelineOutput” is a variable of type- “Microsoft.XLANGs.Pipeline.ReceivePipelineOutputMessages” and should be defined within the Atomic Scope created for debatching. This class extends IEnumerator interface so implements GetCurrent and MoveNext methods.

Note: –

You will have to add reference to “Microsoft.XLANGs.Pipeline” dll present at below location – <Install dir>\Microsoft BizTalk Server 2013\Microsoft.XLANGs.Pipeline.dll

Read more about debatching XML messages in orchestration here.

5) Loop through all the debatched messages using “varReceivePipelineOutput.MoveNext()”. MoveNext function indicates whether the pipeline should move to the next message.

6) Construct Child Message using Message Assignment shape.

    msgChild=null; varReceivePipelineOutput.GetCurrent(msgChild); ->This will retrieves the current XLANG message.

7) Increase the count variable.

    intCount = intCount + 1;

8) Start Child Orchestration for further processing by passing Child Message and PortType (Created to receive response from Child Orchestration at step 10).

Start Orchestration Configuration Image

Note: –

• You will have to create Child orchestration before this step. Developing Child orchestration will be explained below later in this page.
• Create a Configured Port to Receive Port to receive messages from Child Orchestration

     Steps :-

a) Right Click on the Port Surface -> New configured Port- > Give PortName and PortType

b) Select Port Direction as “I’ll always be receiving message on this port” and Port Binding as “Direct” and type as “Self    Correlating” as shown below.

Self Correlated Port Configuration Image

9) Create a Loop to receive all the messages sent out. Here we loop till the value of intcount becomes zero.

     intCount > 0

10) Receive Processed message from the Child Orchestration, this receive shape is connected to the Self Correlated Receive port created in step 8-b-2

11) Reduce count variable

      intCount= intCount -1;

 Full Parent Orchestration Image

Child Orchestration

1) Create a Child Orchestration which receives debatched messages from Parent Orchestration and processes it further.

As first step create Orchestration Parameters

a) Port_SelfCorr -> Port Parameter to receive PortType

b) msgChild

1. Right Click Orchestration Parameters -> New Configured Port Parameter -> Give a desired Name
2. “Use Existing Port Type” and select Self Correlated Port created in Parent Orch as shown below
3. Click Next and give Port Direction as “I’ll always be sending message on this port”

Configured Port Parameter In Child Orchestration Image

3. Do some processing (I have done some transformation).

4. Send the Processed Message back to Parent Orchestration. This send port is connected to the port created while creating Port Parameter

 Full Child Orchestration Image

You can find the complete solution here.

Contact Me:- 

@Gmail, @Facebook , @Twitter, @LinkedIn , @MSDNTechnet, @My Personal Blog 

Getting Error in BizTalk Send Port- Attempted to access an unloaded AppDomain

Recently one of my customer using BizTalk Server 2009 faced this issue, where he had SOAP adapter to connect to a web service but got the below error-

Error: –

“Attempted to access an unloaded AppDomain”

Cause: –

The SOAP adapter runs in the IIS process space. If more than one Web service exists in the IIS AppPool, then every Web service ends up having its own AppDomain.

By default all messaging engine objects are created in the first AppDomain (i.e. the AppDomain corresponding to the first Web service).

If the first Web service is inactive for an extended period of time for any reason, IIS unloads the first AppDomain. When this happens, all services in the hosting process become unusable.

 

Resolution:-

This issue can be successfully resolved in 2010 and above solution by enabling “Default application domain for isolated adapter” in Isolated hosts settings.

Detailed Documentation at- http://msdn.microsoft.com/en-us/library/aa577833.aspx

But before 2010 we did not had this setting in host properties. After some research we ended up making below registry changes: –

Set (or Add if it doesn’t exists) the following registry key to 1:

HKLM\System\\CurrentControlSet\\Services\\BTSSvc.3.0\UseDefaultAppDomainForIsolatedAdapter

Detailed Documentation at- http://msdn.microsoft.com/en-us/library/aa577833(v=bts.20).aspx

 

Important:-

For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the below article. http://support.microsoft.com/kb/322756

Contact Me:- 

@Gmail, @Facebook , @Twitter, @LinkedIn , @MSDNTechnet, @My Personal Blog