All posts by Prashant Singh

My name is Prashant Kumar Singh, I live in Bangalore-India. I started working on BizTalk back in August 2010, since then my main focus is on Integration Platforms using Microsoft BizTalk Server, .NET, WCF, and SOAP/XML/XSLT. Recently I have started working on Azure, specially Logic Apps, API Apps etc. I’m an active blogger and a participant on the MSDN BizTalk Server Forums. I've in-depth and breadth knowledge/experience in almost all BizTalk versions: 2006, 2006 R2, 2009, 2010, 2013, 2013 R2 and 2016. While working in Microsoft I got to learn the internals of BizTalk along with covering it end to end. Currently I am working as Premier Support Engineer at Microsoft. Before this, I have worked with Fidelity and Wipro Technologies. This blog acts as a simple notebook for arranging my thoughts and daily experiences. So far I have understood that success is always inclusive and In Heaven We Feed Each Other (One of my favorite stories- http://www.wisdomcommons.org/wisbits/3241-in-heaven-we-feed-each-other). If it benefits you in any way, I will deem the motive of writing this blog is fulfilled. Please do not hesitate to leave me a comment or directly email me in case you need any help or if you have any suggestion. DISCLAIMER: The views and opinions stated in this blog are mine and do not necessarily reflect those of my employer. Each posting on this blog is provided “AS IS” with no warranties, and confers no rights. Contact Me:- @Gmail, @Facebook, @LinkedIn, @MSDN Technet Profile, @My Personal Blog Site

Monitoring BizTalk Resources programmatically using C#

Hi friends many times as a BizTalk Administrator we fall into situations when we need to constantly monitor BizTalk Servers resources and services like Host Instances Status, any custom services like IIS, CPU usage, available disk space etc.

So, in this blog I have created a console application which can be scheduled to run continuously using Windows Scheduler. It will constantly monitor all resources and attempt to correct it and later drop an email to configured group or user (only if there is an issue).

Note – Proactive Monitoring ensures early resolutions, thus solving upcoming big future problems.

Currently I have focused on monitoring of below items –

1) Check ENTSSO, IIS And Other Custom Services
2) Enumerate And Start All Host Instances
3) Check If Host is in Hung State – “Start Pending” or “Stop Pending”
4) Check Available Drive Space
5) Check CPU Reading
6) Check All Suspended and Custom Orchestration Suspended instance count
7) Check Spool Count, Parts Count and row count for other tables if it is more than the configured value
8) Check BizTalkMsgBox Table Index Fragmentation

In multi-server BizTalk environment this tool needs to be scheduled in only in one environment and provide all other server names of this group in configuration file – “ServerNames” as shown below. It will remotely query all the servers.

<add key=”ServerNames” value=”BTSServer1;BTSServer2″/>

1) Check ENTSSO IIS And Other CustomServices –

In this function we can monitor important Services like ENTSSO, IIS(World Wide Web Publishing Service – w3svc) or any other custom service. You just need to mention these services in config file as shown below –

<add key=”AdditionalServicesToBeMonitored” value=”ENTSSO;W3SVC;RTI”/>

I have used “ServiceController” class to fetch list of all the services, check its status and finally start the same.

ServiceController[] scServices = ServiceController.GetServices(strServerName);

foreach (ServiceController service in scServices)
 {
 if ((service.ServiceName.ToUpper().Contains(strServiceName)) && service.Status != ServiceControllerStatus.Running)
 {
service.Start();}
 }

2) Enumerate And Start Host Instances –

In this function we are enumerating all the In-Process Host Instances using WMI Query and starting them.

Sample code -

//Create EnumerationOptions and run wql query 
 EnumerationOptions enumOptions = new EnumerationOptions();
 enumOptions.ReturnImmediately = false;

//Search for all HostInstances of 'InProcess' type in the Biztalk namespace scope 
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostType=1 And IsDisabled=False", enumOptions);


//Enumerate through the result set and start each HostInstance if it is already stopped 
 foreach (ManagementObject inst in searchObject.Get())
 {
 //Check if ServiceState is 'Stopped' 
 if (inst["ServiceState"].ToString() == "1")
 { inst.InvokeMethod("Start", null); }}

3) Check If Host is in “Start Pending” or “Stop Pending” State

Many times BizTalk Host Instances go in Hung State like “Stop Pending” or “Start Pending” and if you leave these processes for days they won’t come up. So, the only option left is to Terminate(End Process Tree) using Task Manager and start the host again. I totally agree that it’s not the best way as the service will not exit gracefully but in production environment we are left with very little option.

Here also I am using WMI queries to find if the Host is in Hung State i.e. Start or Stop Pending state.

 //Search for all HostInstances of 'InProcess' type in the Biztalk namespace scope which is not Disabled and have Service State as 2(Start Pending) or 3(Stop Pending) 
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostType=1 And IsDisabled=False and (ServiceState = 2 or ServiceState = 3)", enumOptions);

After waiting for some configured milliseconds I am performing again a WMI Query to check if the Host Status is changed or not. Because this issue occurs very rarely and mostly “Start Pending” or “Stop Pending” state will last for very less time when you have performed Start or Stop of Host Instances.

//Query again to check if the status is still the same i.e. Stopped/Start Pending
 ManagementObjectSearcher searchObject1 = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostName='" + inst["HostName"].ToString() + "' and (ServiceState = 2 or ServiceState = 3)", enumOptions);

Once if it’s confirmed that Host is hung we will go ahead and Terminate it forcibly along with all other processes with same parentID.

 var connectionOptions = new ConnectionOptions();
 ManagementScope scope = new ManagementScope(@"\\" + strServerName + @"\root\cimv2", connectionOptions);
 scope.Connect();
 //Get Process ID of the service. BizTalk Host Instance Name is always like BTSSvc$ like BTSSvc$BizTalkServerApplication
 var query = new SelectQuery("select ProcessID from Win32_Service where Name ='BTSSvc$" + strHostName + "'");
 using (var searcher = new ManagementObjectSearcher(scope, query))
 {
 foreach (ManagementObject obj in searcher.Get())
 {
 uint processId = (uint)obj["ProcessId"];
 //Hard Terminate(Terminate Related Processes) - Loop through all the processes on the machine and Kill it if it's Parent Process ID or ProcessID matches the Service Process ID
 var parentIdQuery = new SelectQuery("Select * from Win32_process");
 using (var vsearcher = new ManagementObjectSearcher(scope, parentIdQuery))
 { 
foreach (ManagementObject process in vsearcher.Get()) {
 //Checking if Parent Process ID or ProcessID is same as Service ProcessID
 if (processId == (uint)process["ParentProcessID"] || processId == (uint)process["ProcessID"])
 { process.InvokeMethod("Terminate", null); } } } } }

4) Check Available Drive Space – 

Here too I am using WMI query to check Available Disk Space. This query will be performed remotely on all the servers.

For remotely connecting WMI Query, namespace will be  –

string strNameSpace = @”\\” + strServerName + @”\root\cimv2″;

Query – select FreeSpace,Size,Name from Win32_LogicalDisk where DriveType=3″, enumOptions

string strNameSpace = @"\\" + strServerName + @"\root\cimv2";
 //Get Fixed disk state. Check - http://www.csidata.com/custserv/onlinehelp/vbsdocs/vbs41.htm, DriveType = 2 - Fixed, Drive has fixed (nonremovable) media. This includes all hard drives, including hard drives that are removable. DriveType = 2 -Remote, Network drives. This includes drives shared anywhere on a network.
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher(strNameSpace, "select FreeSpace,Size,Name from Win32_LogicalDisk where DriveType=3", enumOptions);

5) Check CPU Reading – 

I am using Performance Counter – % Processor Time. Here we Sleep the thread to achieve average value for configured time.

PerformanceCounter totalProcessorTimeCounter = new PerformanceCounter("Processor", "% Processor Time", "_Total", strServerName);
 totalProcessorTimeCounter.NextValue();
 System.Threading.Thread.Sleep(intThreadSleepTimeInMilliSeconds);// seconds wait
 double CPUPercentage = totalProcessorTimeCounter.NextValue();
 if (CPUPercentage > doubleAcceptableCPUPercentage)

6) Check All Suspended and Custom Orchestration Suspended instance count

Here also I am using WMI query to fetch total suspended instances – ServiceClass=1(Orchestration) or ServiceClass=4(Messaging) and (ServiceStatus=4(Suspended (resumable)) or ServiceStatus=32(Suspended (non-resumable)) in the BizTalk namespace

//Search for all Service Instances for ServiceClass=1(Orchestration) or ServiceClass=4(Messaging) and (ServiceStatus=4(Suspended (resumable)) or ServiceStatus=32(Suspended (non-resumable)) in the Biztalk namespace scope 
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_ServiceInstance Where (ServiceClass=1 or ServiceClass=4) and (ServiceStatus=4 or ServiceStatus=32)", enumOptions);

It can be configured to get Suspended Instance for individual Orchestrations as well, which can be configured in app.config file.

WQL Query – string.Format(“SELECT * FROM MSBTS_ServiceInstance WHERE (ServiceClass=1 or ServiceClass=4) and (ServiceStatus=4 or ServiceStatus=32) AND ServiceName like \”%{0}%\””, strOrchName);

 //Now checking for individual Orchestrations Only If Suspended Instance Count not zero
 int count = 1;
 foreach (string strOrchName in strOrchestrationNames.Split(';'))
 {
 string strWQL = string.Format("SELECT * FROM MSBTS_ServiceInstance WHERE (ServiceClass=1 or ServiceClass=4) and (ServiceStatus=4 or ServiceStatus=32) AND ServiceName like \"%{0}%\"", strOrchName);

7) Spool Count, Parts Count and other tables with Row Count if more than configured value –

Performing SQL Query on BizTalk Message box to get top 10 tables with row count more that the acceptable configured value. This information can be used to proactively prevent throttling and avoid slowness of BizTalk message processing.

//Checking top 10 tables with Row count more than intAcceptableMsgBoxRowCount
string queryString = “SELECT top 10 T.name TableName, I.Rows NumberOfRows FROM sys.tables T JOIN sys.sysindexes I ON T.OBJECT_ID = I.ID “
+ “WHERE indid IN (0,1) and I.Rows > ” + intAcceptableMsgBoxRowCount.ToString() + “ORDER BY I.Rows DESC,T.name”;
8) Check BizTalkMsgBox Table Index Fragmentation –

To check Fragmentation I am using below query. It checks if Fragmentation is more than the configured acceptable value. In case of high Fragmentation you will have to Rebuild Indexes to improve performance. Execute below stored procedures during down time.

1.bts_RebuildIndexes —-   for BizTalkMsgBoxDb database
2.dtasp_RebuildIndexes—  for BizTalkDTADb database

//Checking Index Fragmentation if the value is more than Acceptable Fragmentation Percentage, Selecting top 10 values
“SELECT top 10 dbtables.[name] as ‘Table’, dbindexes.[name] as ‘Index’, indexstats.avg_fragmentation_in_percent, indexstats.page_count

FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS indexstats INNER JOIN sys.tables dbtables on dbtables.[object_id] = indexstats.[object_id] INNER JOIN sys.schemas dbschemas on dbtables.[schema_id] = dbschemas.[schema_id] INNER JOIN sys.indexes AS dbindexes ON dbindexes.[object_id] = indexstats.[object_id] AND indexstats.index_id = dbindexes.index_id

WHERE indexstats.database_id = DB_ID()and avg_fragmentation_in_percent > ” + doubleAcceptableFragmentationPercentage.ToString() + ” “;

ORDER BY indexstats.avg_fragmentation_in_percent desc”;

 

Complete Solution Code can be found here

App.config file - 

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
 <startup> 
 <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
 </startup>
 <appSettings>
 <!--BizTalk Message Box Connection String-->
 <add key="BizTalkMsgBoxConnectionString" value="Data Source=WIN-4ACT86KFN44;Initial Catalog=BizTalkMsgBoxDb;Integrated Security=true;Pooling=false"/>
 
 <!--SMTP Details for Sending Emails-->
 <add key="IsEmailRequired" value="true"/>
 <add key="SMTPHost" value="smtp.gmail.com"/>
 <add key="FROMEmailID" value="youremailid@gmail.com"/>
 <add key="TOEmailID" value="toemailid@gmail.com,toemailid2@microsoft.com"/>
 <add key="SMTPServerPort" value="587"/>
 <!--<add key="Username" value="youremailid@gmail.com"/>
 <add key="Password" value="yourpassword"/>-->
 <add key="Subject" value="Monitoring BizTalk Resources"/>

<add key="ServerNames" value="WIN-4ACT86KFN44"/>
 <add key="AdditionalServicesToBeMonitored" value="ENTSSO;W3SVC;RTI"/>
 <add key ="AcceptableAvailableDriveFreeSpace" value="80"/>
 <add key="AcceptableTotalSuspendedInstanceCount" value="1"/>
 <add key="OrchestrationNames" value="ProcessCandidateAgeIfno;ProcessOrders;DummyOrchName"/>
 <add key="AcceptableSuspendedInstanceCountForAnOrch" value="1"/>
 <add key="AcceptableCPUPercentage" value="1"/>
 <add key="AcceptableMsgBoxRowCount" value="1"/> 
 <add key="AcceptableFragmentationPercentage" value="1"/>
 <add key ="ThreadSleepTimeInMilliSeconds" value="5000"/>
 </appSettings>
</configuration>

 

C# Code - 

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Management;
using System.Net.Mail;
using System.Configuration;
using System.ServiceProcess;
using System.IO;
using System.Diagnostics;
using System.Data.SqlClient;

namespace MonitoringBizTalkResources
{
 class MonitoringBizTalkResources
 {
 static void Main(string[] args)
 {
 #region Get Values from Config File

string strEmailBody = "";
 var appSettings = ConfigurationManager.AppSettings;

//Get Server Names for checking services
 string strServerNames = "";
 if (appSettings["ServerNames"] != null && !String.IsNullOrEmpty(appSettings["ServerNames"].ToString()))
 { strServerNames = Convert.ToString(appSettings["ServerNames"].ToString()); }

//Getting Additional Services To Be Monitored
 string strAdditionalServicesToBeMonitored = "";
 if (appSettings["AdditionalServicesToBeMonitored"] != null && !String.IsNullOrEmpty(appSettings["AdditionalServicesToBeMonitored"].ToString()))
 { strAdditionalServicesToBeMonitored = Convert.ToString(appSettings["AdditionalServicesToBeMonitored"].ToString()); }




//Getting connection string from Config File
 string strConnectionString = "";
 if (appSettings["BizTalkMsgBoxConnectionString"] != null && !String.IsNullOrEmpty(appSettings["BizTalkMsgBoxConnectionString"].ToString()))
 { strConnectionString = Convert.ToString(appSettings["BizTalkMsgBoxConnectionString"].ToString()); }

//If value of IsEmailRequired field is present in config file will populate the same else will keep it as default = false
 bool IsEmailRequired = false;
 if (appSettings["IsEmailRequired"] != null && !String.IsNullOrEmpty(appSettings["IsEmailRequired"].ToString()))
 { IsEmailRequired = Convert.ToBoolean(appSettings["IsEmailRequired"].ToString()); }

//Get ThreadSleepTimeInMilliSeconds
 int intThreadSleepTimeInMilliSeconds = 5000;
 if (appSettings["ThreadSleepTimeInMilliSeconds"] != null && !String.IsNullOrEmpty(appSettings["ThreadSleepTimeInMilliSeconds"].ToString()))
 { intThreadSleepTimeInMilliSeconds = Convert.ToInt32(appSettings["ThreadSleepTimeInMilliSeconds"].ToString()); }

//If value of AcceptableAvailableDriveSpace field is present in config file will populate the same else will keep it as default
 double doubleAcceptableFreeDriveSpace = 80;
 if (appSettings["AcceptableAvailableDriveFreeSpace"] != null && !String.IsNullOrEmpty(appSettings["AcceptableAvailableDriveFreeSpace"].ToString()))
 { doubleAcceptableFreeDriveSpace = Convert.ToDouble(appSettings["AcceptableAvailableDriveFreeSpace"].ToString()); }

//If value of AcceptableTotalSuspendedInstanceCount field is present in config file will populate the same else will keep it as default
 int intAcceptableTotalSuspendedInstanceCount = 10000;
 if (appSettings["AcceptableTotalSuspendedInstanceCount"] != null && !String.IsNullOrEmpty(appSettings["AcceptableTotalSuspendedInstanceCount"].ToString()))
 { intAcceptableTotalSuspendedInstanceCount = Convert.ToInt32(appSettings["AcceptableTotalSuspendedInstanceCount"].ToString()); }

//Get Orchestration Names from Config
 string strOrchestrationNames = "";
 if (appSettings["OrchestrationNames"] != null && !String.IsNullOrEmpty(appSettings["OrchestrationNames"].ToString()))
 { strOrchestrationNames = appSettings["OrchestrationNames"].ToString(); }

//Get AcceptableSuspendedInstanceCountForAnOrch from config
 int intAcceptableSuspendedInstanceCountForAnOrch = 100;
 if (appSettings["AcceptableSuspendedInstanceCountForAnOrch"] != null && !String.IsNullOrEmpty(appSettings["AcceptableSuspendedInstanceCountForAnOrch"].ToString()))
 { intAcceptableSuspendedInstanceCountForAnOrch = Convert.ToInt32(appSettings["AcceptableSuspendedInstanceCountForAnOrch"].ToString()); }

//Get AcceptableCPUPercentage from Config
 double doubleAcceptableCPUPercentage = 80;
 if (appSettings["AcceptableCPUPercentage"] != null && !String.IsNullOrEmpty(appSettings["AcceptableCPUPercentage"].ToString()))
 { doubleAcceptableCPUPercentage = Convert.ToDouble(appSettings["AcceptableCPUPercentage"].ToString()); }

//Get AcceptableFragmentationPercentage from Config
 double doubleAcceptableFragmentationPercentage = 50;
 if (appSettings["AcceptableFragmentationPercentage"] != null && !String.IsNullOrEmpty(appSettings["AcceptableFragmentationPercentage"].ToString()))
 { doubleAcceptableFragmentationPercentage = Convert.ToDouble(appSettings["AcceptableFragmentationPercentage"].ToString()); }

//Get Acceptable MsgBox RowCount for a table from config
 int intAcceptableMsgBoxRowCount = 30000;
 if (appSettings["AcceptableMsgBoxRowCount"] != null && !String.IsNullOrEmpty(appSettings["AcceptableMsgBoxRowCount"].ToString()))
 { intAcceptableMsgBoxRowCount = Convert.ToInt32(appSettings["AcceptableMsgBoxRowCount"].ToString()); }

#endregion

try
 {
 //Check if Enterprise Single Sign-On Service or IIS is stopped
 checkENTSSOIISAndCustomServices(ref strEmailBody, ref strServerNames, ref strAdditionalServicesToBeMonitored);

//Check all the Host Instances
 enumerateAndStartHostInstances(ref strEmailBody);

//Check if Host is in Start Pending or Stop Pending State
 checkIfHostStartorStopPendingState(ref strEmailBody, ref intThreadSleepTimeInMilliSeconds);

//Check Drive Free Space 
 checkAvailableDriveSpace(ref strEmailBody, ref strServerNames, ref doubleAcceptableFreeDriveSpace);

//Check CPU Reading
 checkCPUReading(ref strEmailBody, ref strServerNames, ref doubleAcceptableCPUPercentage, ref intThreadSleepTimeInMilliSeconds);

//Check All Suspended Instance Count
 checkAllSuspendedInstances(ref strEmailBody, ref intAcceptableTotalSuspendedInstanceCount, ref strOrchestrationNames, ref intAcceptableSuspendedInstanceCountForAnOrch);

//Check Spool Count, Parts Count and other tables with Row Count more than intAcceptableMsgBoxRowCount
 checkBizTalkMsgBoxTablesCount(ref strEmailBody, ref strConnectionString, ref intAcceptableMsgBoxRowCount);

//Check BizTalkMsgBox Table Index Fragmentation
 checkBizTalkMsgBoxTablesIndexFragmentation(ref strEmailBody, ref strConnectionString, ref doubleAcceptableFragmentationPercentage);

if (IsEmailRequired && !String.IsNullOrEmpty(strEmailBody))
 {
 string strHTMLStyle = "";
 strEmailBody = "<html>"+strHTMLStyle+"<body>" + strEmailBody + "</body></html>";
 sendEmail(strEmailBody);
 }
 }
 catch (Exception excep)
 {
 string strExceptionMessage = "MonitoringBizTalkResource - An exception occurred. " + excep.Message;
 if (!String.IsNullOrEmpty(strEmailBody))
 strExceptionMessage = strExceptionMessage + Environment.NewLine + "Operations performed so far - " + strEmailBody;
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", strExceptionMessage, System.Diagnostics.EventLogEntryType.Error);
 }
 }
 //Function to check if any Host is in Stop Pending or Start Pending State
 private static void checkIfHostStartorStopPendingState(ref string strEmailBody, ref int intThreadSleepTimeInMilliSeconds)
 {
 try
 {
 //Create EnumerationOptions and run wql query 
 EnumerationOptions enumOptions = new EnumerationOptions();
 enumOptions.ReturnImmediately = false;

//Search for all HostInstances of 'InProcess' type in the Biztalk namespace scope which is not Disabled and have Service State as 2(Start Pending) or 3(Stop Pending) 
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostType=1 And IsDisabled=False and (ServiceState = 2 or ServiceState = 3)", enumOptions);
 //ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostType=1 And IsDisabled=False and (ServiceState = 1 or ServiceState = 4)", enumOptions); // Test Query for Stopped(1) and Running(4) State

int count = 1;
 //Enumerate through the result set
 foreach (ManagementObject inst in searchObject.Get())
 {
 if (count == 1) // Wait only for the first time
 {
 System.Threading.Thread.Sleep(intThreadSleepTimeInMilliSeconds);// seconds wait
 }

//Query again to check if the status is still the same i.e. Stopped/Start Pending
 ManagementObjectSearcher searchObject1 = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostName='" + inst["HostName"].ToString() + "' and (ServiceState = 2 or ServiceState = 3)", enumOptions);
 //ManagementObjectSearcher searchObject1 = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostName='" + inst["HostName"].ToString() + "' and (ServiceState = 1 or ServiceState = 4)", enumOptions); //Test Query for Stopped(1) and Running(4) State
 foreach (ManagementObject inst1 in searchObject1.Get())
 {
 //Terminate the Host Instance
 terminateHungProcess(inst1["RunningServer"].ToString(), inst1["HostName"].ToString());

//Start the Host Instance
 inst1.InvokeMethod("Start", null);
 //Draft Email Subject
 if (count == 1)
 {
 strEmailBody = strEmailBody + "Below Host Instances were in Stopped/Start Pending state, Terminated the process and attempted to Start<br>";
 strEmailBody = strEmailBody + "<table><tr><th>Server Name</th><th>Host Instance Name</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "" + inst1["RunningServer"] + "" + inst1["HostName"] + "";
 }
 }
 if (count > 1)
 { strEmailBody = strEmailBody + "</table>All above mentioned Host Instances have been Started Successfully<br><br>"; }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in enumerateAndStartHostInstances fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in enumerateAndStartHostInstances fuction call. " + excep.Message);
 }
 }

//Function to Terminate a Process in Hung State and also terminate all related processes(Having same ParentPID)
 private static void terminateHungProcess(string strServerName, string strHostName)
 {
 try
 {
 var connectionOptions = new ConnectionOptions();
 ManagementScope scope = new ManagementScope(@"\\" + strServerName + @"\root\cimv2", connectionOptions);
 scope.Connect();
 //Get Process ID of the service. BizTalk Host Instance Name is always like BTSSvc$<HostName> like BTSSvc$BizTalkServerApplication
 var query = new SelectQuery("select ProcessID from Win32_Service where Name ='BTSSvc$" + strHostName + "'");
 using (var searcher = new ManagementObjectSearcher(scope, query))
 {
 foreach (ManagementObject obj in searcher.Get())
 {
 uint processId = (uint)obj["ProcessId"];
 //Hard Terminate(Terminate Related Processes) - Loop through all the processes on the machine and Kill it if it's Parent Process ID or ProcessID matches the Service Process ID
 var parentIdQuery = new SelectQuery("Select * from Win32_process");
 using (var vsearcher = new ManagementObjectSearcher(scope, parentIdQuery))
 {
 foreach (ManagementObject process in vsearcher.Get())
 {
 //Checking if Parent Process ID or ProcessID is same as Service ProcessID
 if (processId == (uint)process["ParentProcessID"] || processId == (uint)process["ProcessID"])
 {
 process.InvokeMethod("Terminate", null);
 }
 }
 }
 }
 }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in terminateHungProcess fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in terminateHungProcess fuction call. " + excep.Message);
 }
 }

//Function to check and start ENTSSO Service
 private static void checkENTSSOIISAndCustomServices(ref string strEmailBody, ref string strServerNames, ref string strAdditionalServicesToBeMonitored)
 {
 try
 {
 int count = 1;
 foreach (string strServerName in strServerNames.Split(';'))
 {
 ServiceController[] scServices = ServiceController.GetServices(strServerName);
 foreach (string strServiceName in strAdditionalServicesToBeMonitored.Split(';'))
 {
 foreach (ServiceController service in scServices)
 {
 if ((service.ServiceName.ToUpper().Contains(strServiceName)) && service.Status != ServiceControllerStatus.Running)
 {
 if (count == 1)
 {
 strEmailBody = strEmailBody + "Below Services are not running, attempting to start.<br>";
 strEmailBody = strEmailBody + "<table><tr><th>Server Name</th><th>Service Name</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "" + strServerName + "" + service.DisplayName + "";
 service.Start();
 }
 }
 }
 }
 if (count > 1)
 { strEmailBody = strEmailBody + "</table>All above mentioned Services have been Started Successfully<br><br>"; }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in checkENTSSOIISAndCustomServices fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in checkENTSSOServiceAndIIS fuction call. " + excep.Message);
 }
 }

//Function to Enumerate all HostInstances of "InProcess" type and start them 
 private static void enumerateAndStartHostInstances(ref string strEmailBody)
 {
 try
 {
 //Create EnumerationOptions and run wql query 
 EnumerationOptions enumOptions = new EnumerationOptions();
 enumOptions.ReturnImmediately = false;

//Search for all HostInstances of 'InProcess' type in the Biztalk namespace scope 
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_HostInstance Where HostType=1 And IsDisabled=False", enumOptions);
 int count = 1;
 //Enumerate through the result set and start each HostInstance if it is already stopped 
 foreach (ManagementObject inst in searchObject.Get())
 {
 //Check if ServiceState is 'Stopped' 
 if (inst["ServiceState"].ToString() == "1")
 {
 if (count == 1)
 {
 strEmailBody = strEmailBody + "Below Host Instances are not running, attempting to start.<br>";
 strEmailBody = strEmailBody + "<table><tr><th>Server Name</th><th>Host Instance Name</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "" + inst["RunningServer"] + "" + inst["HostName"] + "";
 inst.InvokeMethod("Start", null);
 }
 }
 if (count > 1)
 { strEmailBody = strEmailBody + "</table>All above mentioned Host Instances have been Started Successfully<br><br>"; }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in enumerateAndStartHostInstances fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in enumerateAndStartHostInstances fuction call. " + excep.Message);
 }
 }

//Function to check available drive space
 private static void checkAvailableDriveSpace(ref string strEmailBody, ref string strServerNames, ref double doubleAcceptableFreeDriveSpace)
 {
 try
 {
 int count = 1;
 foreach (string strServerName in strServerNames.Split(';'))
 {
 //Connection credentials to the remote computer, not needed if the logged account has access
 ConnectionOptions oConn = new ConnectionOptions(); //oConn.Username = "DummyUserName";//oConn.Password = "*********"; 
 //Create EnumerationOptions and run wql query 
 EnumerationOptions enumOptions = new EnumerationOptions();
 enumOptions.ReturnImmediately = false;

string strNameSpace = @"\\" + strServerName + @"\root\cimv2";
 //Get Fixed disk state. Check - http://www.csidata.com/custserv/onlinehelp/vbsdocs/vbs41.htm, DriveType = 2 - Fixed, Drive has fixed (nonremovable) media. This includes all hard drives, including hard drives that are removable. DriveType = 2 -Remote, Network drives. This includes drives shared anywhere on a network.
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher(strNameSpace, "select FreeSpace,Size,Name from Win32_LogicalDisk where DriveType=3", enumOptions);
 //Enumerate through the result set 
 foreach (ManagementObject driveInfo in searchObject.Get())
 {
 double percentFreeSpace = 100 * (System.Convert.ToDouble(driveInfo["FreeSpace"]) / System.Convert.ToDouble(driveInfo["Size"]));
 if (percentFreeSpace < doubleAcceptableFreeDriveSpace)
 {
 if (count == 1)
 {
 strEmailBody = strEmailBody + "Available Free Space for below drives is less than acceptable value of " + doubleAcceptableFreeDriveSpace.ToString() + "%
";
 strEmailBody = strEmailBody + "<table><tr><th>Server Name</th><th>Drive Name</th><th>Total Size</th><th>Available Free Space</th><th>Percentage Free Space</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "" + strServerName + "" + driveInfo["Name"] + "" + driveInfo["Size"] + "" + driveInfo["FreeSpace"] + "" + percentFreeSpace + "%";
 }
 }
 }
 if (count > 1)
 { strEmailBody = strEmailBody + "</table><br><br>"; }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in checkAvailableDriveSpace fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in checkAvailableDriveSpace fuction call. " + excep.Message);
 }
 }

//Function to check CPU Reading
 private static void checkCPUReading(ref string strEmailBody, ref string strServerNames, ref double doubleAcceptableCPUPercentage, ref int intThreadSleepTimeInMilliSeconds)
 {
 try
 {
 int count = 1;
 foreach (string strServerName in strServerNames.Split(';'))
 {
 PerformanceCounter totalProcessorTimeCounter = new PerformanceCounter("Processor", "% Processor Time", "_Total", strServerName);
 totalProcessorTimeCounter.NextValue();
 System.Threading.Thread.Sleep(intThreadSleepTimeInMilliSeconds);// seconds wait
 double CPUPercentage = totalProcessorTimeCounter.NextValue();
 if (CPUPercentage > doubleAcceptableCPUPercentage)
 {
 if (count == 1)
 {
 strEmailBody = strEmailBody + "CPU Usage on below servers is higher than the acceptable value of " + doubleAcceptableCPUPercentage.ToString() + "%
";
 strEmailBody = strEmailBody + "<table><tr><th>Server Name</th><th>CPU Percentage</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "" + strServerName + "" + CPUPercentage.ToString() + "%";
 }
 }
 if (count > 1)
 { strEmailBody = strEmailBody + "</table><br><br>"; }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Error Occurred in checkCPUReading function call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Error Occurred in checkCPUReading function call. " + excep.Message);
 }
 }

//Function to Check Suspended Instances
 private static void checkAllSuspendedInstances(ref string strEmailBody, ref int intAcceptableTotalSuspendedInstanceCount, ref string strOrchestrationNames, ref int intAcceptableSuspendedInstanceCountForAnOrch)
 {
 try
 {
 //Create EnumerationOptions and run wql query 
 EnumerationOptions enumOptions = new EnumerationOptions();
 enumOptions.ReturnImmediately = false;

//Search for all Service Instances for ServiceClass=1(Orchestration) or ServiceClass=4(Messaging) and (ServiceStatus=4(Suspended (resumable)) or ServiceStatus=32(Suspended (non-resumable)) in the Biztalk namespace scope 
 ManagementObjectSearcher searchObject = new ManagementObjectSearcher("root\\MicrosoftBizTalkServer", "Select * from MSBTS_ServiceInstance Where (ServiceClass=1 or ServiceClass=4) and (ServiceStatus=4 or ServiceStatus=32)", enumOptions);
 if (searchObject.Get().Count > 0)
 {
 if (intAcceptableTotalSuspendedInstanceCount < searchObject.Get().Count)
 {
 strEmailBody = strEmailBody + "The total number of Suspended instances is higher than the acceptable value of " + intAcceptableTotalSuspendedInstanceCount.ToString() + "
";
 strEmailBody = strEmailBody + "<table><tr><th>Total Number of Suspended Instance</th></tr>";
 strEmailBody = strEmailBody + "" + searchObject.Get().Count.ToString() + "";
 strEmailBody = strEmailBody + "</table><br><br>";
 }

//Now checking for individual Orchestrations Only If Suspended Instance Count not zero
 int count = 1;
 foreach (string strOrchName in strOrchestrationNames.Split(';'))
 {
 string strWQL = string.Format("SELECT * FROM MSBTS_ServiceInstance WHERE (ServiceClass=1 or ServiceClass=4) and (ServiceStatus=4 or ServiceStatus=32) AND ServiceName like \"%{0}%\"", strOrchName);
 ManagementObjectSearcher searcherServiceInstance = new ManagementObjectSearcher(new ManagementScope("root\\MicrosoftBizTalkServer"), new WqlObjectQuery(strWQL), null);
 if (searcherServiceInstance.Get().Count > intAcceptableSuspendedInstanceCountForAnOrch)
 {
 if (count == 1)
 {
 strEmailBody = strEmailBody + "The total number of Suspended instances for below Orchestrations is higher than the acceptable value of " + intAcceptableSuspendedInstanceCountForAnOrch.ToString() + "
";
 strEmailBody = strEmailBody + "<table><tr><th>Orchestration Name</th><th>Number of Suspended Instances</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "" + strOrchName + "" + searcherServiceInstance.Get().Count.ToString() + "";
 }
 }
 if (count > 1)
 { strEmailBody = strEmailBody + "</table><br><br>"; }
 }
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Error Occurred in checkAllSuspendedInstances function call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Error Occurred in checkAllSuspendedInstances function call. " + excep.Message);
 }
 }

//Function to check Fragmentation of BizTalkMsgBoxDb Table Indexes
 private static void checkBizTalkMsgBoxTablesIndexFragmentation(ref string strEmailBody, ref string strConnectionString, ref double doubleAcceptableFragmentationPercentage)
 {
 try
 {
 if (!String.IsNullOrEmpty(strConnectionString))
 {
 //Checking Index Fragmentation if the value is more than Acceptable Fragmentation Percentage, Selecting top 10 values
 string queryString = "SELECT top 10 dbtables.[name] as 'Table', dbindexes.[name] as 'Index', indexstats.avg_fragmentation_in_percent, indexstats.page_count ";
 queryString = queryString + "FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) AS indexstats INNER JOIN sys.tables dbtables on dbtables.[object_id] = indexstats.[object_id] INNER JOIN sys.schemas dbschemas on dbtables.[schema_id] = dbschemas.[schema_id] INNER JOIN sys.indexes AS dbindexes ON dbindexes.[object_id] = indexstats.[object_id] AND indexstats.index_id = dbindexes.index_id ";
 queryString = queryString + "WHERE indexstats.database_id = DB_ID()and avg_fragmentation_in_percent > " + doubleAcceptableFragmentationPercentage.ToString() + " ";
 queryString = queryString + "ORDER BY indexstats.avg_fragmentation_in_percent desc";
 using (SqlConnection connection = new SqlConnection(strConnectionString))
 {
 using (SqlCommand command = new SqlCommand(queryString, connection))
 {
 connection.Open();
 SqlDataReader reader = command.ExecuteReader();
 if (reader.HasRows)
 {
 strEmailBody = strEmailBody + "Fragmentation Percentage for few indexes is more than the Acceptable Value of " + doubleAcceptableFragmentationPercentage.ToString() + "% <br>";
 strEmailBody = strEmailBody + "<table><tr><th>Table Name</th><th>Index Name</th><th>Average Fragmentation in Percent</th><th>Page Count</th></tr>";
 while (reader.Read())
 {
 strEmailBody = strEmailBody + "<tr><td>" + reader["Table"] + "</td><td>" + reader["Index"] + "</td><td>" + reader["avg_fragmentation_in_percent"] + "</td><td>" + reader["page_count"] + "</td></tr>";
 }
 strEmailBody = strEmailBody + "</table>" + Environment.NewLine + "<br><br>";
 }
 }
 }
 }
 else throw new Exception("BizTalk Msg Box Connection String property is NULL");
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in checkBizTalkMsgBoxTablesIndexFragmentation fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in checkBizTalkMsgBoxTablesIndexFragmentation fuction call. " + excep.Message);
 }
 }

//Function to check Spool and Parts table Count and other tables with row count more than intAcceptableMsgBoxRowCount
 private static void checkBizTalkMsgBoxTablesCount(ref string strEmailBody, ref string strConnectionString, ref int intAcceptableMsgBoxRowCount)
 {
 try
 {
 int count = 1;
 if (!String.IsNullOrEmpty(strConnectionString))
 {
 //Checking top 10 tables with Row count more than intAcceptableMsgBoxRowCount
 string queryString = "SELECT top 10 T.name TableName, I.Rows NumberOfRows FROM sys.tables T JOIN sys.sysindexes I ON T.OBJECT_ID = I.ID "
 + "WHERE indid IN (0,1) and I.Rows > " + intAcceptableMsgBoxRowCount.ToString() + "ORDER BY I.Rows DESC,T.name";
 using (SqlConnection connection = new SqlConnection(strConnectionString))
 {
 using (SqlCommand command = new SqlCommand(queryString, connection))
 {
 connection.Open();
 SqlDataReader reader = command.ExecuteReader();
 if (reader.HasRows)
 {
 while (reader.Read())
 {
 if (count == 1)
 {
 strEmailBody = strEmailBody + "The Row Count of Below Tables is more than the acceptable value of " + intAcceptableMsgBoxRowCount + "
";
 strEmailBody = strEmailBody + "<table><tr><th>Table Name</th><th>Current Row Count(Exceeded Value)</th></tr>";
 ++count;
 }
 strEmailBody = strEmailBody + "<tr><td>" + reader["TableName"].ToString() + "</td><td>" + reader["NumberOfRows"] + "</td></tr>";
 }
 if (count > 1)
 strEmailBody = strEmailBody + "</table><br><br>";
 }
 }
 }
 }
 else throw new Exception("BizTalk Msg Box Connection String property is NULL");
 }
 catch (Exception excep)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Exception Occurred in checkBizTalkMsgBoxTablesCount fuction call. " + excep.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Exception Occurred in checkBizTalkMsgBoxTablesCount fuction call. " + excep.Message);
 }
 }

//Functoin to send email
 private static void sendEmail(string strEmailBody)
 {
 //Get SMTP details from config
 var appSettings = ConfigurationManager.AppSettings;

try
 {
 //if ((appSettings["SMTPHost"] != null) && (appSettings["FROMEmailID"] != null) && (appSettings["TOEmailID"] != null) && (appSettings["Subject"] != null) && (appSettings["SMTPServerPort"] != null) && (appSettings["Username"] != null) && (appSettings["Password"] != null))
 //{
 MailMessage mail = new MailMessage();
 SmtpClient SmtpServer = new SmtpClient(appSettings["SMTPHost"].ToString());
 mail.From = new MailAddress(appSettings["FROMEmailID"].ToString());
 mail.To.Add(appSettings["TOEmailID"].ToString());
 mail.Subject = appSettings["Subject"].ToString();
 mail.Body = strEmailBody;
 mail.IsBodyHtml = true;

SmtpServer.EnableSsl = true;
 //SmtpServer.DeliveryMethod = SmtpDeliveryMethod.Network;
 //SmtpServer.UseDefaultCredentials = false;
 SmtpServer.Port = System.Convert.ToInt32(appSettings["SMTPServerPort"]);
 //SmtpServer.Credentials = new System.Net.NetworkCredential(appSettings["Username"].ToString(), appSettings["Password"].ToString());

SmtpServer.Send(mail);
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Email Sent", System.Diagnostics.EventLogEntryType.SuccessAudit);
 //}
 //else
 //{
 // System.Diagnostics.EventLog.WriteEntry("BizTalk Server", "Error occurred while sending email - Some mandatory configuration values is missing. Please check if corresponding values are given in config file- SMTPHost, FROMEmailID, ToEmailID, Subject, SMTPServerPort, Username, Password");
 // throw new Exception("Error occurred while sending email - Some mandatory configuration values is missing. Please check if corresponding values are given in config file- SMTPHost, FROMEmailID, ToEmailID, Subject, SMTPServerPort, Username, Password");
 //}
 }
 catch (Exception expMailSend)
 {
 System.Diagnostics.EventLog.WriteEntry("BizTalk Server", expMailSend.Message, System.Diagnostics.EventLogEntryType.Error);
 throw new Exception("Error occurred while sending email - " + expMailSend.Message);
 }
 }
 }
}

Sample Email –

SampleEmail

Download the complete code from here

I will keep on adding new functionality so make sure you donwload the latest version of code.

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

 

Advertisements

Error – “Microsoft .NET Framework 3.5 Service Pack 1 is required”- Specify an alternate source path

Hello Everyone.

Recently, I wanted to setup BizTalk 2013 R2 system and while installing SQL Server 2014 I ran into below error –

Error Message – “Microsoft .NET Framework 3.5 Service Pack 1 is required”.

Setup failed to find “Microsoft .NET Framework 3.5 Service Pack 1” and therefore we can’t proceed with the installation.

Setup failed to find "Microsoft .NET Framework 3.5 Service Pack 1" and therefore we can't proceed with the installation.

So, I went ahead and tried to install this Feature from Server Manager by following below steps –

  1. Open Server Manager and click on “Add roles and features”

Add Roles and Features Wizard

2. Select “Role-based or feature-based installation”

Add Roles and Features Wizard 2

3. Select appropriate server from the Server Pool

Add Roles and Features Wizard_ChooseServer

4. Select .Net Framework 3.5

Add Roles and Features Wizard_Select.NETFramework 3.5

5. Now you just need to confirm your selection and proceed with installation. But for me it showed below warning –

Warning – “Do you need to specify an alternate source path? One or more installation selections are missing source file on the destination server.”

Add Roles and Features Wizard_SpecifyAlternateSourcePath

If you ignore this warning and proceed with the installation, it will fail with below error.

Error Message – The source files could not be found. Try installing the roles, role services or features again in a new Add Roles and Features Wizard session, and on the Confirmation page of the wizard, click “Specify an alternate source path” to specify a valid location of the source files that are required for the installation. The location must be accessible by the computer account of the destination server.

FeatureInstallationFailed

So, the question against me was how to find and where to find an alternate source for .Net Framework 3.5 Feature. So, I spent lots of time on internet attempting to find same without any luck.

Finally I got a SQL expert who guided me with the solution, which is as below-

The alternate source path can be found in Windows Operating System media within below location –

<OS Media Drive>:\sources\sxs

Add Roles and Features Wizard_SpecifyAlternateSourcePath2

Simply say “OK” and proceed with the installation. It should be successful this time.

Add Roles and Features Wizard_FeatureInstallSuccessful

Later, proceed with SQL Installation, this time you will not receive the same error and installation will be smooth.

Hope this helps.

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

BizTalk Server Migration from 2010 to 2013 R2- Why & How

This document is intended to give an overview of why & how migration from BizTalk Server 2010 to 2013 R2 can be accomplished. It also attempts to aggregate known issues or probable challenges which can erupt during this process.

Why to Migrate

Mainstream Lifecycle of BizTalk 2010 is going to end on 1/12/2016 post which Microsoft Support is limited and for extended support you will have to purchase Extended Support within 90 days of mainstream support end (which can be missed easily).

BizTalk Version Mainstream Support End Extended Support End
BizTalk Server 2010 1/12/2016 1/12/2021
BizTalk Server 2013 7/10/2018 7/11/2023
BizTalk Server 2013 R2 7/10/2018 7/11/2023

Read more about Lifecycle Information for Microsoft Server Products Support

There are lots of disadvantages of being in extended support:

  • Non-Security hotfix support will require extended hotfix agreement, purchased within 90 days of mainstream support ending
  • You incident support will be chargeable
  • You cannot make warranty claims
  • You cannot request design changes and feature requests

You probably do not want to run your business on limited support, so it’s better to upgrade if your current version is already in extended support or your mainstream support is coming.

Planning Migration

Microsoft always claims that the upgrade will be seamless and will not affect anything. However, there is always a big risk of something going wrong. Because the stack we are upgrading has lots of things like Operating System, .Net Framework, SQL, BizTalk etc. so we need to be super careful.

Upgrade vs. Migration

Two approaches can be considered:

· Upgrading of the existing BizTalk Server 2010 Infrastructure to BizTalk Server 2013 R2(In House Upgrade)

· Building a new BizTalk Server 2013 R2 Infrastructure to co-exist with the existing BizTalk Server 2010 Infrastructure to allow for gradual migration of BizTalk Applications

Benefits of In-House Upgrade

This allows you to upgrade BizTalk within the existing infrastructure. For ex- If you have a standalone BizTalk Server 2010 server, you can upgrade Operating System, SQL, BizTalk etc. just by clicking on the installer and proceeding with Upgrade option.

For more details refer In-house BizTalk Upgrade or Upgrading to BizTalk Server 2013 from BizTalk Server 2010-2009

UpgradeImage

Disadvantages of In-house upgrade

· As many softwares like Windows OS, SQL, Visual Studio, BizTalk etc. needs to be upgraded, there is higher risk of failure.

· Generally we would like to upgrade/enhance the hardware as well (better CPU, RAM, OS to 64bit etc.) which is not possible in this case.

This approach is only suitable for small businesses with simpler integration.

Migration

Here the BizTalk environment must be recreated on an operating system supported by BizTalk Server 2013 R2 and later all other upgraded applications needs to be deployed.

Steps to perform Migration

1) Recreate BizTalk Servers 2013 R2 environment with desired and supported Hardwares and Softwares.

BizTalk Server 2013 R2 Supported Softwares

Microsoft Windows Windows Server 2012 R2, Windows Server 2012, Windows 8.1, Windows 7 SP1.
Internet Information Services (IIS) The version that comes with the operating system. KB 224609 lists the versions.
Microsoft .NET Framework .NET Framework 4.5
Microsoft Visual Studio Visual Studio 2013
Microsoft SQL Server SQL Server 2014 or SQL Server 2012 SP1

Refer Hardware and Software Requirements for BizTalk Server 2013 and 2013 R2 for more details.

Follow How to install & Configure BizTalk Server 2013 R2 document for installing and configuring BizTalk.

2) Upgrade existing BizTalk Applications to latest version

a) Copy all the existing projects(folders) to newly created BizTalk 2013 R2 Servers

b) Open each solution one by one which needs to be migrated in Visual Studio 2013, this will automatically open a migration wizard. Make sure Target Framework is .Net Framework 4.5 for all projects. Finally a “Migration Report” will be generated, as shown in pic below. Go through the report to see if it has any error message.

MigrationPopUp

Migration Report

c) Ensure to change the .Net Framework versions for all related class libraries (helper class/dlls) manually to .NET 4.5 versions, by default Helper classes (Class Library) will not be upgraded.

HelperClass.NetFramework

d) Rebuild all the applications and fix any build errors.

e) Upgrade/Replace any obsolete method with their latest version. Ex- Use “System.Configuration.ConfigurationManager.GetSection” instead of “System.Configuration.ConfigurationSettings.GetConfig”.

f) You may also have to upgrade/modify your code to incorporate changes occurred in Standard External Libraries.

Ex-In our case we were using older version of “Microsoft Enterprise Library” dll which was later upgraded to 6.0 which made us to change our implementation/code as well.

g) Ensure to modify deprecated Adapters. For example SQL & SOAP adapters are deprecated in BizTalk 2013 R2, so they must be replaced by their corresponding WCF Adapters.

Note: – Deprecated doesn’t mean non-availability, it means that though you are able to use these adapters in current version of BizTalk but in future version they might get removed. Microsoft even may deny providing support. Ex- SQL adapter was deprecated with BizTalk Server 2009 was released and is now unsupported in BizTalk Server 2010 onwards.

Read here for more details.

h) Take necessary steps for upgrading any other dependencies if required.

Later these applications can be used for regular deployment in any BizTalk environments.

Note: – All Applications should be thoroughly tested in all Pre-PROD environments.

Ideally Migration Effort is minimal as most of the work is done by the wizard, but BizTalk being a middleware depends on many external systems like Databases, SAP, SFTP etc. and so the main cost/effort is of Thorough Testing.

Also, if you decide to perform Code Refactor/Enhancement efforts will be increased accordingly. For example- If you have any SOAP Web Service (.asmx) and you want to upgrade it to WCF (.svc) it will need extra efforts.

Hope it helps.

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

The map name was not provided and is required.

Recently while working on my first ESB POC I faced one strange issue. I was using BRE Resolver to resolve the map name for transformation, however I was getting below error.

Error Message:- There was a failure executing the receive pipeline: “Microsoft.Practices.ESB.Itinerary.Pipelines.ItinerarySelectReceiveXml, Microsoft.Practices.ESB.Itinerary.Pipelines, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35” Source: “ESB Dispatcher” Receive Port: “GetEmpRequest_RP” URI: “E:\Prashant\POC\IN\*Copy*.xml” Reason: The map name was not provided and is required.

I tested my Rule using “Test Policy” functionality in “Business Rule Composer” which was giving the expected output. I even looked into the ESB Traces using Debug View but all in vain.

I was using “Microsoft.Practices.ESB.ResolverProviderMessage” as Document Type as suggested in many tutorials and videos.

After struggling for hours and I found the below blog-

https://maximelabelle.wordpress.com/2010/06/10/resolving-multiple-maps-in-sequence-with-the-busines-rules-engine-in-biztalk-esb-toolkit-2-0/

Which suggests that the type of the incoming message is mistakenly determined to be “Microsoft.Practices.ESB.ResolveProviderMessage” i.e. Resolve instead of Resolver.

debug_view_log

Later by making Document Type as above things started working.

But this proves how little time and money Microsoft has invested in development of ESB Toolkit.

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

Step by Step Guide to Install & Setup ESB Toolkit and Management Portal (Standalone Machine)

Installing and Configuring ESB Portal is really hectic & cumbersome task. I myself faced many roadblocks so thought to document it here, hope it’s useful.

Pre-requisites:

1. Install Both VS 2012 and 2013 (BT 2013 Dev tools & ESB toolkit 2.3 requires some components from VS 2012 which I haven’t figure out yet if you can figure out please share it across).

2. Install SQL Server with Analysis Services and ensure it’s running under the LocalSystem account

3. Install BizTalk Server 2013 R2, LOB WCF Adapters and ESB Toolkit.

4. Configure BizTalk with full BAM configuration. Ensure to check “Enable Analysis Services for BAM aggregations”, give the Server Name and Apply Configurations.

5. Install Enterprise Library 5.0 from below link-

http://www.microsoft.com/en-us/download/details.aspx?id=15104

6. Install Microsoft Report Viewer Redistributable 2008 from below link-

http://www.microsoft.com/en-us/download/details.aspx?id=577

Add Roles & Features:-

Roles:-

a) IIS with IIS 6 compatibility Tools(complete)

b) Windows Authentication & Basic Authentication(Under : Web Server(IIS) -> Security)

Features:-

a) WCF & HTTP Activation for both .Net 3.5 and 4.0/4.5

Setting up the ESB Samples and BAM

1. Add the System Account to the BAM STAR Database

a. Open SQL Management studio and connect to server having BAM Star Schema database

b. Run the following command:

use BAMStarSchema

EXEC sp_addrolemember ‘db_owner’, ‘NT AUTHORITY\SYSTEM’

2. Create a folder named: C:\Projects\Microsoft.Practices.ESB

3. Unzip C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit\ESBSource.zip file into this folder

4. Remove Read-only attribute from all files in C:\Projects\Microsoft.Practices.ESB folder.

5. Open an elevated Visual Studios command prompt

a. Setup BAM

i. Navigate to C:\Program Files (x86)\Microsoft BizTalk Server 2013 R2\Tracking

ii. Run Bm.exe deploy-all -DefinitionFile:”C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit\Bam\Microsoft.BizTalk.ESB.BAM.Itinerary.xml”

    iii. Run Bm.exe deploy-all -DefinitionFile:”C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml”

b. Re-register the Itinerary Visual Studio templates

i. Run devenv.exe /setup

c. Create the strong name key for the Samples

i. Run sn -k c:\Microsoft.Practices.ESB.snk

ii. Copy this to C:\Projects\Microsoft.Practices.ESB\Keys

6. Fix the WCF Resubmit Port

a. Navigate to C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit\Web\ESB.ItineraryServices.WCF

b. Open the web.config file

c. Locate: <soapHeaderMetadata enabled=”true”> and Update to be <soapHeaderMetadata enabled=”false”>

Configure ESB

1) Run ESB Configuration Tool as administrator

2) In the ESB Configuration Tool, from the left pane, select ESB Configuration. On the right pane, for Database Server, specify the database server name where the databases required for ​BizTalk ESB Toolkit will be created.

3) In the IIS Web Services box, specify the user credentials under which the applications required for BizTalk ESB Toolkit are created. Also specify the name of the website in IIS under which the applications are created.

Note: – At this stage, you can click Apply Configuration towards the top of the ESB Configuration Tool to configure the BizTalk ESB Toolkit with the default settings. However, if you want to do a custom configuration, you can perform the remaining steps as well. In such a case, the values you specify in the subsequent steps take precedence over the default values.

4) From the left pane, expand Exception Management, and do the following:

5) Click the Database node, and clear the Enable Exception Management Database check box if you do not want to configure an exception management database.

6) Similarly, on the Database node, select the Use Existing Database checkbox if you want to use an existing database instead of the configuration tool creating a new database. You can also specify the database server name and the database name.

7) On the Exception Web Services node, clear the Enable Exception Services check box if you do not want to configure these services. Alternatively, if you want to run these services under a different website, you can specify that here too.

8) From the left pane, expand ESB Core Components, and do the following:

9) Click the Itinerary Database node, and clear the Itinerary Database check box if you do not want to configure an itinerary database.

10) Similarly, on the Itinerary Database node, select the Use Existing Database checkbox if you want to use an existing database instead of the configuration tool creating a new database. You can also specify the database server name and the database name.

11) On the Core Web Services node, clear the Enable Core Services check box if you do not want to configure these services. Alternatively, if you want to run these services under a different website, you can specify that here too.

12) From the left pane, click Configuration to specify the SSO configuration. If you are installing and configuring the BizTalk ESB Toolkit in a single server environment, you should select File Configuration Source, which is the default selection. However, if you are setting up a multiple-machine deployment, you must select the SSO Configuration Source, and then provide the following values.

13) SSO Server: Name of the SSO server.

14) Configuration file: Click the ellipsis button (…), and then browse to the Esb.config file, which is included in the ​Microsoft BizTalk ESB Toolkit

15) Application Name: Type a name for the SSO application. For example, ESB Toolkit.

16) Contact Information: Type the appropriate contact information in the following format: someone@example.com.

17) Administrator Group Name: Click the ellipsis button (…), and then browse to the appropriate name

18) User Group Name: Click the ellipsis button (…), and then browse to the appropriate name

19) Click Apply Configuration. Open IIS and notice that the applications required for ​BizTalk ESB Toolkit are now created under the website you specified while configuring ​BizTalk ESB Toolkit.

20) In the ESB Configuration Tool, on the ESB BizTalk Applications node, from the right pane, do the following:

21) Select the Enable ESB Core Components in BizTalk Server check box to create the application in the BizTalk Server Administration console. Select the Use Default Binding to bind this application to the default host. Select the Do not use Default Binding if you do not want to bind the application to the default host. In such a case, you must explicitly bind the application to a host, once the application is created.

22) Select the Enable ESB JMS/WMQ Components in BizTalk Server check box to create the application in the BizTalk Server Administration console. Select the Use Default Binding to bind this application to the default host. Select the Do not use Default Binding if you do not want to bind the application to the default host. In such a case, you must explicitly bind the application to a host, once the application is created.

23) Click Apply Configuration to create the applications you selected. Verify that the applications are created in the BizTalk Server Administration console.

Note: – Status of the configuration will be visible on the bottom left of this tool. Wait till it displays message – EsbBizTalkApplication: Configured

Refer: – https://msdn.microsoft.com/en-us/library/jj684558.aspx for more details

Configuring ESB Management Portal

  1. In Visual Studio 2012, open ESB.Portal.sln found in C:\Projects\Microsoft.Practices.ESB\Source\Samples\Management Portal\ESB.Portal
  2. Click OK to the dialog pop up about the solution being unsupported by Visual Studio 2012
  3. Close the Migration Report IE window that opens once migration is complete
  4. In the ESB.BAM.Service.Implementation C# project add a reference to Microsoft.Practices.ServiceLocation.dll. This can be found at C:\Program Files (x86)\Microsoft Enterprise Library 5.0\Bin
  5. Open web.config file found in ESB.Portal project. Comment out the following section if present(ideally shouldn’t be there in BizTalk 2013 R2):

<sectionGroup name=”scripting”>…

  1. Save all
  2. Select the ESB.Portal project, right-click rebuild this project (note the full Solution will not build)
  3. Close visual studios
  4. Open a 32-Bit PowerShell windows as Administrator
  5. Type: Set-ExecutionPolicy –ExecutionPolicy Unrestricted (confirm Y when promoted)
  6. Run Management_Install.cmd file found in C:\Projects\Microsoft.Practices.ESB\Source\Samples\Management Portal\Install\Scripts

If everything is good, there shouldn’t be anything in RED.

  1. Finally Open http://localhost/ESB.Portal

Ensure you click on the Gray Gear Icon and change the selection to All Applications

Note: – Add the User (Identity) for which ESB Application Pools is configured and “Network Service” to BizTalk Isolated Host Users group. This will allow these websites to post messages in BizTalkMessageBox.

BizTalk Tracing

ESB processing can get complicated, so I would always suggest you to enable tracing.

1. In Notepad open the machine.config file for both 32 & 64 bit, placed at below locations

C:\Windows\Microsoft.NET\Framework\v4.0.30319\CONFIG\machine.config for 32-bit
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\CONFIG\machine.config for 64-bit

2. Locate the line with the closing configSections tag.

3. Under that line paste the following section:

<system.diagnostics>

<switches>

<add name=”BizTalkESBToolkit” value=”4″/>

</switches>

</system.diagnostics>

4. Save and close machine.config.

5. Start the DebugView program (Download)

6. In DebugView, on the Capture menu, click Capture Global Win32 to make sure that it is checked.

7. In the BizTalk Server Administration console, restart the BizTalkServerApplication host instance.

Refer below link for more details: –

https://midheach.wordpress.com/2013/05/13/tracing-with-esb-toolkit-2-2/

Known Issues:-

1) ESB Toolkit BRE Itinerary Resolver Fails with Exception:-

The following exception is thrown with ESB 2.2 & 2.3 on Microsoft BizTalk Server 2013 when the ItinerarySelectReceiveXML receive pipeline is used to call the Business Rules Engine to dynamically select an itinerary (using the ESB BRI Resolver).

Error Message:-

Exception has been thrown by the target of an invocation.

Source: Microsoft.Practices.ESB.Resolver.ResolverMgr

Solution: – It requires some changes in ESB.Config file, please check the below link about this issue.

https://support.microsoft.com/en-us/kb/2887594?wa=wsignin1.0

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

Very Strange behavior of XML Disassembler- Property Promotion not happening properly

While working on a pipeline component I encountered a very strange behavior of XML disassembler. I had few promoted properties which I was trying to access in a pipeline component in Validate stage after the execution of XML Disassembler. As we all know that the Pipelines are executed in sequential manner and the output of one pipeline component is used as input to the next one in the sequence(except in Disassemble stage). ( Read more- Understanding Pipeline Execution). So, ideally after execution of XML Disassembler, Promoted Properties should be accessible in later stages. But shockingly there value was NULL. Now let’s have a look into my sample BizTalk application for better understanding. I have a simple schema in which both the properties are promoted- Source&PropertySchema Emp & Property Schema Image I created a simple pipeline component where I tried to read values of ID and Name from message context using below code and later promoted “ReceiveFileName” FetchingPromotedProps Fetching Promoted Properties in Pipeline Component Image Created a simple Custom Receive Pipeline with XML Disassembler and later used my pipeline component in Validate Stage. ReceivePipeline Receive Pipeline Image But strangely when I tried to fetch the promoted properties found out that it’s returning NULL as shown below. PromotedPropsAsNullAfterCommentingTheReadOperation Promoted Properties as NULL in Pipeline component Image Cause:- After some research got to know that it’s a known behavior. Basically, the XmlDisassembler does not read through the incoming message immediately.  It waits until some later component or the MessageBox reads it.  This is done for various performance reasons like memory consumption, etc. So, since my custom pipeline component is next in line, when it gets the Message, data has not yet been read by the XmlDisassembler even though it passed that Stage in the Pipeline, so it hasn’t had a chance to Read and Promote any Context Properties.  This is by design, and what you are seeing is a known side-effect. Its accomplished by wrapping the data in a custom Stream class.  That is why it only gets read and processed when some other component reads it. Solution/Workaround:- The easiest option would be to just do you work with the Properties in an Orchestration. If you absolutely have to use a Pipeline Component, then you will have to somehow force a Read by the XmlDisassembler. For example in our pipeline component we have read the values using below code and forced a Read/Promote by the XmlDisassembler. //Reading message data using XML Document. System.Xml.XmlDocument xmlDoc = new System.Xml.XmlDocument(); xmlDoc.Load(originalStream); Using XMLDoc can have performance implication as the complete message will be loaded in memory, so you can use the below code as well. //Reading message data using Stream, this will not have any performance implication. System.IO.MemoryStream ms = new MemoryStream(); originalStream.CopyTo(ms); ms.Position = 0; inmsg.BodyPart.Data = ms; As shown in below image.  PromotedPropsAfterReadingTheValueUsingStream  Promoted Properties containing Value After performing Forced Read by XML Disassembler Image   Check the discussion over the same issue- XML Diassembler component is not doing property promotion   Hope it helps.   Download the sample application from here. Word version of this blog is here.  

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog