BizTalk Automated migration Tool

Watch the Video – BizTalk Automated Migration Tool Demo –

Microsoft has recently released BizTalk 2020 with lots of new features and enhancements.

There are many reasons to Migrate and can be broadly summarized as below –

  1. Avail Microsoft Support
    • BizTalk 2016 Mainstream Support Ends – 1st November 2022
    • All other previous versions like 2013R2, 2013 or 2010 etc. are long out of mainstream support
    • Upgrade to latest software version of OS, VS, SQL & BizTalk
  2. Enhance BizTalk Environment
    • For better infrastructure, performance, topology – HA/DR
  3. Avail New features
    • BizTalk 2020 is loaded with new features of Hybrid Integration, Improvised Security (TLS1.2, GDPR compliance), Better Monitoring etc.
    • Feature packs – Get new features faster, without waiting for new versions
    • New Features for Developers and Administrators
  4. Move to Cloud
    • Ability to run on Azure VM as IaaS

Microsoft has always attempted to make migrations as seamless and easy as possible by providing various options like –

  1. In-Place Upgrade – Only possible for N-1 version
  2. BizTalk Server Migration Tool – Which performs as-is migration by exporting artifacts from previous environment to new environment.

In-Place Upgrade is more like installing Cumulative Updates or Feature Pack. It requires minimum efforts and is mostly very seamless but has many limitations like only possible for N-1 version, rollback is tough.

BizTalk Server Migration Tool is provided by MSIT and is extremely helpful for as-is migration. On high level it exports MSI, Binding Files etc. from old environment and imports in the new environment. So all the dlls still point to previous version and nothing is changed.

For as-is migration, it definitely saves lots of time and efforts but has limitations like we can’t use new features or enhancements.

Therefore, many organisations prefer to use traditional method of code migration, which allows code upgrade, enhancement and possibility of modernization. But, this process is definitely very lengthy, time taking and resource intensive.

As a BizTalk Developer, Migration operations can be broadly categorized as below –

  1. Migrate to Latest Visual Studio – Open each solution in Visual Studio, by default it will upgrade solution to VS. It will also change the dotnet framework for all BizTalk projects.
  2. DotNet Framework Change – You may have to change DotNet Framework for Class Library Projects like Pipeline Components, Helper/Components, Custom Functoids, Custom Adapters etc. as they aren’t changed with VS Upgrade.
  3. Add BTDF or other Deployment Project (Optional)
  4. Rebuild Solution – Review failures and fix them.
  5. E2E Testing – End to end testing with all the parties involved in integration.

Performing all these operations for hundreds and thousands of applications is extremely tedious, time taking and resource intensive work. Every year companies spend thousands of man-hours just to perform these migration activities.

We attempted to reduce the efforts for developers by automating most of the routine tasks and only focus on things which really require human intervention.

We created a tool which solves top 4 problems and performs them as One-Click Operation.

Operations which can be Automated By Tool –

  1. Upgrade Solutions to Latest Visual Studio
  2. Change .Net Framework for all Projects
  3. Add BTDF Project to solution, with proper changes to Deployment.btdfproj file
  4. Rebuild Solution
  5. Detailed Logging of each operation for quick access to successful and failed solutions

Proposed Steps –

  1. “Browse for Folder with all applications”
  2. Get List of all the Solutions in that folder.
  3. Choose from Migration Operations –
    • Upgrade – This will upgrade to latest visual studio version.
    • Add BTDF Project – Adds BTDF Template project will all the files
    • Build – Performs Rebuild of all the solutions/projects
    • Change .Net Framework to Target Version – Select target Dotnet framework from the dropdown. All the projects will be migrated to this version, especially helpful for helper libraries like pipeline component, custom functoids, adapters and other helper components
  4. Detailed logging can be performed for each operation.
  5. You can also create folders and generate/move outputs to different folders like – “UpgradeSuccessful”, “UpgradeFailed”, “BuildSuccessful” and “BuildFailed”

This will allow developers to only focus on applications for which Build Operation failed.

Detailed Description of each operation –

Migrate Operation –

This operation is similar to manually Opening each solution in Visual Studio which performs

a) Upgrade to latest VS and

b) Change of .Net framework (to default version) for BizTalk projects (only BizTalk Projects(.btproj) not class library projects(.csproj)).

It also creates Backup folder with migrated projects and log file – “UpgradeLog.htm” next to the solution file. You can also choose to delete the backup file, to save space if you already have another backup.

To perform this operation from tool you can use visual studio command line arguments –

devenv {SolutionFile|ProjectFile} /Upgrade [/Out OutputFilename]

Refer –

https://docs.microsoft.com/en-us/visualstudio/ide/reference/upgrade-devenv-exe?view=vs-2019

Build Operation –

Similarly, Build operation can also be performed by VS command line arguments –

devenv SolutionName /Rebuild [SolnConfigName [/Project ProjName [/ProjectConfig ProjConfigName]] [/Out OutputFilename]]

Refer –

https://docs.microsoft.com/en-us/visualstudio/ide/reference/rebuild-devenv-exe?view=vs-2019

Most of the errors in this step is related to missing reference of common libraries or helper projects/dlls.

Change DotNet Framework –

You can allow users to choose target framework from a dropdown and later upgrade all projects in solution to this version. These values can be fetched from App.config file.

Your tool can get the list of all the projects referred/present in Solution (Solution file) and later change their .Net framework for each of them.

Regular VS Upgrade option performs .Net framework changes only to BizTalk projects and that too only to default version.

Note – For BizTalk 2020 we found a bug with .Net Framework 4.7.2, so we decided to migrate all the projects to 4.8 version. This tool was extremely helpful in mass framework change.

It’s also extremely helpful for helper class libraries like pipeline components, custom functoids, custom adapters or C# Helper/Component projects which are not upgraded in Upgrade process.

Dotnet Framework can be upgraded just by updating the project files.

  1. In the project file, locate the entry for the target Framework version. Ex -if your project is designed to use the .NET Framework 4.5, locate <TargetFrameworkVersion> v4.5 </ TargetFrameworkVersion > in the <PropertyGroup Label = “Globals” > element of the < Project > element.

If the<TargetFrameworkVersion> element isn’t present, your project doesn’t use the.NET Framework and no change is required.

2. Change the value to the Framework version you want, such as v4.7.2 or v4.8.

You can also use Regex pattern replace –

string regexPattern = “<TargetFrameworkVersion>*.*</TargetFrameworkVersion>”;

3. Save the changes and close the editor.

Refer –https://docs.microsoft.com/en-us/cpp/build/how-to-modify-the-target-framework-and-platform-toolset?view=vs-2019

Add BTDF Project –

In this step you can add a template of BTDF project with Deployment folder and all other files –

  • Deployment.btdfproj
  • InstallWizard.xml
  • License.rtf
  • PortBindingsMaster.xml
  • SettingsFileGenerator.xml
  • UnInstallWizard.xml

You can also update the Deployment.btdfproj file with many routine information like –

  • ProjectName – Solution File
  • Include Options for
    • Schemas
    • Maps/Transforms
    • Orchestrations
    • PipelineComponents
    • Helpers – Components
    • And many other options

These options can be read and populated by fetching project names or file’s extension from solution file.

For ex – If Solution contains projects with name as “Schemas” or files with extension “.xsd” then make IncludeSchemas as True.

Similarly, for Maps, Orchestrations, PipelineComponents etc. can be deduced either from name( if naming convention is followed for BizTalk projects) or file extensions like .btm for maps, .odx for orchestrations, .btp for pipelines etc.

Sample BTDF Template -
    <!-- Deploy schemas -->
    <IncludeSchemas>True</IncludeSchemas>
    <!-- Deploy orchestrations -->
    <IncludeOrchestrations>True</IncludeOrchestrations>
    <!-- Deploy maps -->
    <IncludeTransforms>True</IncludeTransforms>

Many sections like below which are mostly constant can be updated from App.Config file.

    <!-- Deploy custom functoids -->
    <IncludeCustomFunctoids>False</IncludeCustomFunctoids>
    <!-- Deploy BRE -->
    <IncludeVocabAndRules>False</IncludeVocabAndRules>
    <!-- Deploy IIS virtual directories -->
    <IncludeVirtualDirectories>False</IncludeVirtualDirectories>
    <!-- Deploy NUnit tests -->
    <IncludeDeploymentTest>False</IncludeDeploymentTest>
    <!-- Deploy Log4Net configuration file named $(ProjectName).log4net -->
    <Includelog4net>False</Includelog4net>

This tool has helped us in saving thousands of hours monotonous, repetitive and boring work.

Hope it helps you as well.

Feel free to get in touch in case of any query. Thanks.

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

Related Article –

PLANNING MIGRATION TO BIZTALK 2020 – WHY AND HOW?

https://prashantbiztalkblogs.wordpress.com/2020/08/28/planning-migration-to-biztalk-2020-why-and-how/

Contact Me –

Check out my other blogs –

Advertisement

Planning Migration to BizTalk 2020 – Why and how?

Watch the Video – Migration to BizTalk 2020 – Why and How?

As BizTalk 2020 is out most of the customers have either started migration to planning to migrate.

In this short blog, I will try to give a broader picture of planning BizTalk Migration and answer most basic questions.

So, let’s start with the most mammoth question first –

Question 1 – Why to Migrate?

There are many reasons to Migrate and can be broadly summarized as below –

  1. Avail Microsoft Support
    • BizTalk 2016 Mainstream Support Ends – 1st November 2022
    • All other previous versions like 2013R2, 2013 or 2010 etc. are long out of mainstream support
    • Upgrade to latest software version of OS, VS, SQL & BizTalk
  2. Enhance BizTalk Environment
    • For better infrastructure, performance, topology – HA/DR
  3. New features
    • BizTalk 2020 is loaded with new features of Hybrid Integration, Improvised Security (TLS1.2, GDPR compliance), Better Monitoring etc.
    • Feature packs – Get new features faster, without waiting for new versions
    • New Features for Developers and Administrators
  4. Move to Cloud
    • Ability to run on Azure VM as IaaS

Question 2 – What are my Migration Options?

a) Migrate to Azure –

Migration from BizTalk to Logic Apps/Azure Integration Services –

FeaturesBizTalk ServerAzureCapability/Description
Message RoutingBizTalk MessageBoxAzure Service Bus Queues & Topics, Event GridUse Topics and subscriptions, Event-driven approach
SchemasBizTalk SchemasLogic Apps along with Integration AccountSchemas can be uploaded to Integration Accounts or converted to JSON Schema and used
MappingBizTalk Mapper, XSLT, C#Azure Integration AccountUpload map(XSL file) to integration account
PipelinesBizTalk Pipeline and pipeline componentsLogic Apps along with Function AppLogic App for workflow and Function App for pipeline component logic
AdaptersOut-of-the-box adapters (application and protocol)Logic AppConnectors
OrchestrationBizTalk OrchestrationLogic AppDefine Workflow definition
RulesBizTalk Rule Engine (BRE)Function AppCreate custom Azure Functions to perform BRE like operations
BAMBizTalk Activity MonitoringLogic App, OMSTracked-properties, OMS portal. Specially Log Analytics for Logic Apps is very helpful
EDI Parties, Aggreement, PartnersBizTalk EDIAzure B2B integration using Logic Apps and Integration AccountUse Integration accounts to manage EDI Parties, Aggreements, Partners, upload certificates, maps, schemas etc.
API (REST, SOAP)Bindings in adapters (WCF)API ManagementAPI Management offers OpenAPI Swagger, WSDL and many other API related functionalities. API Logic can be written in Logic Apps and Function App
OperationsBizTalk Admin ConsoleAzure Portal, OMS, ARM-templates, PowerShell, CLIManaging Azure Services
MonitoringBizTalk Health MonitorAzure Monitor, Application InsightsAuzre OMS offers complete monitoring solution. Can also use Serverless360.
ArchivingBizTalk databases (Archive)Azure Storage, Application Insights, Log AnalysisLogging & Tracking

Refer below articles for more detail –

Migration BizTalk to Azure

https://www.serverless360.com/blog/migrating-biztalk-to-azure

b) Migrate to BizTalk Server 2020 –

  • In-Place Upgrade
  • Migrate to New Environment
    • As-Is Migration
    • BizTalk Server Migration Tool
  • Migrate and Improve
    • Traditional Migration of every solution
    • Possible enhancement of code, removing/replacing deprecated items

Let’s understand these options in detail –

In-Place Upgrade

Upgrading of the existing BizTalk Server 2016 Infrastructure to BizTalk Server 2020(In House Upgrade).

In-Place Upgrade Pros & Cons –

Advantages –

  • Extremely minimal efforts and time to upgrade
  • One click upgrade of BizTalk within the existing infrastructure.

Disadvantages –

  • Can only be done with N-1 Version
  • High Risk of Failure – Big stack to upgrade, must be compatible
    • OS, SQL, VS, BizTalk, 3rd Party softwares etc.
  • As-Is Upgrade – No change to infrastructure or code.
  • Code changes later will require upgrade to new VS, rebuild etc.
  • This approach is only suitable for small businesses with simpler integration

Migration – Parallel New Setup

Advantages –

  • Migration can be done in phases
  • Can scale up infrastructure
  • No downtime as we will have parallel environments
  • Can do the enhancements like replace deprecated features, code enhancements etc.

Disadvantages –

  • Requires a lot of efforts as all the applications need to be rebuilt, deployed and tested
  • Time taking, costly and lots of monotonous activities

Question 3 – I chose Migration to parallel environment. Can it be automated?

Option 1 – Migration Options – BizTalk Migration Tool(MSIT)

Microsoft’s internal team has provided a tool which assists heavily in as-is migration.

Refer https://docs.microsoft.com/en-in/archive/blogs/biztalk_server_team_blog/announcing-biztalk-server-migration-tool

Migration Tool Features –

Traditional Migration –

High Level Steps involved –

  • Recreate BizTalk Servers 2020 environment with desired (supported) Hardware & Software’s.
  • Application Level Migration
    • Uprade to Latest Visual Studio
    • Change of .Net Framework
    • Incorporate references and 3rd Party Softwares
    • Rebuild Code
    • End to End Testing
  • Possible Enhancements if any

Question 4 – Can we automate BizTalk Traditional Migration?

Yes, you can create One click Upgrade & Migration of thousands of BizTalk Solutions following below steps –

Major Operations to Automate –

  • Upgrade to Latest Visual Studio using Command line –

devenv {SolutionFile|ProjectFile} /Upgrade [/Out OutputFilename]

  • Rebuild Solutions – Use VS command line –

devenv SolutionName /Rebuild [SolnConfigName [/Project ProjName [/ProjectConfig ProjConfigName]] [/Out OutputFilename]]

  • Change DotNet Framework Version for all Projects – Specially helpful for class libraries – Pipeline components, custom functoids, custom adapters, Helpers

Replace TargetFrameworkVersion section in project files (.btpproj) with target framework

<TargetFrameworkVersion>DotNetTargetFramework</TargetFrameworkVersion>

  • Add BTDF Project or BizTalk Deployment project to solution

Use a template Deployment Folder with all the required files, update it with proper values at runtime

Refer below article for detailed implementation –

BIZTALK AUTOMATED MIGRATION

https://prashantbiztalkblogs.wordpress.com/2020/08/28/biztalk-automated-migration/

Question 5 – What’s New in BizTalk 2020?

I’m sure many of you must be already aware of the new features, so I’m bringing this topic in the end. It’s better to refer MS Official articles for more detail. I will just provide a brief summary.

Refer – What’s New in BizTalk Server 2020

https://docs.microsoft.com/en-us/biztalk/install-and-config-guides/whats-new-in-biztalk-server-2020

Support for newer platforms –

  • Visual Studio 2019
  • Windows Server 2019, Windows Server 2016, Windows 10
  • SQL Server 2019, SQL Server 2017, SQL Server 2016 SP2
  • DotNet Framework 4.7.2 or above
  • Office 2019, Office 2016

Hardware and Software Requirements for BizTalk Server 2020

New Office 365 Adapters – SMTP & POP3 Adapters are deprecated and replaced by Office 365 Adapters
– Adapter for Mail – Send and receive messages using Office 365 e-mail
– Adapter for Schedule – Set and update appointments using Office 365 schedules
– Adapter for Contact – Define people and groups using Office 365 contacts

Note – A new BizTalk TMS service will run in BizTalk application server, which will be responsible to get token for every office 365 account and cache it.

Better Integration with Azure – Hybrid Integration
– Logic Apps Adapter
– Service Bus Adapter
– Event Hubs Adapter
– Integration with API Management
– Publish Orchestration/Schema endpoints via APIM
– Publish APIs to external, partner & internal developers to unlock the potential of their data and services
– Adapter for Blob Storage
– Read and write binary files in Azure Blob Storage

Deployment and Administration Enhancements –

  • Design and Deploy with VSTS
    • Aplication Lifecycle Management to deploy and update BizTalk Server instances using Visual Studio Team System (VSTS)
  • Backup to Blob Storage
    • Backup BizTalk Server database job log files to Azure Blob Storage
  • Advanced Scheduling
    • Set up recurrence on BizTalk Server receive locations with greater precision using time zones, months, weeks, and days

Security Enhancements –

  • Transport Layer Security 1.2
    • Securely deploy BizTalk Server using TLS 1.2 authentication and encryption
  • SQL Server Encrypted Data
    • Read and write to SQL Server always encrypted columns using BizTalk Adapter for SQL Server

Enhanced Administration

  • REST APIs
    • Manage BizTalk environments using standard RESTful Web services APIs. Can create BizTalk 360 like portal yourself.
  • Read-only Operator Role
    • Facilitate dev ops model, provide production stamp without ability to update anything
  • Group Managed Service Accounts
    • Extend windows GMSA support to BizTalk operations and services
    • No need to change passwords anymore with GMSA
  • Audit Log
    • Making BizTalk further secure by maintaining audit trails of all management operation – One of the most important feature for auditing

Enhanced Analytics and Reporting

  • Application Insights
    • Send BizTalk tracking data to Application Insights for extensible analytics – understand Trends on usage, performance, custom business metrics
  • Event Hubs
    • Send BizTalk Server tracking data to Event Hubs
  • Power BI
    • Analyze & report BizTalk tracking data using Power BI. Use operational data templates for visualizations and dashboards

Few Other Important Enhancements –

  • XSLT 3.0 Support
    • Out of box wiring to work with Saxon XSLT3.0
  • SQL Availability Group support for BAM DTS Package via SSIS Catalog
    • Helpful for HA of BAM DBs
  • Partially disabled receive locations
    • Allows Fault Tolerance
  • Throughput improvements for Dynamic Send Ports with Ordered Delivery
  • Support for SSO Affiliate applications in SFTP adapter

Deprecated Features

ProgramStatusReplacement
POP3 and SMTP adaptersDeprecatedOffice 365 adapters
Support for ACS authentication in adaptersRemovedSAS authentication
SOAP adapterDeprecatedWCF-BasicHttp Adapter
Old SQL adapterRemovedWCF-SQL Adapter
BPEL supportDeprecatedNone
JDE OneWorld adapterDeprecatedNone
OWC redistRemovedNone
BAM PortalDeprecatedNone
WCF-NetTcpRelay adapterDeprecatedNone
SamplesRemovedRemoved from BizTalk Server installation.

Hope this article helps you in you BizTalk Migration Planning.

Feel free to contact me in case of any query. Thanks.

Contact Me:- 

@Gmail@Facebook , @Twitter, @LinkedIn @MSDNTechnet, @My Personal Blog 

Related Blog – BizTalk Migration Tool Demo

BizTalk Automated migration Tool | Prashant BizTalk And Azure Integration Blogs (wordpress.com)

Contact Me –

Check out my other blogs –

BizTalk Automated migration Tool

Watch the Video – BizTalk Automated Migration Tool Demo – Microsoft has recently released BizTalk 2020 with lots of new features and enhancements. There are many reasons to Migrate and can be broadly summarized as below – Avail Microsoft SupportBizTalk 2016 Mainstream Support Ends – 1st November 2022All other previous versions like 2013R2, 2013 or…

PROGRAMMATICALLY Change service account’s password for Windows Services, iis app pools and scheduled tasks – C#

Nowadays it’s becoming a very common practice to frequently (half-yearly or quarterly) change password for all AD Service Accounts. It’s done as part of password compliance security guidelines, specially post GDPR implementations. Earlier for BizTalk environments we use to have “Never Expire Password” for service account and used it to configure BizTalk Services (like Host…