Saturday, February 17, 2018

Zero downtime Azure App Service deployments with EF6 Code First Migrations and MVC5

This post is about how we deploy our production web sites to an Azure Web App Service and execute Entity Framework 6 code first migrations as part of a VSTS Release process.


A little bit of history:

What we were doing was running dbMigrator.Update() on the start of the website.

So in Startup.cs we had this:

var efConfiguration = new Configuration();

var dbMigrator = new System.Data.Entity.Migrations.DbMigrator(efConfiguration);

dbMigrator.Update();

This worked well, it upgraded the database, and applied any seed data. The downside was the site became unusable until all this had finished which for us was about 1 minute.

This also had a major downside when we turned on Azure auto scaling on the App Plan, when it scaled to more than 1 instance, all instances started up, and all of them called dbMigrator.Update(). Resulting in a fair number of exceptions and the site failing to start.


What we now do:

We needed to remove the database logic from Startup.cs and do it as part of the VSTS Release process instead. So we deploy the site to a staging slot, execute database migrations via Migrate.exe into the production database (on the build server), and then swap the staging slot to production.

This does mean the production web site code is running against a newer database schema until the site swaps over, but this is fine as long as the developers code to handle current and current-1 database versions.  This is how production now looks, sharing the same database.

clip_image002

How to implement this:

1. The build Process

As well as packaging up the website into its own Artifact we now package up as another Artifact all the files we need in order to run migrate.exe, so we have 2 extra build tasks as part of our Main branch build:

clip_image004

clip_image006

clip_image008

In the contents section:

line 1 : Copies our dll’s containing our entity framework migrations

line 2: Copies the DeployDatabase.ps1 PowerShell script below

line 3: Copies the migrate.exe provided by Entity Framework in the packages folder.

The DeployDatabase.ps1 PowerShell file contains:

#

# DeployDatabase.ps1

#

[CmdletBinding()]

Param(

[Parameter(Mandatory=$True,Position=1)]

[string]$webAppName,

[Parameter(Mandatory=$False,Position=2)]

[string]$slotName,

[Parameter(Mandatory=$False,Position=3)]

[string]$slotResourceGroup

)

cls

$ErrorActionPreference = "Stop" # Stop as soon as an error occurs

if($slotName -ne $null -and $slotName -ne '') {

$isSlot = $True

}

else {

$isSlot = $False

}

Write-Host "PSScriptRoot : " $PSScriptRoot

Write-host "Web app: " $webAppName

Write-Host "Using slot: " $isSlot " " $slotName

Write-Host "SlotResourceGroup: " $slotResourceGroup

$dll = "SiteDataAccess.Extended.Customer.dll"

Write-Host "Using dll: " $dll

if($isSlot) {

$GetWebSite = Get-AzureRmWebAppSlot -Name $webAppName -Slot $slotName -ResourceGroupName $slotResourceGroup

}

else {

$GetWebSite = Get-AzureRmWebApp -Name $webAppName

}

$Connection = $GetWebSite.SiteConfig.ConnectionStrings | Where {$_.name -eq "ExtendedSiteDBContext"}

$ConnectionString = $Connection.ConnectionString

Write-host "Executing Database migrations and seeding with Migrate.exe"

& "$PSScriptRoot\migrate.exe" $dll /connectionString=$ConnectionString /connectionProviderName="System.Data.SqlClient" /verbose

if ($LastExitCode -ne 0) {

throw 'migrate.exe returned a non-zero exit code...'

}

Write-host "Finished executing Database migrations and seeding with Migrate.exe"

Write-host "Finished"

What that script does when called from an Azure Powershell task in a Release definition is lookup the connectionString from the web app service in Azure and then uses that to call migrate.exe. We did this so we did not have to store any connectionStrings in VSTS. If the sql user in the connectionString used when executing migrate.exe needs different permissions to that of the website you could change this to use VSTS release variables instead.


2. The Release definition

When we release a site our process is:

· Stop the deployment slot ‘stage’

· Deploy the website zip to slot ‘stage’

· Update the database using the PowerShell script

· Start the Stage site

· Swap Stage with production

· Ping production site

· Stop the stage site (to save resources)


Our release definition looks like this:

Release definition

Task Groups

clip_image010

clip_image012

clip_image014

The Update Database task is just an Azure PowerShell task containing:

clip_image016

In Azure the App Service Plan is configured so that the production site and the stage slot are almost identical (baring a few appSetting values). They share the same connectionString and every appSetting and ConnectionString value has the ‘Slot Setting’ checkbox ticked.


Our big gotcha

Was during the swap slots task the website will be warm started automatically by that process hitting localhost under http, or it will try the domain name of the site but again under http. We had this in our filter.config

// Ensure all http connections are redirected to https

filters.Add(new RequireHttpsAttribute());


Which meant the warm start process instantly failed to start the site, the swap process continued and the site then started up from cold in production. We could see that because our ping task that pinged the production site, was taking a minute to respond.

What we had to do was create a custom version of the RequireHttpsattribute, once we did this our ping task responds in 1-3 seconds:

// Ensure all http connections are redirected to https

filters.Add(new CustomRequireHttpsAttribute());


And the code for the attribute is:

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple = false)]

public class CustomRequireHttpsAttribute : RequireHttpsAttribute

{

    protected override void HandleNonHttpsRequest(AuthorizationContext filterContext)

{

    string ipAddress = filterContext.RequestContext.HttpContext.Request.UserHostAddress;

    var userAgent = filterContext.RequestContext.HttpContext.Request.UserAgent.ToLower();

    // 0.0.0.0 and 127.0.0.1 and ::1 are used by the Azure App Service Swap Slot process to Warm up the site before swapping the slot to production

    if(ipAddress == "0.0.0.0" || ipAddress == "::1" || ipAddress == "127.0.0.1")

    //if(userAgent.Contains("sitewarmup")) // doesnt work

    {

        return;

    }

    base.HandleNonHttpsRequest(filterContext);

    }

}

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.