Overview
With ARM comes the ability to define your infrastructure as code, in a JSON template file. This means that your environment can effectively be source controlled and release managed, and can become a standard part of your application lifecycle, just as source code and applications are.
Put simply, you can enable continuous deployment of your dev/test environments, ensuring they’re in a consistent state, and controlled as part of your development sprints. Update the template, check the change into source control, a CI build is triggered automatically, which publishes the changed template scripts as an artifact.
Next, the Release Manager part of Visual Studio Team Services (VSTS) can be configured to automatically create a release based on the successful build/publish, and then execute the deployment of the template to any environments you setup.
Since ARM templates are idempotent, the Azure Resource Manager will only apply differences when make changes to the environment. This post will take you through the process of setting up the above configuration with a simple ARM template.
This post will take you through:
- Creating an ARM template
- Deploying the template via VS2015
- Setting up VSTS (Visual Studio Team Services) to add the project
- Setting up the VSTS Build definition
- Setting up the VSTS Release definition
- Verifying the continuous release process
- Next Steps
Create the template
Lets start with a simple template. I’m using VS2015 for this example, and I have the 2.9 Azure SDK installed.
Create a new project using the “Azure Resource Group” template
Choose the template “Web app + SQL”, click OK. This will give you a single project in your new solution that has a Scripts folder, containing “Deploy-AzureResourceGroup.ps1” and a Templates folder with two files in:
- WebSiteSQLDatabase.json – the template itself
- WebSiteSQLDatabase.parameters.json – saved parameters to use during deployment
When editing ARM templates, the JSON Outline view is really useful, it gives you a hierarchical view of the parameters (inputs), variables (calculated or defined values) and resources (the actual assets to be deployed) in your template. You can see straight away that although we just wanted a Website and a SQL Database, we actually get a SQL Server with a database and a configuration setting, a Hosting Plan, a Website with a connectionstrings resource, some AutoScaleSettings and some other AppInsights alerts. If there are some resources you don’t particularly want, you can highlight them in the outline view and press delete to remove them, the editor will remove the text from the JSON file for you. Conversely, if you right click on the resources node, there is an option to add resources in as well. Very handy.
We will leave things as is, but we’re going to end up with a website and sqlserver that have some strange names (eg. “webSitewhnsbsw3kskko”– this is because their values are based on the id of the resourcegroup we’re going to deploy to:
"variables": {
"webSiteName": "[concat('webSite', uniqueString(resourceGroup().id))]",
"sqlserverName": "[concat('sqlserver', uniqueString(resourceGroup().id))]"
},
This suits our requirement for now to be able to spin up multiple environments using the same template, but usually I’d change these variables to be based on a parameter to denote the environment, which will still guarantee us unique DNS names, but will be more readable and useful, for example:
"variables": {
"webSiteName": "[concat(parameters(‘environment’), '-webSite')]",
"sqlserverName": "[concat(parameters(‘environment’), '-sqlserver')]"
},
Next, run the deployment manually to verify it all works ok:
Right click the project node in the Solution Explorer, choose Deploy –> New Deployment…
In the next dialog, select your account from the dropdown (if you have more than one), and enter your credentials if it’s prompting you to. This will then query your account for available subscriptions. Choose the subscription you would like to deploy to, then from the Resource Group drop down, choose <Create New>, and give the group a name and location. (I’ve gone for “MyAppResourceGroup” based in North Europe. Click Create – this will actually provision the new resource group in Azure when you click Create.
The next step is to setup the parameters you’d like to use for this deployment. Click Edit Parameters. This will parse the WebSIteSQLDatabase.parameters.json file and prompt you for values for each of the parameters. Again fill out the parameters.
NOTE: For passwords, you have the option of saving them as clear text in the parameters file (not recommended), or using the Azure Key Vault to store them in (assuming you have one setup). You must either use Key Vault, or tick the checkbox to save “Save passwords as plain text in the parameters file” – otherwise the deployment can only run interactively, and it will prompt for a password during the deployment.
Clicking Save will write these values to the .parameters.json file. You can then click “Deploy” and watch the output window to see Visual Studio kick off the powershell command to deploy the template to your subscription.
If all goes well, you should see this:
Successfully deployed template 'd:\my documents\visual studio 2015\projects\myappresourcegroup\myappresourcegroup\templates\websitesqldatabase.json' to resource group 'MyAppResourceGroup'.
You can then go to your Azure portal and look at your new resource group and resources:
Now we’re ready to start the CD piece.
Setup VSTS Project for Source Control
I’m going to go through the full steps here assuming you don’t already have a VSTS project and repository setup – if you do it will save a fair amount of time and you can skip the create steps.
1. Login to your vsts account using a web browser. ( https://yourvstsacct.visualstudio.com)
2. In the overview window under Recent Projects & Teams, click “New” to create a new Team project, give it a name, process template and version control type (I’m going for TFVC for my example).
3. Click Create Project and wait while it provisions everything for you.
4. Once that’s done, switch back into VS2015, Right click the solution node in the Solution Explorer window, choose “Add solution to source control…”
5. Choose TFVC, click OK
6. Select the VSTS account you’ve just set the new project up in
7. Check the project you created in the right hand pane and click Connect.
You should now see “+” icons next to the files in your solution, indicating they’re ready to check-in.
8. In the Team Explorer window, click on Pending Changes, Review what’s included (should be 8 files), add a comment, then click Check-in. You can verify what’s been checked in by going to your VSTS project in a web browser, clicking on the Code menu, where you should see something like this:
The important part of our source are the two files – the template and the parameters files, nothing else is used for our continuous deployment process.
Setup VSTS Build
In this instance, the term Build is a bit of a misnomer. We don’t have any source code to build, but we do have some source and we want to produce an output. The output will be a zip file known as an artifact, that can be published to the release manager for deployment to our environments. So our build process is going to be very simple.
In the web browser still, click on the BUILD menu. The “All Build Definitions” under “Build Definitions” in the explorer tree will be highlighted, but there isn’t anything setup yet.
The first task is to create an empty build definition that knows where to look for the source files:
1. Click on the green + icon above the explorer to create a new build definition
2. Select “Empty” for the template type, click Next
3. For repository source, choose your Team Project, which should already be selected, and make sure the repository is the correct one.
4. Click the “Continuous integration” checkbox
5. Leave the default agent queue as Hosted – this means the build will be carried out by Visual Studio Team System in the cloud.
6. Click Create
Next we need to add a build step to publish the template files into an artifact, so they are visible to the release manager:
1. Click “Add build step…”
2. Click on “Utility” in the left hand menu, then click the Add button to the right of “Publish Build Artifacts”.
3. Click close
4. Click the “…” to the right of Path to Publish, and navigate to and select the Templates subfolder in your project
(See this MSDN article for more details on this dialog).
6. Give the artifact a name, I’m using “ARM Template”
7. For artifact type, select “Server”
8. Click the Save button, which will prompt you for a name for the Build definition – complete the name, click OK.
9. It’s a good idea to ensure only changes to the template part of our source control tree trigger a build – you can control this in the Triggers menu, change the Filter from Include –> $/<projectName> to the templates subfolder in your project, in my case that’s “$/MyApp/MyAppResourceGroup/MyAppResourceGroup/Templates” – you can use the “…” to browse the source tree and make this selection. Then click Save again to update the definition.
Now we’re ready and able to carry out a “build” to ensure our artifact is published. Although we’ve setup a CI type build which will be triggered by changes in the source tree, you can also trigger a build manually using the “Queue build…” button. Leave all the parameters as defaults, and watch your build progress. It will wait a little while for an agent to become free then you should see the output logs from the build in real time as it progresses.
Define the Release
Still in VSTS within your web browser, navigate to the “Release” tab. Like the build tab when we first arrived, there’s not a lot in here. Start by clicking the green + arrow again to create a new Release Definition.
Again, start with an empty definition template, and click OK – you will now see the edit page for your new definition, which has no name and just a default environment set up.
First, lets give the release definition a name – this release will be triggered each time a new build is available, and hence I’m calling this my “Environment CD release”.
Rename the Default Environment to “Dev” (by clicking on the name and typing). Now we need to add the tasks to execute to deploy the ARM template to our dev environment.
Before we add any tasks, we will tell the release what artifacts we’re going to be working with.
1. Click on the Artifacts menu, then “Link an artifact source”.
2. The dialog should pre-populate with type: Build, Project (your project name), and source (whatever you called the artifact from your build – in my case, CI ARM Publish) – if it doesn’t auto fill then select those values yourself then click the Link button.
Now we’re going to make this a continuous deployment, so we want it to be triggered whenever a new artifact is built.
1. Click on the Triggers tab.
2. Choose “Continuous deployment”
3. Select “CI ARM Publish” as the trigger/artifact source
Now to setup the actual deployment task.
1. Click on the Environments tab – note we’re now in the context of the “dev” environment – its the only one we’ve built so far.
2. Click the Add tasks button
3. Click the Add button next to “Azure Resource Group Deployment”
4. Click close
In the right hand pane now, you should see properties relating to the task we’ve added, for the environment selected.
To explain the options:
The connection type is which API to use to carry out the deployment – we’re using ARM, which enables us to use templates and do what we’re trying to do – if you choose Classic then you wont be able to deploy using an ARM template…
The Azure RM Subscription is referencing a service endpoint that we need to configure in VSTS to enable the release agent to connect to our Azure subscription. If you haven’t set one of these up already, click on the Manage link, then click the green Add to create a new Service Endpoint, and select Azure Resource Manager as the type. You will then need to provide some details relating to the Service Principal. To set that up, please see my last blog post Using Azure Resource Manager with Azure Automation and Service Principal – although the post is targeting Automation, the same process applies for creating the service principal, and you’ll end up with information similar to below as output from the script, which matches what you’ll be asked for when creating the VSTS service endpoint connection:
*************************************************************************** Connection Name: MSDN(SPN) Subscription Id: 12c37ade-****-****-****-******77747 Subscription Name: MSDN Service Principal Id: e0c6****-****-****-****-****b5234f9d Service Principal key: myp***ord Tenant Id: 4a3c53d8-****-****-***-****09be4420 ***************************************************************************
Action defines what we want the task to do for us, in this instance, create or update a resource group. If this is the first deployment, it will create the resource group, otherwise it will look to update it (if anything has changed in the template).
The resource group and location defines the destination resource group to deploy to for this environment.
For the template and template parameters, use the “…” to browse the artifact package to locate the two json files we need, these will then become the input to the Azure Resource Manager deploy command.
You can then use override template parameters to override any of the settings defined as “parameters” in your template.json file – this feature is really handy, and we’ll come back to it later.
Once that dialog’s complete, there are a couple more steps to complete to enable the Continuous Release trigger. Click on the “…” inside your environment definition box.
And select “Deployment conditions” from the menu. In here, select the trigger to be “After release creation”, then click OK. (That will add the small lightning symbol on the environment as shown in my screen clip). The icon of the person with 0 next to it indicates there are no approvers for this release – ie. it will automatically be approved and will deploy.
Now Save the release definition and it’s ready to run.
Verify
Next to the title for the release definition (“Definition: Environment CD Release”, you should see a vertical bar and then a link to “Releases” – use this to swap back to the view of Releases from the edit view.
1. Click the green + button to create a new Release, select “Create Release” from the dropdown menu.
2. Under artifacts, you should see your artifact name (CI ARM Publish in my case), and a dropdown box. This dropdown will have a list of all the successful builds that have been created for that artifact. Select the latest (highest number, first in the list).
3. Then for your environment (Dev), ensure “Automated deployment: After release creation” is selected. If this isn’t an option and VSTS tells you you can’t because the environment’s not configured, you need to go back to the “Deployment conditions” menu in the last section.
4. Click Create
You should now see Release-1 appear in the list of releases with a blue bar under “Environments”. Blue means the release is in progress, green means it’s succeeded and red means it’s failed.
Double click on the Release-1 row to see more information. If you choose the “Logs” view on the far right, you will see the console output from the release agent as it’s executing the deployment.
All being well, you should end up with a summary like this:
And a quick look in the Azure portal will confirm your resource group is there with the required resources, the last deployment date should be today, and if you click on that, you’ll see the deployment time, and if you click on that, you’ll see more detail about the operations that were carried out.
Next Steps
Generally it’s advisable to use continuous deployment for dev/test environments, and use a more controlled mechanism for deploying to your production environment. However, there’s nothing to stop you adding governance and approvals into the same Release configuration, such that the template will only be deployed to production when the approvals chain have accepted the release. This will ensure the same template will be applied to production as has been applied to dev/test.
This process also lends itself nicely to allowing team members to spin up new environments at will, for example if carrying out performance tests or specific scenario testing that’s short lived. It’s therefore a good idea to create a single Release which combines the ARM environment artifact and deployment, then also carries out a full deployment of all the code artifacts required to setup the whole application, and carry out any seeding of databases, etc. This may sound obvious, but sometimes full systems comprise of many web APIs, web apps and other processes which may usually be built and released independently (each subsystem with its own Release configuration). If you don’t have a single release configured that combines the whole system, then carrying out the full release will mean adding your temporary environment to every component’s release configuration – which could be quite time consuming – and when you want to destroy the environment, you need to undo all those changes. Much easier to create a release with a parameter to define which resource groups your deployment will drop into.
In the next blog post on this subject, I’ll go through the steps to add another CD environment with approvals (so the test manager has to agree to the test environment being deployed to), and how to use the parameter override to customise the deployments.
Pingback: Continuous Deployment of an ASP.net Core app to Azure using VSTS – russellyoung.net()