This is 5 part blog series, on how to “Cost Optimization for On-demand Private Build Server for SharePoint”. Although I specifically write this for SharePoint’s build server, however the concept and ideas is also applicable for anyone looking for optimizing their cost running private CI/CD (Continuous Integration/Continuous Delivery) in Visual Studio online. The series in these articles are:
- Part 1, Introduction to Visual Studio Online Build and Release Agent
- Part 2, Cost Analysis and Optimizing Cost
- Part 3, Step-by-Step Creation Private Build Server for SharePoint
- Part 4, Azure Automation Service
- Part 5, Integrates Azure Automation Service in Build Pipeline
In Part 4
In Part 4, we are able to schedule the private build server start-up and shutdown time. Azure Automation Service helps to run Runbooks and manage the schedule for the runbook job. We have reduced the VM uptime to 50% by starts it at 07:00 AM and shut it down at 07:00 PM daily. This strategy leaves non-office hours work uncovered by private build server services.
Integrates Azure Automation Service in Build Pipeline
The private build server must be always available, but we must keep the price at a fraction. In order to do that, we have to start and shutdown the server on-demand. The server must be started before the build/release task starts, and must be shut-down just after the build/release task ends.
Introducing Azure Automation Webhook
A webhook allows you to start a particular runbook in Azure Automation through a single HTTP request. It opens new possibilities that external services such as Visual Studio Team Services, GitHub etc, to orchestrate different service automation by posting to the HTTP endpoint of the webhook. More information about Azure Automation webhook can be found here: https://docs.microsoft.com/en-us/azure/automation/automation-webhooks
In order to create webhook, you just open the runbook and click on webhook. Following are the continuation step from “Start-MyAzureVM” runbooks earlier in Part-2.
- Create Webhook
- Open “Start-MyAzureVM” runbook. In Overview page, click Webhook
- In Add Webhook , click on “Create new webhook” and specify Name, and expiration time.
Important! Don’t forget to copy the webhook URL, and save it somewhere before pressing OK.
- In Configure parameters and run settings, you specify the Resource Name and VM name, to point to Resource Name and VM name of the private build server. (Same as the Schedule Job step)
- Click create.
- Create new runbook by importing from “Stop Azure V2 VMs” in the gallery. Also create new webhook to stop the VM.
- You can check the webhook, created for specific Runbooks by checking on its Resources -> Webhooks
Introducing Agentless Phase in Build/Release pipeline
Agentless phase is a collection of steps that run without consuming hosted agent minutes time – it will run on the server (I believe the VSTS server) instead of running in the agent. This phase can run even though all build/release agents are offline. For more details, please refers here : https://docs.microsoft.com/en-us/vsts/build-release/concepts/process/phases
Hence, go ahead to VSTS and update your build/release definition to add “Invoke REST API” task in Agentless phase. I will leave the detail steps for your final exploration, and I will just provide the final screenshot in the build/release definition.
It’s been quite sometimes from the beginning of the first post. Here in the final post, I leave the final piece without step-by-step since many documentations have covered it.
We are able to create private build server with same behaviour as if it always online, the same behaviour as if we subscribe to additional hosted pipelines. Using Azure Automation, we are able to avail the private build server on-demand making it offline most of the time and online when build or release are needed. Reducing build VM uptime in a month, will reduce the subscription cost of the VM.