Pulumi and Azure DevOps

For those who are not yet familiar with Pulumi, you can read the previously published article, Infrastructure as Code (IaC) with Pulumi: A Hands-On Guide.
In that article, we delve into what Pulumi is, its advantages compared to other Infrastructures as Code (IaC) tools, and how to get started. 

Azure DevOps is a suite of development tools provided by Microsoft that encompasses the entire application lifecycle. For the context we will cover, two main components will be used: 

– Azure Repos: A version control system that allows teams to store and manage their source code. 
– Azure Pipelines: A continuous integration and continuous deployment (CI/CD) tool that enables automation of the building, testing, and deployment of applications on any platform or language. 

This article will detail integrating both tools to leverage their benefits and improve your infrastructure deployment workflow. Some advantages of this integration are: 

– CI/CD Automation: Using Azure Pipelines, you can automate the continuous integration and deployment of your infrastructure defined with Pulumi. This ensures that every change in the infrastructure code goes through a consistent validation and deployment process, reducing the risk of errors and improving efficiency. 
– Infrastructure Lifecycle Management: With Azure Repos, you can version your infrastructure code just like you version your application code. This allows you to keep a detailed record of changes, revert to previous versions if necessary, and collaborate more effectively with other developers. 
– Improved Collaboration: Azure DevOps offers a unified platform that integrates development tools, version control, CI/CD, and project management. By combining this with Pulumi, development and operations teams can work together more efficiently, using a common and familiar set of tools. 

Now that we know both tools, let’s see how we can start working with them. The first thing to do is to have the Pulumi code in Azure Repos, so that it can later be used in Azure Pipelines. Once we have the code in the Azure repository, we can start creating the pipeline that will continuously deploy the infrastructure.

Prerequisites: 

Pulumi extension for Azure 
In Azure DevOps, we will install the Pulumi Azure Pipelines Task extension, which will allow us to use Pulumi activities easily in our pipelines. 

Azure Storage to Keep the State 

Pulumi needs a state to be able to control the changes that the architecture undergoes and thus update the resources appropriately. For the Azure pipeline to access this state, we will need to store it in a cloud resource. The most suitable option when working with Azure is to use an Azure Storage Blob, which we will need to create beforehand. 

Azure Key Vault 
When working with Pulumi, it is often necessary to use some keys or secrets as configuration variables, and the most suitable option is to use Azure Key Vault to store them. Therefore, an Azure Key Vault will also be created beforehand with the necessary secrets for the Pulumi project. 

Azure Pipelines Library 
To define the configuration variables needed for our Pulumi project, the most appropriate approach is to create Azure DevOps libraries. For the secrets, another library can be connected to the previously created Azure Key Vault, allowing the Azure pipeline to access these keys. One of the advantages of using Azure DevOps libraries is that we can create multiple libraries to manage the variables for each stack. 

Once these resources are created, you can proceed to create the pipeline that will deploy the infrastructure defined in our Pulumi project. 

In our pipeline, it will be necessary to include two tasks: 

The first task will be optional depending on the language used to create the Pulumi project. In the example shown, pnpm is installed to download all the dependency packages needed for our Pulumi project (this is necessary because we are using TypeScript for this Pulumi example). For this case, the file with the dependencies is located inside the infrastructure folder. 

The second task, from the extension installed earlier, will be responsible for deploying our Pulumi-defined architecture. Some of the parameters needed to configure it are:

– cwd: Directory where the Pulumi code is located. In the example, it is inside a folder named infrastructure. 
– createStack: Flag indicating whether to create a stack if the chosen stack does not exist.
– args: Here you will include the necessary arguments for the Pulumi script. Firstly, you should include the –yes parameter to accept all conditions, avoiding the script’s interactive execution. After this, you should include the configuration variables used in the Pulumi project following the pattern -c <project_name>:<configuration_variable_name>=<value>. In our case, the project name is pulumi-training, and the values for these configuration variables are stored in an Azure Library as explained. 
– azureSubscription: Name of the service connection used for the deployment. 
– command: Pulumi operation that we want to execute, in this case, up. 
– loginArgs: Since we have the state stored in Azure Storage, the address to log into it is needed. 
– stack: Name of the stack to be used in the deployment. 

As Azure Storage is being used to store the state, it will be necessary to include three environment variables in the task using the env parameter: 

– AZURE_STORAGE_ACCOUNT: Name of the storage account used. 
– AZURE_STORAGE_KEY: Key of the storage account. 
– PULUMI_CONFIG_PASSPHRASE: Key used to encrypt the configuration variables in the state. 

With these two tasks included in our pipeline, we will be able to automatically deploy our architecture using Azure DevOps by simply running the pipeline. 

Wrapping up 

Integrating Pulumi with Azure DevOps provides a powerful and efficient way to manage your infrastructure as code. By leveraging Azure Repos for version control and Azure Pipelines for continuous integration and continuous deployment, you can automate the deployment of your infrastructure, ensuring consistency and reducing the risk of errors. Using Azure Storage for state management and Azure Key Vault for securely storing configuration secrets further enhances the robustness and security of your deployment process. 

With the detailed steps outlined above, you can set up a pipeline that automates the deployment of your Pulumi-defined architecture. This integration not only streamlines the deployment workflow but also fosters better collaboration between development and operations teams by utilizing a unified platform. By following these best practices, you can ensure a more reliable, efficient, and secure infrastructure management process.