In a previous blog post, I discussed Workload Identity Federation in AKS, the successor to the Azure Pod Identity solutions and a more elegant solution for providing keyless identities to systems.
This solution has now been added to Azure DevOps, simplifying service connection management. How does it simplify? Well, it removes the need to manage their secrets: fewer problems, fewer tickets, and less downtime.
Another great thing this solution brings to us is eliminating the secret key extraction technique from the pipeline logs. In the past and even now, you can extract the service connection credentials with a simple double convert to base 64, which wipes the history that it was a secret, and you can exfil the secret directly from the pipeline logs. With this solution, guess what? There's no secret to exfil; the most you can get out of it is the access token, which has a time limit, and it's scoped on that particular service connection, so you cannot use it with the connection's identity. Win-win!
How does it work?
In short, Workload identity federation is a method implemented through OpenID Connect for Azure DevOps. It enables authentication to Azure using temporary, credentialless access without assigning managed identities to self-hosted agents. This involves establishing trust between your Azure DevOps organization and a service principal. Azure DevOps then issues a token that can be used for authentication with the Azure API.
Creating and converting service connections
Creating a new service connection using Workload Identity is as simple as creating a regular service principal. When you make the service connection, you can choose a workload identity federation.
The recommended method is automatic and end-to-end if you have the permission to do it. Otherwise, you should check the manual process and ask your IT admin for the details, or you can tell your IT Admin to create and complete the service connection.
Now if you go the manual route, you have to fill in some steps:
Short explanation of the fields:
Issuer URL:
- Description: A unique URL used to identify your Azure DevOps organization in the format
https://vstoken.dev.azure.com/<organisation-id>. Theorganisation-idis the GUID (Globally Unique Identifier) of your Azure DevOps organization. - Example: For an organization with the GUID
f66a4bc2-08ad-4ec0-a25e-e769dab3b294, the Issuer URL would behttps://vstoken.dev.azure.com/f66a4bc2-08ad-4ec0-a25e-e769dab3b294.
Subject Identifier:
- Description: A structured identifier mapping to your service connection, formatted as
sc://<organisation-name>/<project-name>/<service-connection-name>. Here,organisation-nameis the name of your Azure DevOps organization,project-nameis the name of your Azure DevOps project, andservice-connection-nameis the name of your specific service connection. - Example: For a service connection within the organization named "my-organization", in a project called "my-project", the subject identifier would be
sc://my-organisation/my-project/my-service-connection.
Audience:
- Description: This field is consistently set to
api://AzureADTokenExchangeto denote the intended recipient or the audience of the token.
The existing service connections can be converted to Workload Identity, so there is no need to recreate them and break your pipelines. This option still requires administrative rights in the Azure subscription; however, it's a simple click-click-next procedure if you have them.
After you press on the convert button, you will be presented with a modal that will tell you that you can revert back the process in case you have issues, and you will but that's for the caveats section.
Pressing the convert button will start a conversion process which usually takes 1 minute and will look like this once it done.
The revert conversion link will do as it says and go back to using a Client ID and Secret Key for the pipelines to work.
Caveats and current kinks
he first caveat is that the access token expires after about an hour, with failures around the 35-minute mark, so your mileage may vary. I know that Microsoft is working on a fix or potential solution to this problem, but if you have long-running jobs that require that token, consider that.
The second problem is with Terraform; now, if you use the Terraform tasks from Azure DevOps, then you don't have a problem per se, but if you're doing it custom, like I do it and a lot of people that don't want breaking changes to occur during the night because somebody figured to modify something. You must create a pseudo-PowerShell / AzCLI script that sets up the variables for Terraform to use OIDC.
Below is the PowerShell script and an example of a job from an Azure DevOps pipeline that shows how to use it.
#!/usr/bin/env pwsh
if ($env:SYSTEM_DEBUG -eq 'true')
{
$InformationPreference = 'Continue'
$VerbosePreference = 'Continue'
$DebugPreference = 'Continue'
Get-ChildItem -Path Env: -Force -Recurse -Include * |
Sort-Object -Property Name |
Format-Table -AutoSize |
Out-String
}
# Propagate Azure context to Terraform
az account show 2>$null |
ConvertFrom-Json |
Set-Variable -Name account
if (!$account)
{
throw 'Not logged into Azure CLI, no context to propagate as ARM_* environment variables'
}
if (![guid]::TryParse($account.user.name, [ref][guid]::Empty))
{
throw 'Azure CLI logged in with a User Principal instead of a Service Principal'
}
$env:ARM_CLIENT_ID ??= $account.user.name
$env:ARM_CLIENT_SECRET ??= $env:servicePrincipalKey # requires addSpnToEnvironment: true
$env:ARM_OIDC_TOKEN ??= $env:idToken # requires addSpnToEnvironment: true
$env:ARM_SUBSCRIPTION_ID ??= $account.id
$env:ARM_TENANT_ID ??= $account.tenantId
$env:ARM_USE_CLI ??= (!($env:idToken -or $env:servicePrincipalKey)).ToString().ToLower()
$env:ARM_USE_OIDC ??= ($null -ne $env:idToken).ToString().ToLower()
if ($env:ARM_CLIENT_SECRET)
{
Write-Verbose -Message 'Using ARM_CLIENT_SECRET'
}
elseif ($env:ARM_OIDC_TOKEN)
{
Write-Verbose -Message 'Using ARM_OIDC_TOKEN'
}
else
{
Write-Warning -Message 'No credentials found to propagate as ARM_* environment variables. Using ARM_USE_CLI = true.'
}
Write-Output -Object 'Terraform azure provider environment variables:'
Get-ChildItem -Path Env: -Recurse -Include ARM_* |
ForEach-Object -Process {
if ($_.Name -match 'SECRET|TOKEN')
{
$_.Value = '<redacted>'
}
$_
}|
Sort-Object -Property Name |
Format-Table -HideTableHeadersIf you're doing init, plan, and apply in separate jobs/steps, you need to reference the script every time; otherwise, it will fail with a red herring error that you cannot use user credentials in the TF plan.
jobs:
- deployment: deploy_${{ parameters.jobName }}
environment: ${{ parameters.environment }}
strategy:
runOnce:
deploy:
steps:
- checkout: self
- task: TerraformInstaller@1
displayName: Install Terraform $(terraformVersion)
inputs:
terraformVersion: ${{ parameters.terraformVersion }}
- task: AzureCLI@2
displayName: Run Terraform - Init
inputs:
azureSubscription: ${{ parameters.serviceConnection}}
scriptType: "pscore"
scriptLocation: "inlineScript"
inlineScript: |
set_terraform_azurerm_vars.ps1
useGlobalConfig: true
addSpnToEnvironment: true # This flag is required to set the idToken environment variable.
failOnStandardError: true
workingDirectory: "$(System.DefaultWorkingDirectory)/terraform/azIAC/layers/${{ parameters.applicationName}}"
env:
ARM_USE_AZUREAD: trueAs you can see, it's a great step in the right direction, but there are some kinks that need to be ironed out. I highly recommend using the feature, and I already converted all my service connections to workload identity. I hate managed secret keys, and I also like not having the possibility of exfiltrating the secret keys.
That being said, have a good one!
