Quick Guide to Kubernetes Workload Identity on AKS

IMPORTANT: the steps below are not relevant anymore; the steps in the quick guide have been updated; see https://github.com/gbaeke/quick-guides/blob/main/workload-identity/README.md for the correct steps.

Some things that have changed:

  • you can now use managed identities instead of app registrations; federated token configuration is at the managed identity level
  • you do not need to install the webhook
  • the azwi CLI is not required

I recently had to do a demo about Workload Identity on AKS and threw together some commands to enable and verify the setup. It contains bits and pieces from the documentation plus some extras. I wrote another post some time ago with more background.

All commands are for bash and should be run sequentially in the same shell to re-use the variables.

Step 1: Enable OIDC issuer on AKS

You need an existing AKS cluster for this. You can quickly deploy one from the portal. Note that workload identity is not exclusive to AKS.

CLUSTER=<AKS_CLUSTER_NAME>
RG=<AKS_CLUSTER_RESOURCE_GROUP>

az aks update -n $CLUSTER -g $RG --enable-oidc-issuer

After enabling OIDC, retrieve the issuer URL with ISSUER_URL=$(az aks show -n $CLUSTER -g $RG --query oidcIssuerProfile.issuerUrl -o tsv). To check, run echo $ISSUER_URL. It contains a URL like https://oidc.prod-aks.azure.com/GUID/. You can issue the command below to obtain the OpenID configuration. It will list other URLs that can be used to retrieve keys that allow Azure AD to verify tokens it receives from Kubernetes.

curl $ISSUER_URL/.well-known/openid-configuration

Step 2: Install the webhook on AKS

Use the Helm chart to install the webhook. First, save the Azure AD tenant Id to a variable. The tenantId will be retrieved with the Azure CLI so make sure you are properly logged in. You also need Helm installed and a working Kube config for your cluster.

AZURE_TENANT_ID=$(az account show --query tenantId -o tsv)
 
helm repo add azure-workload-identity https://azure.github.io/azure-workload-identity/charts
 
helm repo update
 
helm install workload-identity-webhook azure-workload-identity/workload-identity-webhook \
   --namespace azure-workload-identity-system \
   --create-namespace \
   --set azureTenantID="${AZURE_TENANT_ID}"

Step 3: Create an Azure AD application

Although you can create the application directly in the portal or with the Azure CLI, workload identity has a CLI to make the whole process less error-prone and easier to script. Install azwi with brew: brew install Azure/azure-workload-identity/azwi.

Run the following commands. First, we save the application name in a variable. Use any name you like.

APPLICATION_NAME=WorkloadDemo
azwi serviceaccount create phase app --aad-application-name $APPLICATION_NAME

You can now check the application registrations in Azure AD. In my case, WorkloadDemo was created.

App registration in Azure AD

If you want to grant this application access rights to resources in Azure, first grab the appId:

APPLICATION_CLIENT_ID="$(az ad sp list --display-name $APPLICATION_NAME --query '[0].appId' -otsv)"

Now you can use commands such as az role assignment create to grant access rights. For example, here is how to grant the Reader role to your current Azure CLI subscription:

SUBSCRIPTION_ID=$(az account show --query id -o tsv)

az role assignment create --assignee-object-id $APPLICATION_CLIENT_ID --role "Reader" --scope /subscriptions/$SUBSCRIPTION_ID

Step 4: Create a Kubernetes service account

Although you can create the service account with kubectl or via a YAML manifest, azwi can help here as well:

SERVICE_ACCOUNT_NAME=sademo
SERVICE_ACCOUNT_NAMESPACE=default

azwi serviceaccount create phase sa \
  --aad-application-name "$APPLICATION_NAME" \
  --service-account-namespace "$SERVICE_ACCOUNT_NAMESPACE" \
  --service-account-name "$SERVICE_ACCOUNT_NAME"

This creates a service account that looks like the below YAML manifest:

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: <value of APPLICATION_CLIENT_ID>
  labels:
    azure.workload.identity/use: "true"
  name: sademo
  namespace: default

This is a regular Kubernetes service account. Later, you will configure your pod to use the service account.

The label is important because the webhook we installed earlier acts on service accounts with this label to perform all the behind-the-scenes magic! 😉

Note that workload identity does not use the Kubernetes service account token. That token is used to authenticate to the Kubernetes API server. The webhook will ensure that there is another token, its path is in $AZURE_FEDERATED_TOKEN_FILE, which is the token sent to Azure AD.

Step 5: Configure the Azure AD app for token federation

The application created in step 5 needs to be configured to trust specific tokens issued by your Kubernetes cluster. When AAD receives such a token, it returns an Azure AD token that your application in Kubernetes can use to authenticate to Azure.

Although you can manually configure the Azure AD app, azwi can be used here as well:

SERVICE_ACCOUNT_NAMESPACE=default

azwi serviceaccount create phase federated-identity \
  --aad-application-name "$APPLICATION_NAME" \
  --service-account-namespace "$SERVICE_ACCOUNT_NAMESPACE" \
  --service-account-name "$SERVICE_ACCOUNT_NAME" \
  --service-account-issuer-url "$ISSUER_URL"

In the AAD app, you will see:

Azure AD app federated credentials config

You find the above by clicking Certificates & Secrets and then Federated credentials.

Step 6: Deploy a workload

Run the following command to create a deployment and apply it in one step:

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: azcli-deployment
  namespace: default
  labels:
    app: azcli
spec:
  replicas: 1
  selector:
    matchLabels:
      app: azcli
  template:
    metadata:
      labels:
        app: azcli
    spec:
      serviceAccount: sademo
      containers:
        - name: azcli
          image: mcr.microsoft.com/azure-cli:latest
          command:
            - "/bin/bash"
            - "-c"
            - "sleep infinity"
EOF

This runs the latest version of the Azure CLI in Kubernetes.

Grab the first pod name (there is only one) and exec into the pod’s container:

POD_NAME=$(kubectl get pods -l "app=azcli" -o jsonpath="{.items[0].metadata.name}")

kubectl exec -it $POD_NAME -- bash

Step 7: Test the setup

In the container, issue the following commands:

echo $AZURE_CLIENT_ID
echo $AZURE_TENANT_ID
echo $AZURE_FEDERATED_TOKEN_FILE
cat $AZURE_FEDERATED_TOKEN_FILE
echo $AZURE_AUTHORITY_HOST

# list the standard Kubernetes service account secrets
cd /var/run/secrets/kubernetes.io/serviceaccount
ls 

# check the folder containing the AZURE_FEDERATED_TOKEN_FILE
cd /var/run/secrets/azure/tokens
ls

# you can use the AZURE_FEDERATED_TOKEN_FILE with the Azure CLI
# together with $AZURE_CLIENT_ID and $AZURE_TENANT_ID
# a password is not required since we are doing federated token exchange

az login --federated-token "$(cat $AZURE_FEDERATED_TOKEN_FILE)" \
--service-principal -u $AZURE_CLIENT_ID -t $AZURE_TENANT_ID

# list resource groups
az group list

If the last command works, that means you successfully logged on with workload identity ok AKS. You can list resource groups because you granted the Azure AD app the Reader role on your subscription.

Note that the option to use token federation was added to Azure CLI quite recently. At the time of this writing, May 2022, the image mcr.microsoft.com/azure-cli:latest surely has that capability.

Conclusion

I hope the above commands are useful if you want to quickly test or demo Kubernetes workload identity on AKS. If you spot errors, be sure to let me know!

Azure API Management Consumption Tier

In the previous post, I talked about a personal application I use to deploy Azure resources to my lab subscription. The architecture is pretty straightforward:

After obtaining an id token from Azure Active directory (v1 endpoint), API calls go to API Management with the token in the authorization HTTP header.

API Management is available in several tiers:

API Management tiers

The consumption tier, with its 1.000.000 free calls per month per Azure subscription naturally is the best fit for this application. I do not need virtual network support or multi-region support or even Active Directory support. And I don’t want the invoice either! 😉 Note that the lack of Active Directory support has nothing to do with the ability to verify the validity of a JWT (JSON Web Token).

I created an instance in West Europe but it gave me errors while adding operations (like POSTs or GETs). It complained about reaching the 1000 operations limit. Later, I created an instance in North Europe which had no issues.

Define a product

A product contains one or more APIs and has some configuration such as quotas. You can read up on API products here. You can also add policies at the product level. One example of a policy is a JWT check, which is exactly what I needed. Another example is adding basic authentication to the outgoing call:

Policies at the product level

The first policy, authentication, configures basic authentication and gets the password from the BasicAuthPassword named value:

Named values in API Management

The second policy is the JWT check. Here it is in full:

JWT Policy

The policy checks the validity of the JWT and returns a 401 error if invalid. The openid-config url points to a document that contains useful information to validate the JWT, including a pointer to the public keys that can be used to verify the JWT’s signature (https://login.microsoftonline.com/common/discovery/keys). Note that I also check for the name claim to match mine.

Note that Active Directory is also configured to only issue a token to me. This is done via Enterprise Applications in https://aad.portal.azure.com.

Creating the API

With this out of the way, let’s take a look at the API itself:

Azure Deploy API and its defined operations

The operations are not very RESTful but they do the trick since they are an exact match with the webhookd server’s endpoints.

To not end up with CORS errors, All Operations has a CORS policy defined:

CORS policy at the All operations level

Great! The front-end can now authenticate to Azure AD and call the API exposed by API management. Each call has the Azure AD token (a JWT) in the authorization header so API Management van verify the token’s validity and pass along the request to webhookd.

With the addition of the consumption tier, it makes sense to use API Management in many more cases. And not just for smaller apps like this one!

Simple Azure AD Authentication in a single page application (SPA)

Adding Azure AD integration to a website is often confusing if you are just getting started. Let’s face it, not everybody has the opportunity to dig deep into such topics. For https://deploy.baeke.info, I wanted to enable Azure AD authentication so that only a select group of users in our AD tenant can call the back-end webhooks exposed by webhookd. The architecture of the application looks like this:

Client to webhook

The process is as follows:

  • Load the client from https://deploy.baeke.info
  • Client obtains a token from Azure Active Directory; the user will have to authenticate; in our case that means that a second factor needs to be provided as well
  • When the user performs an action that invokes a webhook, the call is sent to API Management
  • API Management verifies the token and passes the request to webhookd over https with basic authentication
  • The response is received by API Management which passes it unmodified to the client

I know you are an observing reader that is probably thinking: “why not present the token to webhookd?”. That’s possible but then I did not have a reason to use API Management! 😉

Before we begin you might want to get some background information about what we are going to do. Take a look at this excellent Youtube video that explains topics such a OAuth 2.0 and OpenID Connect in an easy to understand format:

Create an application in Azure AD

The first step is to create a new application registration. You can do this from https://aad.portal.azure.com. In Azure Active Directory, select App registrations or use the new App registrations (Preview) experience.

For single page applications (SPAs), the application type should be Web app / API. As the App ID URI and Home page URL, I used https://deploy.baeke.info.

In my app, a user will authenticate to Azure AD with a Login button. Clicking that button brings the user to a Microsoft hosted page that asks for credentials:

Providing user credentials

Naturally, this implies that the authentication process, when finished, needs to find its way back to the application. In that process, it will also bring along the obtained authentication token. To configure this, specify the Reply URLs. If you also develop on your local machine, include the local URL of the app as well:

Reply URLs of the registered app

For a SPA, you need to set an additional option in the application manifest (via the Manifest button):

"oauth2AllowImplicitFlow": true

This implicit flow is well explained in the above video and also here.

This is basically all you have to do for this specific application. In other cases, you might want to grant access from this application to other applications such as an API. Take a look at this post for more information about calling the Graph API or your own API.

We will just present the token obtained by the client to API Management. In turn, API Management will verify the token. If it does not pass the verification steps, a 401 error will be returned. We will look at API Management in a later post.

A bit of client code

Just browse to https://deploy.baeke.info and view the source. Authentication is performed with ADAL for Javascript. ADAL stands for the Active Directory Authentication Library. The library is loaded with from the CDN.

This is a simple Vue application so we have a Vue instance for data and methods. In that Vue instance data, authContext is setup via a call to new AuthenticationContext. The clientId is the Application ID of the registered app we created above:

authContext: new AuthenticationContext({ 
clientId: '1fc9093e-8a95-44f8-b524-45c5d460b0d8',
postLogoutRedirectUri: window.location
})

To authenticate, the Login button’s click handler calls authContext.login(). The login method uses a redirect. It is also possible to use a pop-up window by setting popUp: true in the object passed to new AuthenticationContext() above. Personally, I do not like that approach though.

In the created lifecycle hook of the Vue instance, there is some code that handles the callback. When not in the callback, getCachedUser() is used to check if the user is logged in. If she is, the token is obtained via acquireToken() and stored in the token variable of the Vue instance. The acquireToken() method allows the application to obtain tokens silently without prompting the user again. The first parameter of acquireToken is the same application ID of the registered app.

Note that the token (an ID token) is not encrypted. You can paste the token in https://jwt.ms and look inside. Here’s an example (click to navigate):

Calling the back-end API

In this application, the calls go to API Management. Here is an example of a call with axios:

axios.post('https://geba.azure-api.net/rg/create?rg='                             + this.createrg.rg , null, this.getAxiosConfig(this.token)) 
.then(function(result) {
console.log("Got response...")
self.response = result.data;
})
.catch(function(error) {
console.log("Error calling webhook: " + error)
})
...

The third parameter is a call to getAxiosConfig that passes the token. getAxiosConfig uses the token to create the Authorization header:

getAxiosConfig: function(token) { 
const config = {
headers: {
"authorization": "bearer " + token
}
}
return config
}

As discussed earlier, the call goes to API Management which will verify the token before allowing a call to webhookd.

Conclusion

With the source of https://deploy.baeke.info and this post, it should be fairly straightforward to enable Azure AD Authentication in a simple single page web application. Note that the code is kept as simple as possible and does not cover any edge cases. In a next post, we will take a look at API Management.

%d bloggers like this: