Sadly no post about good Belgian beer 🍺.
Draft 2 is an open-source project that aims 🎯 to make things easier for developers that build Kubernetes applications. It can improve the inner dev loop, where the developers code and test their apps, in the following ways:
- Automate the creation of a Dockerfile
- Automate the creation of Kubernetes manifests, Helm charts, or Kustomize configs
- Generate a GitHub Action workflow to build and deploy the application when you push changes
I have worked with Draft 1 in the past, and it worked quite well. Now Microsoft has integrated Draft 2 in the Azure CLI to make it part of the Kubernetes on Azure experience. A big difference with Draft 1 is that Draft 2 makes use of GitHub Actions (Wait? No Azure DevOps? 😲) to build and push your images to the development cluster. It uses GitHub OpenID Connect (OIDC) for Azure authentication.
That is quite a change and lots of bits and pieces that have to be just right. Make sure you know about Azure AD App Registrations, GitHub, GitHub Actions, Docker, etc… when the time comes to troubleshoot.
Let’s see what we can do? 👀
Prerequisites
At this point in time (June 2022), Draft for Azure Kubernetes Service is in preview. Draft itself can be found here: https://github.com/Azure/draft
The only thing you need to do is to install or upgrade the aks-preview extension:
|
1 |
az extension add --name aks-preview --upgrade |
Next, type az aks draft -h to check if the command is available. You should see the following options:
|
1 2 3 4 5 |
create generate-workflow setup-gh up update |
We will look at the first four commands in this post.
Running draft create
With az aks draft create, you can generate a Dockerfile for your app, Kubernetes manifests, Helm charts, and Kustomize configurations. You should fork the following repository and clone it to your machine: https://github.com/gbaeke/draft-super
After cloning it, cd into draft-super and run the following command (requires go version 1.16.4 or higher):
|
1 2 |
CGO_ENABLED=0 go build -installsuffix 'static' -o app cmd/app/* ./app |
The executable runs a web server on port 8080 by default. If that conflicts with another app on your system set the port with the port environment variable: run PORT=9999 ./app instead of just ./app. Now we know the app works, we need a Dockerfile to containerize it.
You will notice that there is no Dockerfile. Although you could create one manually, you can use draft for this. Draft will try to recognize your code and generate the Dockerfile. We will keep it simple and just create Kubernetes manifests. When you run draft without parameters, it will ask you what you want to create. You can also use parameters to specify what you want, like a Helm chart or Kustomize configs. Run the command below:
|
1 |
az aks draft create |
The above command will download the draft CLI for your platform and run it for you. It will ask several questions and display what it is doing.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
[Draft] --- Detecting Language --- ✔ yes [Draft] --> Draft detected Go Checksums (72.289458%) [Draft] --> Could not find a pack for Go Checksums. Trying to find the next likely language match... [Draft] --> Draft detected Go (23.101180%) [Draft] --- Dockerfile Creation --- Please Enter the port exposed in the application: 8080 [Draft] --> Creating Dockerfile... [Draft] --- Deployment File Creation --- ✔ manifests Please Enter the port exposed in the application: 8080 Please Enter the name of the application: super-api [Draft] --> Creating manifests Kubernetes resources... [Draft] Draft has successfully created deployment resources for your project 😃 [Draft] Use 'draft setup-gh' to set up Github OIDC. |
In your folder you will now see extra files and folders:
- A
manifestsfolder with two files:deployment.yamlandservice.yaml - A
Dockerfile
The manifests are pretty basic and just get things done:
- create a Kubernetes deployment that deploys 1 pod
- create a Kubernetes service of type LoadBalancer; that gives you a public IP to reach the app
The app name and port you specified after running az aks draft create is used to create the deployment and service.
The Dockerfile looks like the one below:
|
1 2 3 4 5 6 7 8 9 10 11 12 |
FROM golang ENV PORT 8080 EXPOSE 8080 WORKDIR /go/src/app COPY . . RUN go mod vendor <strong>RUN go build -v -o app</strong> RUN mv ./app /go/bin/ CMD ["app"] |
This is not terribly optimized but it gets the job done. I would highly recommend using a two-stage Dockerfile that results in a much smaller image based on alpine, scratch, or distroless (depending on your programming language).
For my code, the Dockerfile will not work because the source files are not in the root of the repo. Draft cannot know everything. Replace the line that says RUN go build -v -o app with RUN CGO_ENABLED=0 go build -installsuffix 'static' -o app cmd/app/*
To check that the Dockerfile works, if you have Docker installed, run docker build -t draft-super . It will take some time for the base Golang image to be pulled and to download all the dependencies of the app.
When the build is finished, run docker run draft-super to check. The container should run properly.
The az aks draft create command did a pretty good job detecting the programming language and creating the Dockerfile. As we have seen, minor adjustments might be required and the Dockerfile will probably not be production-level quality.
GitHub OIDC setup
At the end of the create command, draft suggested using setup-gh to setup GitHub. Let’s run that command:
|
1 |
az aks draft setup-gh |
Draft will ask for the name of an Azure AD app registration to create. Make sure you are allowed to create those. I used draft-super for the name. Draft will also ask you to confirm the Azure subscription ID and a name of a resource group.
⚠️Although not entirely clear from the question, use the resource group of your AKS cluster (not the MC_ group that contains your nodes!). The setup-gh command will grant the service principal that it creates the Contributor role on the group. This ensures that the GitHub Action azure/aks-set-context@v2.0 works.
Next, draft will ask for the GitHub organization and repo. In my case, that was gbaeke/draft-super. Make sure you have admin access to the repo. GitHub secrets will need to be created. When completed, you should see something like below:
|
1 2 3 4 5 6 |
Enter app registration name: draft-super ✔ <YOUR SUB ID> Enter resource group name: rg-aks ✔ Enter github organization and repo (organization/repoName): gbaeke/draft-super█ [Draft] Draft has successfully set up Github OIDC for your project 😃 [Draft] Use 'draft generate-workflow' to generate a Github workflow to build and deploy an application on AKS. |
Draft has done several things:
- created an app registration (check Azure AD)
- the app registration has federated credentials configured to allow a GitHub workflow to request an Azure AD token when you do pull requests, or push to main or master
- secrets in your GitHub repo:
AZURE_CLIENT_ID,AZURE_SUBSCRIPTION_ID,AZURE_TENANT_ID; these secrets are used by the workflow to request a token from Azure AD using federated credentials - granted the app registration contributor role on the resource group that you specified; that is why you should use the resource group of AKS!
The GitHub workflow you will create in the next step will use the OIDC configuration to request an Azure AD token. The main advantage of this is that you do not need to store Azure secrets in GitHub. The action that does the OIDC-based login is azure/login@v1.4.3.
Draft is now ready to create a GitHub workflow.
Creating the GitHub workflow
Use az aks draft generate-workflow to create the workflow file. This workflow needs the following information as shown below:
|
1 2 3 4 5 6 7 |
Please enter <strong>container registry name</strong>: draftsuper767 ✔ Please enter <strong>container name</strong>: draft-super█ Please enter <strong>cluster resource group name</strong>: rg-aks Please enter <strong>AKS cluster name</strong>: clu-git Please enter name of the <strong>repository branch</strong> to deploy from, usually main: master [Draft] --> Generating Github workflow [Draft] Draft has successfully generated a Github workflow for your project 😃 |
⚠️ Important: use the short name of ACR. Do not append azurecr.io!
⚠️ The container registry needs to be created. Draft does not do that. For best results, create the ACR in the resource group of the AKS cluster because that ensures the service principal created earlier has access to ACR to build images and to enable admin access.
Draft has now created the workflow. As expected, it lives in the .github/workflows local folder.
The workflow runs the following actions:
- Login to Azure using only the client, subscription, and tenant id. No secrets required! 👏 OIDC in action here!
- Run
az acr buildto build the container image. The image is not built on the GitHub runner. The workflow expects ACR to be in the AKS resource group. - Get a Kubernetes context to our AKS cluster and create a secret to allow pulling from ACR; it will also enable the admin user on ACR
- Deploy the application with the
Azure/k8s-deploy@v3.1action. It uses the manifests that were generated withaz aks draft createbut modifies the image and tag to match the newly built image.
Now it is time to commit our code and check the workflow result:
Houston, we have a problem 🚀
For this blog post, I was working in a branch called draft, not main or master. I also changed the workflow file to run on pushes to the draft branch. Of course, the federated tokens in our app registration are not configured for that branch, only master and main. You have to be specific here or you will not get a token. This is the error on GitHub:
To fix this, just modify the app registration and run the workflow again:
After running the workflow again, if buildImage fails, check that ACR is in the AKS resource group and that the service principal has Contributor access to the group. I ran az role assignment list -g rg-aks to see the directly assigned roles and checked that the principalName matched the client ID (application ID) of the draft-super app registration.
If you used the FQDN of ACR instead of just the short name. you can update the workflow environment variable accordingly:
After this change, the image build should be successful.
If you used the wrong ACR name, the deploy step will fail. The image property in deployment.yaml will be wrong. Make the following change in deployment.yaml:
|
1 2 3 4 5 |
- image: draftsuper767.azurecr.io.azurecr.io/draft-super to - image: draftsuper767.azurecr.io/draft-super |
Commit to re-run the workflow. You might need to cancel the previous one because it uses kubectl rollout to check the health of the deployment.
And finally, we have a winner…
In k9s:
You can now make changes to your app and commit your changes to GitHub to deploy new versions or iterations of your app. Note that any change will result in a new image build.
What about the az aks draft up command? It simply combines the setup of GitHub OIDC and the creation of the workflow. So basically, all you ever need to do is:
- create a resource group
- deploy AKS to the resource group
- deploy ACR to the resource group
- Optionally run
az aks update -n -g --attach-acr(this gives the kubelet on each node access to ACR; as we have seen, draft can also create a pull secret) - run
az aks draft createfollowed byaz aks draft up
Conclusion
When working with Draft 2, ensure you first deploy an AKS cluster and Azure Container Registry in the same resource group. You need the Owner role because you will change role-based access control settings.
During OIDC setup, when asked for a resource group, type the AKS resource group. Draft will ensure the service principal it creates, has proper access to the resource group. With that access, it will interact with ACR and log on to AKS.
When asked for the ACR name, use the short name. Do not append azurecr.io! From that point on, it should be smooth sailing! ⛵
In a follow-up post, we will take a look at the draft update command.