Tag: draft

Trying out Draft 2 on AKS

Sadly no post about good Belgian beer 🍺.

Draft 2 is an open-source project that aims 🎯 to make things easier for developers that build Kubernetes applications. It can improve the inner dev loop, where the developers code and test their apps, in the following ways:

  • Automate the creation of a Dockerfile
  • Automate the creation of Kubernetes manifests, Helm charts, or Kustomize configs
  • Generate a GitHub Action workflow to build and deploy the application when you push changes

I have worked with Draft 1 in the past, and it worked quite well. Now Microsoft has integrated Draft 2 in the Azure CLI to make it part of the Kubernetes on Azure experience. A big difference with Draft 1 is that Draft 2 makes use of GitHub Actions (Wait? No Azure DevOps? 😲) to build and push your images to the development cluster. It uses GitHub OpenID Connect (OIDC) for Azure authentication.

That is quite a change and lots of bits and pieces that have to be just right. Make sure you know about Azure AD App Registrations, GitHub, GitHub Actions, Docker, etc… when the time comes to troubleshoot.

Let’s see what we can do? 👀

Prerequisites

At this point in time (June 2022), Draft for Azure Kubernetes Service is in preview. Draft itself can be found here: https://github.com/Azure/draft

The only thing you need to do is to install or upgrade the aks-preview extension:

az extension add --name aks-preview --upgrade

Next, type az aks draft -h to check if the command is available. You should see the following options:

create           
generate-workflow
setup-gh
up
update

We will look at the first four commands in this post.

Running draft create

With az aks draft create, you can generate a Dockerfile for your app, Kubernetes manifests, Helm charts, and Kustomize configurations. You should fork the following repository and clone it to your machine: https://github.com/gbaeke/draft-super

After cloning it, cd into draft-super and run the following command (requires go version 1.16.4 or higher):

CGO_ENABLED=0 go build -installsuffix 'static' -o app cmd/app/*
./app

The executable runs a web server on port 8080 by default. If that conflicts with another app on your system set the port with the port environment variable: run PORT=9999 ./app instead of just ./app. Now we know the app works, we need a Dockerfile to containerize it.

You will notice that there is no Dockerfile. Although you could create one manually, you can use draft for this. Draft will try to recognize your code and generate the Dockerfile. We will keep it simple and just create Kubernetes manifests. When you run draft without parameters, it will ask you what you want to create. You can also use parameters to specify what you want, like a Helm chart or Kustomize configs. Run the command below:

az aks draft create

The above command will download the draft CLI for your platform and run it for you. It will ask several questions and display what it is doing.

[Draft] --- Detecting Language ---
✔ yes
[Draft] --> Draft detected Go Checksums (72.289458%)

[Draft] --> Could not find a pack for Go Checksums. Trying to find the next likely language match...
[Draft] --> Draft detected Go (23.101180%)

[Draft] --- Dockerfile Creation ---
Please Enter the port exposed in the application: 8080
[Draft] --> Creating Dockerfile...

[Draft] --- Deployment File Creation ---
✔ manifests
Please Enter the port exposed in the application: 8080
Please Enter the name of the application: super-api
[Draft] --> Creating manifests Kubernetes resources...

[Draft] Draft has successfully created deployment resources for your project 😃
[Draft] Use 'draft setup-gh' to set up Github OIDC.

In your folder you will now see extra files and folders:

  • A manifests folder with two files: deployment.yaml and service.yaml
  • A Dockerfile

The manifests are pretty basic and just get things done:

  • create a Kubernetes deployment that deploys 1 pod
  • create a Kubernetes service of type LoadBalancer; that gives you a public IP to reach the app

The app name and port you specified after running az aks draft create is used to create the deployment and service.

The Dockerfile looks like the one below:

FROM golang
ENV PORT 8080
EXPOSE 8080

WORKDIR /go/src/app
COPY . .

RUN go mod vendor
RUN go build -v -o app  
RUN mv ./app /go/bin/

CMD ["app"]

This is not terribly optimized but it gets the job done. I would highly recommend using a two-stage Dockerfile that results in a much smaller image based on alpine, scratch, or distroless (depending on your programming language).

For my code, the Dockerfile will not work because the source files are not in the root of the repo. Draft cannot know everything. Replace the line that says RUN go build -v -o app with RUN CGO_ENABLED=0 go build -installsuffix 'static' -o app cmd/app/*

To check that the Dockerfile works, if you have Docker installed, run docker build -t draft-super . It will take some time for the base Golang image to be pulled and to download all the dependencies of the app.

When the build is finished, run docker run draft-super to check. The container should run properly.

The az aks draft create command did a pretty good job detecting the programming language and creating the Dockerfile. As we have seen, minor adjustments might be required and the Dockerfile will probably not be production-level quality.

GitHub OIDC setup

At the end of the create command, draft suggested using setup-gh to setup GitHub. Let’s run that command:

az aks draft setup-gh

Draft will ask for the name of an Azure AD app registration to create. Make sure you are allowed to create those. I used draft-super for the name. Draft will also ask you to confirm the Azure subscription ID and a name of a resource group.

⚠️Although not entirely clear from the question, use the resource group of your AKS cluster (not the MC_ group that contains your nodes!). The setup-gh command will grant the service principal that it creates the Contributor role on the group. This ensures that the GitHub Action azure/aks-set-context@v2.0 works.

Next, draft will ask for the GitHub organization and repo. In my case, that was gbaeke/draft-super. Make sure you have admin access to the repo. GitHub secrets will need to be created. When completed, you should see something like below:

Enter app registration name: draft-super
✔ <YOUR SUB ID>
Enter resource group name: rg-aks
✔ Enter github organization and repo (organization/repoName): gbaeke/draft-super█
[Draft] Draft has successfully set up Github OIDC for your project 😃
[Draft] Use 'draft generate-workflow' to generate a Github workflow to build and deploy an application on AKS.

Draft has done several things:

  • created an app registration (check Azure AD)
  • the app registration has federated credentials configured to allow a GitHub workflow to request an Azure AD token when you do pull requests, or push to main or master
  • secrets in your GitHub repo:AZURE_CLIENT_ID, AZURE_SUBSCRIPTION_ID,AZURE_TENANT_ID; these secrets are used by the workflow to request a token from Azure AD using federated credentials
  • granted the app registration contributor role on the resource group that you specified; that is why you should use the resource group of AKS!

The GitHub workflow you will create in the next step will use the OIDC configuration to request an Azure AD token. The main advantage of this is that you do not need to store Azure secrets in GitHub. The action that does the OIDC-based login is azure/login@v1.4.3.

Draft is now ready to create a GitHub workflow.

Creating the GitHub workflow

Use az aks draft generate-workflow to create the workflow file. This workflow needs the following information as shown below:

Please enter container registry name: draftsuper767
✔ Please enter container name: draft-super█
Please enter cluster resource group name: rg-aks
Please enter AKS cluster name: clu-git
Please enter name of the repository branch to deploy from, usually main: master
[Draft] --> Generating Github workflow
[Draft] Draft has successfully generated a Github workflow for your project 😃

⚠️ Important: use the short name of ACR. Do not append azurecr.io!

⚠️ The container registry needs to be created. Draft does not do that. For best results, create the ACR in the resource group of the AKS cluster because that ensures the service principal created earlier has access to ACR to build images and to enable admin access.

Draft has now created the workflow. As expected, it lives in the .github/workflows local folder.

The workflow runs the following actions:

  • Login to Azure using only the client, subscription, and tenant id. No secrets required! 👏 OIDC in action here!
  • Run az acr build to build the container image. The image is not built on the GitHub runner. The workflow expects ACR to be in the AKS resource group.
  • Get a Kubernetes context to our AKS cluster and create a secret to allow pulling from ACR; it will also enable the admin user on ACR
  • Deploy the application with the Azure/k8s-deploy@v3.1 action. It uses the manifests that were generated with az aks draft create but modifies the image and tag to match the newly built image.

Now it is time to commit our code and check the workflow result:

Looks fine at first glance…

Houston, we have a problem 🚀

For this blog post, I was working in a branch called draft, not main or master. I also changed the workflow file to run on pushes to the draft branch. Of course, the federated tokens in our app registration are not configured for that branch, only master and main. You have to be specific here or you will not get a token. This is the error on GitHub:

Oops

To fix this, just modify the app registration and run the workflow again:

Quick and dirty fix: update mainfic with a subject identifier for draft; you can also add a new credential

After running the workflow again, if buildImage fails, check that ACR is in the AKS resource group and that the service principal has Contributor access to the group. I ran az role assignment list -g rg-aks to see the directly assigned roles and checked that the principalName matched the client ID (application ID) of the draft-super app registration.

If you used the FQDN of ACR instead of just the short name. you can update the workflow environment variable accordingly:

ACR name should be the short name

After this change, the image build should be successful.

Looking better

If you used the wrong ACR name, the deploy step will fail. The image property in deployment.yaml will be wrong. Make the following change in deployment.yaml:

- image: draftsuper767.azurecr.io.azurecr.io/draft-super

to

- image: draftsuper767.azurecr.io/draft-super

Commit to re-run the workflow. You might need to cancel the previous one because it uses kubectl rollout to check the health of the deployment.

And finally, we have a winner…

🍾🍾🍾

In k9s:

super-api deployed to default namespace

You can now make changes to your app and commit your changes to GitHub to deploy new versions or iterations of your app. Note that any change will result in a new image build.

What about the az aks draft up command? It simply combines the setup of GitHub OIDC and the creation of the workflow. So basically, all you ever need to do is:

  • create a resource group
  • deploy AKS to the resource group
  • deploy ACR to the resource group
  • Optionally run az aks update -n -g --attach-acr (this gives the kubelet on each node access to ACR; as we have seen, draft can also create a pull secret)
  • run az aks draft create followed by az aks draft up

Conclusion

When working with Draft 2, ensure you first deploy an AKS cluster and Azure Container Registry in the same resource group. You need the Owner role because you will change role-based access control settings.

During OIDC setup, when asked for a resource group, type the AKS resource group. Draft will ensure the service principal it creates, has proper access to the resource group. With that access, it will interact with ACR and log on to AKS.

When asked for the ACR name, use the short name. Do not append azurecr.io! From that point on, it should be smooth sailing! ⛵

In a follow-up post, we will take a look at the draft update command.

Streamlined Kubernetes Development with Draft

A longer time ago, I wrote a post about draft. Draft is a tool to streamline your Kubernetes development experience. It basically automates, based on your code, the creation of a container image, storing the image in a registry and installing a container based on that image using a Helm chart. Draft is meant to be used during the development process while you are still messing around with your code. It is not meant as a deployment mechanism in production.

The typical workflow is the following:

  • in the folder with your source files, run draft create
  • to build, push and install the container run draft up; in the background a Helm chart is used
  • to see the logs and connect to the app in your container over an SSH tunnel, run draft connect
  • modify your code and run draft up again
  • rinse and repeat…

Let’s take a look at how it works in a bit more detail, shall we?

Prerequisites

Naturally, you need a Kubernetes cluster with kubectl, the Kubernetes cli, configured to use that cluster.

Next, install Helm on your system and install Tiller, the server-side component of Helm on the cluster. Full installation instructions are here. If your cluster uses rbac, check out how to configure the proper service account and role binding. Run helm init to initialize Helm locally and install Tiller at the same time.

Now install draft on your system. Check out the quickstart for installation instructions. Run draft init to initialize it.

Getting some source code

Let’s use a small Go program to play with draft. You can use the realtime-go repository. Clone it to your system and checkout the httponly branch:

git clone https://github.com/gbaeke/realtime-go.git
git checkout httponly

You will need a redis server as a back-end for the realtime server. Let’s install that the quick and dirty way:

kubectl run redis --image=redis --replicas=1 
kubectl expose deploy/redis –port 6379

Running draft create

In the realtime-go folder, run draft create. You should get the following output:

draft create output

The command tries to detect the language and it found several. In this case, because there is no pack for Coq (what is that? 😉) and HTML, it used Go. Knowing the language, draft creates a simple Dockerfile if there is no such file in the folder:

FROM golang
ENV PORT 8080
EXPOSE 8080

WORKDIR /go/src/app
COPY . .

RUN go get -d -v ./...
RUN go install -v ./...

CMD ["app"]

Usually, I do not use the Dockerfile created by draft. If there already is a Dockerfile in the folder, draft will use that one. That’s what happened in our case because the folder contains a 2-stage Dockerfile.

Draft created some other files as well:

  • draft.toml: configuration file (more info); can be used to create environments like staging and production with different settings such as the Kubernetes namespace to deploy to or the Dockerfile to use
  • draft.tasks.toml: run commands before or after you deploy your container with draft (more info); we could have used this to install and remove the redis container
  • .draftignore: yes, to ignore stuff

Draft also created a charts folder that contains the Helm chart that draft will use to deploy your container. It can be modified to suit your particular needs as we will see later.

Helm charts folder and a partial view on the deployment.yaml file in the chart

Setting the container registry

In older versions of draft, the source files were compressed and sent to a sever-side component that created the container. At present though, the container is built locally and then pushed to a registry of your choice. If you want to use Azure Container Registry (ACR), run the following commands (set and login):

draft config set registry REGISTRYNAME.azurecr.io
az acr login -n REGISTRYNAME

Note that you need the Azure CLI for the last command. You also need to set the subscription to the one that contains the registry you reference.

With this configuration, you need Docker on your system. Docker will build and push the container. If you want to build in the cloud, you can use ACR Build Tasks. To do that, use these commands:

draft config set container-builder acrbuild
draft config set registry REGISTRYNAME.azurecr.io
draft config set resource-group-name RESOURCEGROUPNAME

Make sure your are logged in to the subscription (az login) and login to ACR as well before continuing. In this example, I used ACR build tasks.

Note: because ACR build tasks do not cache intermediate layers, this approach can lead to longer build times; when the image is small as in this case, doing a local build and push is preferred!

Running draft up

We are now ready to run draft up. Let’s do so and see what happens:

results of draft up

YES!!!! Draft built the container image and released it. Run helm ls to check the release. It did not have to push the image because it was built in ACR and pushed from there. Let’s check the ACR build logs in the portal (you can also use the draft logs command):

acr build log for the 2-stage Docker build

Fixing issues

Although the container is properly deployed (check it with helm ls), if you run kubectl get pods you will notice an error:

container error

In this case, the container errors out because it cannot find the redis host, which is a dependency. We can tell the container to look for redis via a REDISHOST environment variable. You can add it to deployment.yaml in the chart like so:

environment variable in deployment.yaml

After this change, just run draft up again and hope for the best!

Running draft connect

With the realtime-go container up and running, run draft connect:

output of draft connect

This maps a local port on your system to the remote port over an ssh tunnel. In addition, it streams the logs from the container. You can now connect to http://localhost:18181 (or whatever port you’ll get):

Great success! The app is running

If you want a public IP for your service, you can modify the Helm chart. In values.yaml, set service.type to LoadBalancer instead of ClusterIP and run draft up again. You can verify the external IP by running kubectl get svc.

Conclusion

Working with draft while your are working on one or more containers and still hacking away at your code really is smooth sailing. If you are not using it yet, give it a go and see if you like it. I bet you will!

%d bloggers like this: