Tag: fluxv2

Quick Guide to Flux v2 on AKS

Now that the Flux v2 extension for Azure Kubernetes Service and Azure Arc is generally available, let’s do a quick guide on the topic. A Quick Guide, at least on this site 😉, is a look at the topic from a command-line perspective for easy reproduction and evaluation.

This Quick Guide is also on GitHub.

Requirements

You need the following to run the commands:

  • An Azure subscription with a deployed AKS cluster; a single node will do
  • Azure CLI and logged in to the subscription with owner access
  • All commands run in bash, in my case in WSL 2.0 on Windows 11
  • kubectl and a working kube config (use az aks get-credentials)

Step 1: Register AKS-ExtensionManager and configure Azure CLI

Flux v2 is installed via an extension. The extension takes care of installing Flux controllers in the cluster and keeping them up-to-date when there is a new version. For extensions to work with AKS, you need to register the AKS-ExtensionManager feature in the Microsoft.ContainerService namespace.

# register the feature
az feature register --namespace Microsoft.ContainerService --name AKS-ExtensionManager

# after a while, check if the feature is registered
# the command below should return "state": "Registered"
az feature show --namespace Microsoft.ContainerService --name AKS-ExtensionManager | grep Registered

# ensure you run Azure CLI 2.15 or later
# the command will show the version; mine showed 2.36.0
az version | grep '"azure-cli"'

# register the following providers; if these providers are already
# registered, it is safe to run the commands again

az provider register --namespace Microsoft.Kubernetes
az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.KubernetesConfiguration

# enable CLI extensions or upgrade if there is a newer version
az extension add -n k8s-configuration --upgrade
az extension add -n k8s-extension --upgrade

# check your Azure CLI extensions
az extension list -o table

Step 2: Install Flux v2

We can now install Flux v2 on an existing cluster. There are two types of clusters:

  • managedClusters: AKS
  • connectedClusters: Azure Arc-enabled clusters

To install Flux v2 on AKS and check the configuration, run the following commands:

RG=rg-aks
CLUSTER=clu-pub

# list installed extensions
az k8s-extension list -g $RG -c $CLUSTER -t managedClusters

# install flux; note that the name (-n) is a name you choose for
# the extension instance; the command will take some time
# this extension will be installed with cluster-wide scope

az k8s-extension create -g $RG -c $CLUSTER -n flux --extension-type microsoft.flux -t managedClusters --auto-upgrade-minor-version true

# list Kubernetes namespaces; there should be a flux-system namespace
kubectl get ns

# get pods in the flux-system namespace
kubectl get pods -n flux-system

The last command shows all the pods in the flux-system namespace. If you have worked with Flux without the extension, you will notice four familiar pods (deployments):

  • Kustomize controller: installs manifests (.yaml files) from configured sources, optionally using kustomize
  • Helm controller: installs Helm charts
  • Source controller: configures sources such as git or Helm repositories
  • Notification controller: handles notifications such as those sent to Teams or Slack

Microsoft adds two other services:

  • Flux config agent: communication with the data plane (Azure); reports back information to Azure about the state of Flux such as reconciliations
  • Flux configuration controller: manages Flux on the cluster; checks for Flux Configurations that you create with the Azure CLI

Step 3: Create a Flux configuration

Now that Flux is installed, we can create a Flux configuration. Note that Flux configurations are not native to Flux. A Flux configuration is an abstraction, created by Microsoft, that configures Flux sources and customizations for you. You can create these configurations from the Azure CLI. The configuration below uses a git repository https://github.com/gbaeke/gitops-flux2-quick-guide. It is a fork of https://github.com/Azure/gitops-flux2-kustomize-helm-mt.

⚠️ In what follows, we create a Flux configuration based on the Microsoft sample repo. If you want to create a repo and resources from scratch, see the Quick Guides on GitHub.

# create the configuration; this will take some time
az k8s-configuration flux create -g $RG -c $CLUSTER \
  -n cluster-config --namespace cluster-config -t managedClusters \
  --scope cluster \
  -u https://github.com/gbaeke/gitops-flux2-quick-guide \
  --branch main  \
  --kustomization name=infra path=./infrastructure prune=true \
  --kustomization name=apps path=./apps/staging prune=true dependsOn=["infra"]

# check namespaces; there should be a cluster-config namespace
kubectl get ns

# check the configuration that was created in the cluster-config namespace
# this is a resource of type FluxConfig
# in the spec, you will find a gitRepository and two kustomizations

kubectl get fluxconfigs cluster-config -o yaml -n cluster-config

# the Microsoft flux controllers create the git repository source
# and the two kustomizations based on the flux config created above
# they also report status back to Azure

# check the git repository; this is a resource of kind GitRepository
# the Flux source controller uses the information in this
# resource to download the git repo locally

kubectl get gitrepo cluster-config -o yaml -n cluster-config

# check the kustomizations
# the infra kustomization uses folder ./infrastructure in the
# git repository to install redis and nginx with Helm charts
# this kustomization creates other Flux resources such as
# Helm repos and Helm Releases; the Helm Releases are used
# to install nginx and redis with their respective Helm
# charts

kubectl get kustomizations cluster-config-infra -o yaml -n cluster-config

# the app kustomization depends on infra and uses the ./apps
# folder in the repo to install the podinfo application via
# a kustomize overlay (staging)

kubectl get kustomizations cluster-config-apps -o yaml -n cluster-config

In the portal, you can check the configuration:

Flux config in the Azure Portal

The two kustomizations that you created, create other configuration objects such as Helm repositories and Helm releases. They too can be checked in the portal:

Configuration objects in the Azure Portal

Conclusion

With the Flux extension, you can install Flux on your cluster and keep it up-to-date. The extension not only installs the Flux open source components. It also installs Microsoft components that enable you to create Flux Configurations and report back status to the portal. Flux Configurations are an abstraction on top of Flux, that makes adding sources and kustomizations easier and more integrated with Azure.

An introduction to Flux v2

If you have read my blog and watched my Youtube channel, you know I have worked with Flux in the past. Flux, by weaveworks, is a GitOps Kubernetes Operator that ensures that your cluster state matches the desired state described in a git repository. There are other solutions as well, such as Argo CD.

With Flux v2, GitOps on Kubernetes became a lot more powerful and easier to use. Flux v2 is built on a set of controllers and APIs called the GitOps Toolkit. The toolkit contains the following components:

  • Source controller: allows you to create sources such as a GitRepository or a HelmRepository; the source controller acts on several custom resource definitions (CRDs) as defined in the docs
  • Kustomize controller: runs continuous delivery pipelines defined with Kubernetes manifests (YAML) files; although you can use kustomize and define kustomization.yaml files, you do not have to; internally though, Flux v2 uses kustomize to deploy your manifests; the kustomize controller acts on Kustomization CRDs as defined here
  • Helm controller: deploy your workloads based on Helm charts but do so declaratively; there is no need to run helm commands; see the docs for more information
  • Notification controller: responds to incoming events (e.g. from a git repo) and sends outgoing events (e.g. to Teams or Slack); more info here

If you throw it all together, you get something like this:

GitOps Toolkit components that make up Flux v2 (from https://toolkit.fluxcd.io/)

Getting started

To get started, you should of course look at the documentation over at https://toolkit.fluxcd.io. I also created a series of videos about Flux v2. The first one talks about Flux v2 in general and shows how to bootstrap a cluster.

Part 1 in the series about Flux v2

Although Flux v2 works with other source control systems than GitHub, for instance GitLab, I use GitHub in the above video. I also use kind, to make it easy to try out Flux v2 on your local machine. In subsequent videos, I use Azure Kubernetes Services (AKS).

In Flux v2, it is much easier to deploy Flux on your cluster with the flux bootstrap command. Flux v2 itself is basically installed and managed via GitOps principles by pushing all Flux v2 manifests to a git repository and running reconciliations to keep the components running as intended.

Kustomize

Flux v1 already supported kustomize but v2 takes it to another level. Whenever you want to deploy to Kubernetes with YAML manifests, you will create a kustomization, which is based on the Kustomization CRD. A kustomization is defined as below:

apiVersion: kustomize.toolkit.fluxcd.io/v1beta1
kind: Kustomization
metadata:
  name: realtimeapp-dev
  namespace: flux-system
spec:
  healthChecks:
  - kind: Deployment
    name: realtime-dev
    namespace: realtime-dev
  - kind: Deployment
    name: redis-dev
    namespace: realtime-dev
  interval: 1m0s
  path: ./deploy/overlays/dev
  prune: true
  sourceRef:
    kind: GitRepository
    name: realtimeapp-infra
  timeout: 2m0s
  validation: client

A kustomization requires a source. In this case, the source is a git repository called realtimeapp-infra that was already defined in advance. The source just points to a public git repository on Github: https://github.com/gbaeke/realtimeapp-infra.

The source contains a deploy folder, which contains a bases and an overlays folder. The kustomization points to the ./deploy/overlays/dev folder as set in path. That folder contains a kustomization.yaml file that deploys an application in a development namespace and uses the base from ./deploy/bases/realtimeapp as its source. If you are not sure what kustomize exactly does, I made a video that tries 😉 to explain it:

An introduction to kustomize

It is important to know that you do not need to use kustomize in your source files. If you point a Flux v2 kustomization to a path that just contains a bunch of YAML files, it will work equally well. You do not have to create a kustomization.yaml file in that folder that lists the resources (YAML files) that you want to deploy. Internally though, Flux v2 will use kustomize to deploy the manifests and uses the deployment order that kustomize uses: first namespaces, then services, then deployments, etc…

The interval in the kustomization (above set at 1 minute) means that your YAML files are applied at that interval, even if the source has not changed. This ensures that, if you modified resources on your cluster, the kustomization will reset the changes to the state as defined in the source. The source itself has its own interval. If you set a GitRepository source to 1 minute, the source is checked every 1 minute. If the source has changes, the kustomizations that depend on the source will be notified and proceed to deploy the changes.

A GitRepository source can refer to a specific branch, but can also refer to a semantic versioning tag if you use a semver range in the source. See checkout strategies for more information.

Deploying YAML manifests

If the above explanation of sources and kustomizations does not mean much to you, I created a video that illustrates these aspects more clearly:

In the above video, the source that points to https://github.com/gbaeke/realtimeapp-infra gets created first (see it at this mark). Next, I create two kustomizations, one for development and one for production. I use a kustomize base for the application plus two overlays, one for dev and one for production.

What to do when the app container images changes?

Flux v1 has a feature that tracks container images in a container registry and updates your cluster resources with a new image based on a filter you set. This requires read/write access to your git repository because Flux v1 set the images in your source files. Flux v2 does not have this feature yet (November 2020, see https://toolkit.fluxcd.io/roadmap).

In my example, I use a GitHub Action in the application source code repository to build and push the application image to Docker Hub. The GitHub action triggers a build job on two events:

  • push to main branch: build a container image with a short sha as the tag (e.g. gbaeke/flux-rt:sha-94561cb
  • published release: build a container image with the release version as the tag (e.g. gbaeke/flux-rt:1.0.1)

When the build is caused by a push to main, the update-dev-image job runs. It modifies kustomization.yaml in the dev overlay with kustomize edit:

update-dev-image:
    runs-on: ubuntu-latest
    if: contains(github.ref, 'heads')
    needs:
    - build
    steps:
    - uses: imranismail/setup-kustomize@v1
      with:
        kustomize-version: 3.8.6
    - run: git clone https://${REPO_TOKEN}@github.com/gbaeke/realtimeapp-infra.git .
      env:
        REPO_TOKEN: ${{secrets.REPO_TOKEN}}
    - run: kustomize edit set image gbaeke/flux-rt:sha-$(git rev-parse --short $GITHUB_SHA)
      working-directory: ./deploy/overlays/dev
    - run: git add .
    - run: |
        git config user.email "$EMAIL"
        git config user.name "$GITHUB_ACTOR"
      env:
        EMAIL: ${{secrets.EMAIL}}
    - run: git commit -m "Set dev image tag to short sha"
    - run: git push

Similarly, when the build is caused by a release, the image is updated in the production overlay’s kustomization.yaml file.

Conclusion

If you are interested in GitOps as an alternative for continuous delivery to Kubernetes, do check out Flux v2 and see if it meets your needs. I personally like it a lot and believe that they are setting the standard for GitOps on Kubernetes. I have not covered Helm deployments, monitoring and alerting features yet. I will create additional videos and posts about those features in the near future. Stay tuned!

%d bloggers like this: