Category: rust

Introduction

In October 2021, Microsoft announced the public preview of AKS support for deploying WebAssembly System Interface (WASI) workloads in Kubernetes. You can read the announcement here. In short, that means we can run another type of workload on Kubernetes, besides containers!

WebAssembly is maybe best known for the ability to write code with languages such as C#, Go and Rust that can run in the browser, alongside JavaScript code. One example of this is Blazor, which allows you to build client web apps with C#.

Besides the browser, there are ways to run WebAssembly modules directly on the operating system. Because WebAssembly modules do not contain machine code suitable for a specific operating system and CPU architecture, you will need a runtime that can interpret the WebAssembly byte code. At the same time, WebAssembly modules should be able to interface with the operating system, for instance to access files. In other words, WebAssembly code should be able to access specific parts of the operating system outside the sandbox it is running in by default.

The WebAssembly System Interface (or WASI) allows WebAssembly modules to interact with the outside world. It allows you to declare what the module is allowed to see and access.

One example of a standalone runtime that can run WebAssembly modules is wasmtime. It supports interacting with the host environment via WASI as discussed above. For example, you can specify access to files on the host via the –dir flag and be very specific about what files and folders are allowed.

An example with Rust

In what follows, we will create Hello World-style application with Rust. You do not have to know anything about Rust to follow along. As a matter of fact, I do not know that much about Rust either. I just want a simple app to run on Azure Kubernetes Service later. Here’s the source code:

use std::env;

fn main() {
  println!("Content-Type: text/plain\n");
  println!("Hello, world!");

  printenv();
  
}

fn printenv() {
  for (key, value) in env::vars() {
    println!("{}: {}", key, value);
  }
}

Note: Because I am a bit more comfortable with Go, I first created a demo app with Go and used TinyGo to build the WebAssembly module. That worked great with wasmtime but did not work well on AKS. There is probably a good explanation for that. I will update this post when I learn more.

To continue with the Rust application, it is pretty clear what it does: it prints the Content-Type for a HTTP response, a Hello, World! message, and all environment variables. Why we set the Content-Type will become clearer later on!

To build this app, we need to target wasm32-wasi to build a WebAssembly module that supports WASI as well. You can run the following commands to do so (requires that Rust is installed on your system):

rustup target add wasm32-wasi
cargo build --release --target wasm32-wasi

The rustup command should only be run once. It adds wasm32-wasi as a supported target. The cargo build command then builds the WebAssembly module. On my system, that results in a file in the target/wasm32-wasi/release folder called sample.wasm (name comes from a setting in cargo.toml) . With WebAssembly support in VS Code, I can right click the file and use Show WebAssembly:

We can run this module with cargo run but that runs the app directly on the operating system. In my case that’s Ubuntu in Windows 11’s WSL2. To run the WebAssembly module , you can use wasmtime:

wasmtime sample.wasm

The module will not read the environment variables from the host. Instead, you pass environment variables from the wasmtime cli like so (command and result shown below):

wasmtime --env test=hello sample.wasm

Content-Type: text/plain

Hello, world!
test: hello

Publishing to Azure Container Registry

A WebAssembly can be published to Azure Container Registry with wasm-to-oci (see GitHub repo). The command below publishes our module:

wasm-to-oci push sample.wasm <ACRNAME>.azurecr.io/sample:1.0.0

Make sure you are logged in to ACR with az acr login -n <ACRNAME>. I also enabled anonymous pull on ACR to not run into issues with pulls from WASI-enabled AKS pools later. Indeed, AKS will be able to pull these artefacts to run them on a WASI node.

Here is the artefact as shown in ACR:

Running the module on AKS

To run WebAssembly modules on AKS nodes, you need to enable the preview as described here. After enabling the preview, I deployed a basic Kubernetes cluster with one node. It uses kubenet by default. That’s good because Azure CNI is not supported by WASI node pools.

az aks create -n wademo -g rg-aks --node-count 1

After finishing the deployment, I added a WASI nodepool:

az aks nodepool add \
    --resource-group rg-aks \
    --cluster-name wademo \
    --name wasipool \
    --node-count 1 \
    --workload-runtime wasmwasi

The aks-preview extension (install or update it!!!) for the Azure CLI supports the –workload-runtime flag. It can be set to wasmwasi to deploy nodes that can execute WebAssembly modules. The piece of technology that enables this is the krustlet project as described here: https://krustlet.dev. Krustlet is basically a WebAssembly kubelet. It stands for Kubernetes Rust Kubelet.

After running the above commands, the command kubectl get nodes -o wide will look like below:

NAME                                STATUS   ROLES   AGE    VERSION         INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION     CONTAINER-RUNTIME
aks-nodepool1-23291395-vmss000000   Ready    agent   3h6m   v1.20.9         10.240.0.4    <none>        Ubuntu 18.04.6 LTS   5.4.0-1059-azure   containerd://1.4.9+azure
aks-wasipool-23291395-vmss000000    Ready    agent   3h2m   1.0.0-alpha.1   10.240.0.5    <none>        <unknown>            <unknown>          mvp

As you can see it’s early days here! 😉 But we do have a node that can run WebAssembly! Let’s try to run our module by deploying a pod via the manifest below:

apiVersion: v1
kind: Pod
metadata:
  name: sample
  annotations:
    alpha.wagi.krustlet.dev/default-host: "0.0.0.0:3001"
    alpha.wagi.krustlet.dev/modules: |
      {
        "sample": {"route": "/"}
      }
spec:
  hostNetwork: true
  containers:
    - name: sample
      image: <ARCNAME>.azurecr.io/sample:1.0.0
      imagePullPolicy: Always
  nodeSelector:
    kubernetes.io/arch: wasm32-wagi
  tolerations:
    - key: "node.kubernetes.io/network-unavailable"
      operator: "Exists"
      effect: "NoSchedule"
    - key: "kubernetes.io/arch"
      operator: "Equal"
      value: "wasm32-wagi"
      effect: "NoExecute"
    - key: "kubernetes.io/arch"
      operator: "Equal"
      value: "wasm32-wagi"
      effect: "NoSchedule"

Wait a moment! There is a new acronym here: WAGI! WASI has no network primitives such as sockets so you should not expect to build a full webserver with it. WAGI, which stands for WebAssembly Gateway Interface, allows you to run WASI modules as HTTP handlers. It is heavily based on CGI, the Common Gateway Interface that allows mapping HTTP requests to executables (e.g. a Windows or Linux executable) via something like IIS or Apache.

We will need a way to map a route such as / to a module and the response to a requests should be HTTP responses. That is why we set the Content-Type in the example by simply printing it to stdout. WAGI will also set several environment variables with information about the incoming request. That is the reason we print all the environment variables. This feels a bit like the early 90’s to me when CGI was the hottest web tech in town! 😂

The mapping of routes to modules is done via annotations, as shown in the YAML. This is similar to the modules.toml file used to start a Wagi server manually. Because the WASI nodes are tainted, tolerations are used to allow the pod to be scheduled on such nodes. With the nodeSelector, the pod needs to be scheduled on such a node.

To run the WebAssembly module, apply the manifest above to the cluster as usual (assuming the manifest is in pod.yaml:

kubectl apply -f pod.yaml

Now run kubectl get pods. If the status is Registered vs Running, this is expected. The pod will not be ready either:

NAME    READY   STATUS       RESTARTS   AGE
sample  0/1     Registered   0          108m

In order to reach the workload from the Internet, you need to install nginx with a value.yaml file that contains the internal IP address of the WASI node as documented here.

After doing that, I can curl the public IP address of the nginx service of type LoadBalancer:

~ curl IP

Hello, world!
HTTP_ACCEPT: */*
QUERY_STRING: 
SERVER_PROTOCOL: HTTP/1.0
GATEWAY_INTERFACE: CGI/1.1
REQUEST_METHOD: GET
SERVER_PORT: 3001
REMOTE_ADDR: 10.240.0.4
X_FULL_URL: http://10.240.0.5:3001/
X_RAW_PATH_INFO: 
CONTENT_TYPE: 
SERVER_NAME: 10.240.0.5
SCRIPT_NAME: /
AUTH_TYPE: 
PATH_TRANSLATED: 
PATH_INFO: 
CONTENT_LENGTH: 0
X_MATCHED_ROUTE: /
REMOTE_HOST: 10.240.0.4
REMOTE_USER: 
SERVER_SOFTWARE: WAGI/1
HTTP_HOST: 10.240.0.5:3001
HTTP_USER_AGENT: curl/7.58.0

As you can see, WAGI has set environment variables that allows your handler to know more about the incoming request such as the HTTP User Agent.

Conclusion

Although WebAssembly is gaining in popularity to build browser-based applications, it is still early days for running these workloads on Kubernetes. WebAssembly will not replace containers anytime soon. In fact, that is not the actual goal. It just provides an additional choice that might make sense for some applications in the future. And as always, the future will arrive sooner than expected!