Teleport or Artifact Streaming in Azure Kubernetes

I don't know about you, but I've spent too much time staring at my terminal or Lens instance watching Kubernetes events, waiting for the dreaded "Pulling image..." status to change. It's one of those things that we've all accepted as normal - you deploy a pod, then wait as the container runtime downloads the entire image before your application can start.

The worst part? As our applications grow more complex and our images get larger, this problem worsens. With 1-2 GB images, you don't wait that long, but I had a time when I needed to use Windows nodes on Kubernetes, and those images can get up to 30 GB in size. You don't want to imagine how long that takes. In that case, I needed a Windows AKS node to run a simple script in Exchange Online to update distribution groups, but it couldn't run on Linux because of some module limitations.

Anyway, after some time and very insistent feedback towards the module developers, the problem was solved, and I could return to the plain ol' Linux nodes. While a huge image size problem was solved, the time spent waiting for regular images didn't get solved until Artifact streaming was announced.

What is Artifact Streaming, and Why Should You Care?

Let's start with the basics. Artifact Streaming changes how Kubernetes pulls and starts container images. Instead of downloading the entire image before starting the container (the traditional approach), containers can start with just the essential files and then download the rest in the background while your application runs.

Think of it as downloading an entire movie before watching and streaming it. With streaming, you can start watching immediately while the rest downloads in the background—that's exactly what's happening with container images now.

This concept originated in Azure with a feature initially called "Teleport" before being renamed "Artifact Streaming." Google has implemented a similar approach called "Image Streaming" in GKE. The idea is simple but powerful—don't make applications wait for data they might not need immediately.

How Much Faster Are We Talking?

According to Azure's measurements, Artifact Streaming can reduce container startup times by over 15% on average, but that's conservative in my experience. When testing with larger images (4-5GB range), I've seen truly dramatic improvements —pods that used to take 1-2 minutes to start now becoming ready in 10-20 seconds.

This directly translates to real-world benefits:

1. Faster autoscaling responses - When your cluster needs to scale up to handle traffic spikes, new nodes become productive almost immediately rather than spending minutes just pulling images.
2. Improved developer experience—Let's be honest: Waiting for image pulls during development and testing is mind-numbingly boring. Faster startup means quicker feedback loops and less time wasted.
3. More reliable deployments—We've all experienced timeouts during deployment because image pulls took too long. With Artifact Streaming, that problem largely disappears.
4. Better disaster recovery - When a node fails and workloads must be rescheduled, recovery happens much more quickly.

The Technical Magic Behind It

Microsoft doesn't say exactly how Artifact Streaming works. Still, if you check the GKE documentation for Image Streaming, you will notice similarities between the technologies, so I assume they work almost the same. If not, then my bad :)

So rather than using the traditional approach, where the container runtime downloads the entire image before mounting it, it uses a remote filesystem implementation that creates a streaming connection to the container registry. When a pod starts, the container begins execution with this remote filesystem, fetching only the specific files the application needs to initialize. Meanwhile, the rest of the image continues downloading in the background and is cached locally for future use.

This is particularly beneficial for large images where applications only need a small subset of files to start running. Why wait for 5GB to download when your app only needs 50MB to initialize?

Setting It Up in Azure Kubernetes Service

Enabling Artifact Streaming is quite straightforward if you're running AKS. First, you'll need to ensure your prerequisites are met:

• Kubernetes version 1.25 or later
• Ubuntu 22.04 or Azure Linux nodes
• Premium tier Azure Container Registry

To enable Artifact Streaming for your images in ACR, you can use the Azure CLI:

# Enable artifact streaming for a specific repository
az acr repository update --name myregistry --repository myrepo --enable-artifact-streaming true

# Or for a specific image tag
az acr repository update --name myregistry --image myrepo:tag --enable-artifact-streaming true

Then, you'll need to enable it on your AKS node pools:

# Enable artifact streaming on an existing node pool
az aks nodepool update --resource-group myResourceGroup --cluster-name myCluster --name mynodepool --enable-artifact-streaming

After these steps, your deployments will automatically begin using Artifact Streaming for eligible images. It's one of those rare features that just works after you turn it on - no application changes are required.

The Limitations You Should Know About

Now, I'm not going to pretend this technology is perfect. Like any newer feature, Artifact Streaming comes with some limitations you should be aware of:

1. Image compatibility - Initially, only Linux AMD64 images were supported, with Windows and ARM64 support coming later (targeting 2024).
2. Size considerations - While it works best for images under 30GB, extremely large images might still face bottlenecks.
3. Read-heavy startup patterns - The benefits might be less pronounced if your application needs to read many scattered files during startup.
4. Registry requirements - Azure requires a Premium SKU registry during preview.
5. Node OS requirements - You need specific OS versions that support the underlying filesystem features.

Problems I've had with it

It wouldn't be me if I didn't have a problem with a particular system or feature. If you're using Kubernetes to run PowerShell automation scripts like I do, you will find out very fast that many problems arise when you activate artifact streaming. Loading modules, for example, is one of the problems that you will encounter. If you're using PowerShell Azure Functions, they load the modules at runtime. Guess what? You don't have the modules loaded yet because of this feature, so you will get a lot of issues if you forget that you enabled this feature.

My solution to this problem? I don't enable Artifact Streaming for images that contain PowerShell modules. Is this great? No, I would have loved to have it fully on, but that's how it is. In my tests, only PowerShell has a problem with this feature, and I've tested .NET, Go, Python, TypeScript and a bit of Rust as a side project.

What This Means for Container Design

Artifact Streaming is somewhat of a game-changer for container image design. In the past, we spent a lot of energy fine-tuning image sizes with multi-stage builds, distroless bases, and layer management to cope with the need to pull the whole image before starting. Now, we can prioritize building container images that are both maintainable and complete without being overly stressed about every byte affecting startup times. That said, keeping images small is still a good idea, but the downside of a slightly larger image is no longer as harsh. I’m confident this technology will soon support all container types and become a standard feature across Kubernetes distributions, given its significant performance benefits.

Artifact Streaming solves the dreaded image pull time, one of the most persistent pain points in Kubernetes deployments. Streaming only what's needed for startup while fetching the rest in the background enables significantly faster pod startup times and more efficient scaling.

I recommend exploring this feature if you're running workloads on AKS. The setup is straightforward, and the performance benefits are immediate and substantial. Keep an eye out for those on other platforms - this approach is likely to spread, given its clear advantages.

That being said, have a great one!