Skip to main content

John Evdemon

A Comparison of Azure's Container Services

7 min read

Posting some opinions and info here.  Please consult the official Azure docs on containers if you have any questions.

 

What to Use When

Use Case

Container Offering

Bursting Workloads

ACI, Azure Batch

On-Demand Workloads

ACI, Azure Functions

Dev/Test/PoC Workloads

ACI, Standalone VM

24/7 Prod Workloads

AKS, Azure Web App for Containers

Web Hosting

Azure Web App for Containers, AKS

Prod Orchestration

AKS, ACS Engine

Custom Orchestration

ACS Engine

 

Azure Container Service Offerings

- Azure Container Instances (ACI)

  • "containers as a service" - billed only for the time the container is active
  • Excellent for bursty load types or tasks that run on a set schedule
  • use the portal or CLI to create a container
  • no need to worry about provisioning VMs to host the container or installing software.
  • ACI instances can be created with a single line CLI or PowerShell command and are up and running in minutes.
  • If you need multiple containers ACI supports container groups
    • Container groups enable numerous containers on the same host and local network.
  • ACI containers can be configured with CPU and RAM to meet your requirements and support Linux and Windows containers.
  • It is not yet possible to attach ACI instances to an Azure virtual network.
  • Running an ACI instance 24/7 will be slightly more expensive than running a VM of the same size.

 

- Azure Web Apps for Containers

  • you are billed whether the webapp/container is active or not
  • no container orchestration or discovery support (e.g. no "mesh")
  • use a custom image to create a container for your Web App
    • can only deploy one image to an AppService
  • can scale to multiple instances but each instance uses the same image
  •  containers pull their images from the Azure Container Registry (ACR)
  • create a new web app and configure it to point to your container registry.
  • You pay the standard price for the size of web applications you select - no additional cost for using containers.

 

- Azure Functions

  • you are billed only when the Function is invoked
  • Limit: 10 minute maximum execution time for consumption based functions.
  • With Azure functions 2.0 and Azure functions on Linux we can use containers
    • Limited to Linux only
    • Need to use the microsoft/azure-functions-runtime as the base image for the container

 

- Azure Batch

  • Solution for large-scale parallel batch processing.
  • Encapsulates all  application files inside of a container image.
  • Deploys batch VMs but the VMs are running an OS version with container support
    • your image is loaded onto the container
    • container has access to the full resources of the VM it is running on (one container per VM)
  • Can pre-fetch a container image for your batch pool to reduce deployment times.

 

Orchestrators - used to control multiple containers

- ACS (Azure Container Service - being replaced by AKS)

  • NOTE: ACS will retire on January 31, 2020.  See here for guidance on moving from ACS to AKS.
  • create a container orchestration cluster running Docker Sware, Kubernetes or Mesos.
  • this is IaaS - you are  responsible for managing the VM’s, applying updates etc.
  • ACS is deployed like a PaaS service, but what gets implemented is a set of IaaS infrastructure (vNet, NSG’s, virtual machines etc.)
    • pre-configured to run your selected orchestrator

 

- AKS (Azure Kubernetes Service - think of it as managed K8S as a Service)

  • semi-managed Kubernetes service and is the replacement for ACS.
  • No cluster-level SLA
    • Azure provides an SLA only for the underlying machines
  • Unlike ACS, only the K8S orchestrator is supported
  • Only for stateless services - state must be stored externally (CosmosDB, SQL DB, file system, etc)
  • While ACS deployed ALL IaaS infrastructure, AKS only offers part of the service as a managed PaaS offering
    • Kuberntes master nodes are hidden from you and are operated by Microsoft
    • the only VM resources you see are the worker nodes
    • worker nodes are IaaS and need to be managed and updated by you
    • you only pay for the worker nodes - the managed part is free
  • No Windows Container support - use ACS for this
  • intended for deploying Microservices
  • AKS is committed to full upstream Kubernetes parity, which means you benefit from the community and ecosystem around Kubernetes.
    • While Microsoft supports AKS, if you have issues with Kubernetes you will have to rely on the community for help.
  • Kubernetes is open source

 

- Azure Container Service Engine (ACSE)

  • Open sourced engine that ACS and AKS are built on top of
  • uses JSON files to detail the configuration of your orchestrator
  • ACS Engine generates ARM templates which you can deploy to Azure
  •  engine is a container orchestrator running on IaaS infrastructure that is all customer managed (no PaaS service here).
  •  You can use ACS engine to create clusters that aren’t currently supported in AKS
    • like Windows containers, or mixed Windows and Linux clusters, later or beta versions of Orchestrators,
    • support for Orchestrators other than Kubernetes etc.
  • enables more control over the orchestrator cluster that gets created
  • intended for deploying Microservices

 

- Service Fabric

  • SF is an app server capable of hosting and orchestrating containers
  • Azure itself runs on Service Fabric.  Service Fabric is the "productization" of Azure's hosting fabric.
    • SF is a more mature platform than Kubernetes
  • No cluster-level SLA
    • Azure provides an SLA only for the underlying machines
  •  Stateful services are fully supported using the virtual actor pattern
    • "actors" are shared-nothing units of computation that can maintain their own state
    • Other orchestrators only support stateless  services (state is stored externally via CosmosDB, SQL DB, cache, file system, etc)
  • Billing is hourly for all the VMs, storage and network resources that are used by SF
  • best for legacy .NET Windows applications
  • Service Fabric is open source

 

- Service Fabric Mesh 

  • SF Mesh is not Service Fabric:
    • SF Mesh is a multi-tenant service for hosting containers.  SF is an app server capable of hosting and orchestrating containers.
    • All deployments to SF Mesh must use containers
    • SF Mesh cannot host Ser­vice Fab­ric Ap­pli­ca­tions (e.g. native SF apps that don't use containers)
  • Capabilities:
    • deploy/operate containerized apps without having to manage VMs, storage or networking
    • you do not manage or see the VMs or the cluster
    • You focus on your application and its associated resources (containers, network, routes etc)
    • Limitations:
      • only available in US West, US East, and Europe West
      • Number of applications: 5
      • Number of cores per application: 12
      • Total RAM per application: 48 GB
      • Number of network and ingress end points: 5
      • Number of Azure volumes that you can attach: 10
      • Number of service replicas: 3
      • Largest container you can deploy is limited to 4 cores, 16-GB RAM.
  • supports both Windows and Linux containers
  • SF Mesh is not open source

 

- AKS and SF will continue to converge in orchestration capability

  • AKS will most likely never have the stateful capabilities that Service Fabric provides

 

Related services

- Azure Container Registry (ACR)

  • Private container registry.
  • Create a registry and have AKS connect to it using Azure Role Based Authentication.
  • ACR supports geo-replication - provides an easy way to distribute images globally.

 

- Azure Files and Managed Disks

  • Tools for persisting data for containers.
    • ACI can only use Azure Files, not Managed Disks
    • AKS and ACS can use Azure Files and Managed Disks
  • Managed Disks enable better performance for container storage