Skip to content

Welkin Infrastructure Requirements

Note

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

This page describes infrastructure capabilities and a starting layout for running Welkin in production. The layout is one of several possible setups, designed for redundancy and fault tolerance in a cloud setting. It is not a universal requirement for the number or size of physical servers.

How much infrastructure capacity does Welkin need?

Starting capacity for cloud-oriented setups

We found the following cloud-oriented layout to be a good starting point. The counts describe Kubernetes Nodes, which are typically run on VMs:

  • Management Cluster:
    • 3x 2 CPUs, 4 GB RAM, 60 GB local disk for control plane Nodes
    • 5x 2 CPUs, 8 GB RAM, 80 GB local disk for worker Nodes
  • Workload Cluster:
    • 3x 2 CPUs, 4 GB RAM, 60 GB local disk for control plane Nodes
    • 2x 2 CPUs, 6 GB RAM, 60 GB local disk for Welkin worker Nodes
    • Add additional capacity as required by your application and Additional Managed Services.

The required capacity depends on the performance of your infrastructure. This, in turn, is strongly influenced by the CPU model, memory bandwidth, network bandwidth, disk bandwidth, etc. that you invested in. Adjust this starting capacity to your application load and Additional Managed Services. Also include the resources used by platform components, such as logging and monitoring, and headroom for failures and maintenance.

For Welkin Managed, Elastisys uses the Capacity Management process to monitor and adjust capacity over time. Welkin Enterprise administrators can use the same process for their Environments.

Welkin Managed and Welkin Enterprise

Welkin Managed provides cloud-based Environments with an architecture that prioritizes redundancy and fault tolerance so that we at Elastisys can operate the managed service securely. As a result, Welkin Managed customers cannot choose an alternative layout that consolidates components onto fewer servers. We adjust capacity within this architecture as needed.

Welkin Enterprise customers can choose layouts suited to their infrastructure, including bare-metal servers and on-premise infrastructure. For example, larger servers can host more components, reducing the number of physical servers needed. The chosen layout must still provide sufficient capacity, isolation and availability for the Management Cluster, Workload Cluster(s) and applications.

Welkin Enterprise capacity planning

For Welkin Enterprise capacity planning, the starting point should be to ensure that at the very least, a comparable amount of resources are available as in the cloud-oriented setup. However, because servers can be significantly larger and have hundreds of gigabytes of RAM, it may also be sensible to co-locate e.g. all the Management Cluster functionality on a lower number of servers, to make better use of resources. It may also be sensible to run workloads on top of control plane Nodes, depending on the setup.

Please see the section below about the trade-offs when using fewer Nodes or servers.

Trade-offs when using fewer Nodes or servers

Consolidating components can make better use of larger servers, but equal CPU and memory totals do not guarantee equal availability. Consider the following before choosing a layout:

  • Failure impact and maintenance capacity: Each server failure can affect more components when they share a server. Ensure surviving servers have enough capacity to run the affected workloads during a failure or maintenance. Multiple VMs on one physical server share that server's failure risk.
  • Control-plane availability: Preserve the quorum needed by etcd and distribute control-plane components across independent failure domains according to your availability goals. Co-location can cause several quorum members to fail together. See Kubernetes high availability topologies.
  • Pod Disruption Budgets: A PodDisruptionBudget limits voluntary disruptions, such as evictions during a Node drain. With fewer eligible Nodes, replacement Pods may have nowhere to run, and a drain can be blocked to preserve availability. A budget does not prevent involuntary disruptions such as a server failure.
  • Pod placement: Pod Topology Spread Constraints and Pod anti-affinity can require replicas to run on separate Nodes or Zones. With too few eligible Nodes or topology domains, Pods can remain Pending even when CPU and memory are available. Review scheduling constraints with Application Developers and ensure replicas are spread across independent failure domains.

Before going live with production workloads

Welkin strongly recommend completing the Go-live Checklist as a joint effort between Platform Administrators together with Application Developers before production. Use its load, rolling Node reboot, failure and recovery tests to verify that the chosen layout meets your capacity and availability goals. Elastisys does this together with its Welkin Managed customers on the Premium plan. Standard plan customers can contact Elastisys to have a Go-live Checklist carried out on a time and material basis.

The Role of Infrastructure Provider Audits

Elastisys recommends that a provider audit is carried out when Welkin is to be installed on a new infrastructure. For Welkin Managed, Elastisys has carried out the audit with its cloud partners, and continuously keeps up to date with changes by these partners. For Welkin Enterprise, Elastisys supports customers in carrying out these toward their own infrastructure.

The audit checks that the infrastructure meets the fundamental conditions for running Welkin. Its findings inform the capacity planning and configuration needed for secure and stable operations on that infrastructure.

What infrastructure capabilities do I need?

For Welkin Enterprise Customers

This section is about Welkin Enterprise, as all these matters are taken care of for you by Elastisys in Welkin Managed.

This depends a lot on your goals. For example:

  • If you want to tolerate one data center failure, then you MUST have three data centers, due to Kubernetes quorum requirements (further reading on etcd).
  • If you want to tolerate one Node failure, then you MUST have a load-balancer with health-checks which can route traffic away from the failed Node.

Ideally, your infrastructure SHOULD provide:

  • VMs or physical servers running Ubuntu 24.04 LTS
  • Firewall to protect the VMs
  • Two private networks, one for the Management Cluster and one for the Workload Cluster
  • Two load-balancers
  • CSI-compatible block storage
  • S3-compatible object storage
  • An OpenTofu or Terraform Provider to configure all above via an API

Furthermore, your infrastructure SHOULD not block:

  • Internal traffic between the VMs
  • Access to the Swedish NTP servers (outgoing packets on UDP port 123).

Note that the Swedish NTP servers are mandated by MSBFS 2020:7 "Myndigheten för samhällsskydd och beredskaps föreskrifter om säkerhetsåtgärder i informationssystem för statliga myndigheter" 3 kap § 13.

  • Access to Let's Encrypt (outgoing and incoming packets on TCP port 80)

For initial setup, platform administrators need SSH access to the VMs. This can be done in a few ways:

  • Expose the VMs on the public Internet and allowlist the platform administrators' VPNs.
  • Expose a Bastion Host running Ubuntu on the public Internet and allowlist the platform administrators' VPNs.
  • Provide the platform administrators with a Linux-compatible VPN client to access the VMs via SSH.

What if I lack ...?

  • ... a policy which allows SSH access to the VMs: Platform administrators needs to be empowered to use the tools which make them most efficient. Doing initial setup via "desktop share", such as VNC, Remote Desktop or Citrix DaaS, is tedious and error-prone. Let us help you by reading your Exception Policy, fill out your Exception Request Form and provide you with detailed Compensating Controls to mitigate risk associated with the exception.
  • ... an API to configure my infrastructure (making the use of Infrastructure as Code impossible): Platform Administrators will communicate with your infrastructure team via email or service tickets. In fact, we recommend that initial setup includes a detailed architecture diagram of the infrastructure, both to reduce misunderstandings, but also to serve as documentation, as required by many security standards.
  • ... access to the Swedish NTP servers: Welkin can be configured with alternative NTP servers. This will ensure you conform with clock synchronization requirements.
  • ... access to LetsEncrypt: This page discusses alternative strategy for certificate provisioning. Note that this may reduce automation, hence is more human time intensive and error prone.
  • ... an S3-compatible object storage: Consider using a public cloud-based fully-managed S3-compatible object storage.
  • ... CSI-compatible block storage: If you have Rook/Ceph experts, then you can set up a Rook/Ceph Cluster. Note that this will increase infrastructure footprint and the skills needed for troubleshooting from your team. Always prefer using your existing block storage solution, if one exists.
  • ... load-balancers: If Welkin may speak BGP with your routers, then you may use kube-vip. If not, then this will make it significantly harder to tolerate Node failures. We could potentially compensate by using DNS with health-checks, as provided by AWS Route53, however fail-over times become a lot larger, e.g., up to 5 minutes. If DNS failover is unsuitable for you, then we recommend a manual failover process.
  • ... private networks: You can use public IP addresses for the VMs and protect them using your firewall.
  • ... three data centers: To tolerate one data center failure, you need three data centres. This requirement is due to Kubernetes needing a quorum. The Additional Managed Services also benefit from multiple data centers for tolerating failure. You also need to have a load-balancing service which is "stretched" across all data centers. This means that if one data center goes down, your load balancer needs to be able to direct traffic to the healthy data centers. This is usually achieved via BGP Anycast.
  • ... Internet access (or only intermittent access): See air-gapped.