Skip to content

[Superseded by ADR-0046] Let upstream projects handle CRDs

  • Status: superseded by ADR-0046
  • Deciders: Arch Meeting
  • Date: 2021-04-29

Technical Story: #446 #369 #391 #402 #436.

Context and Problem Statement

CustomResourceDefinitions (CRDs) are tricky. They are essentially a mechanism to change the API of Kubernetes. Helm 2 had zero support for CRDs. Helm 3 has support for installing CRDs, but not upgrading them.

How should we handle CRDs?

Decision Drivers

  • CRDs add complexity and need to be treated specially.
  • Generally need to “trim fat” and rely on upstream.

Considered Options

  • Install and upgrade CRDs as part of the bootstrap step, which is a Helm 2 legacy.
  • Rely on whatever mechanism is proposed by upstream Helm Charts.

Decision Outcome

Chosen option: "Rely on upstream", because it trims fat and reduces astonishment.

At installation, rely on upstream approach to install CRDs (see below). At upgrade, propagate upstream migration steps in Welkin migration steps in each release notes. An issue template was created to ensure we won't forget.

Since we "vendor in" all Charts, CRDs can be discovered using:

grep -R 'kind: CustomResourceDefinition'

Positive Consequences

  • Less astonishing, compared to installing Chart "by hand".
  • Less maintenance, i.e., there is only one source of truth for CRDs.

Negative Consequences

  • None really.

Detailed Audit

A detailed audit was performed of all CRDs in Welkin on 2021-04-27.

As a summary, all projects encourage installing CRDs as part of standard helm install. Most projects encourage following manual migration steps to handle CRDs. Some projects handle CRD upgrades.

A detailed analysis is listed below:

cert-manager

  • Installation: The cert-manager Helm Chart includes the installCRDs value -- by default it is set to false. If set to true, then CRDs are automatically installed when installing cert-manager, albeit not using the CRDs mechanism provided by Helm.
  • Upgrade: CRDs are supposed to be upgraded manually.

Dex

Dex can be configured without CRDs. ADR-0012 argues for that approach.

gatekeeper

Prometheus (kube-prometheus-stack)

Velero