This page describes how to install a Customer Application
You are solely responsible for Customer Applications.
If you are an Elastisys Managed Services customer, please review your responsibilities in ToS 5.2.
Specifically, you are responsible for performing due diligence for the project discussed in this page. At the very least, you must:
- assess project ownership, governance and licensing;
- assess project roadmap and future suitability;
- assess project compatibility with your use-case;
- assess business continuity, i.e., what will you do if the project is abandoned;
- subscribe to security advisories related to the project;
- apply security patches and updates, as needed;
- regularly test disaster recovery.
This page is a preview for self-managed cluster-wide resources
Compliant Kubernetes restricts application developers to manage CustomResourceDefinitions (CRDs) and other cluster-wide resources for security purposes. This means that some applications that require such cluster-wide resources are not possible to install for you as an application developer. As a trade-off, we are launching this preview feature that allows for self-management of specific cluster-wide resources required for certain popular applications. It is disabled by default, so please ask your platform administrator to enable this preview feature.
For Elastisys Managed Services Customers
Flux is an open-source tool for continuous delivery (CD) and GitOps in Kubernetes. It allows you to automate and manage the deployment of applications and configurations in a Kubernetes cluster using a Git repository as the source of truth.
Flux is a CNCF Graduated project.
This page will help you install Flux in a Compliant Kubernetes environment.
This installation guide has been tested with Flux version 2.1.2.
This guide depends on the self-managed cluster resources feature to be enabled. This is so Flux gets the necessary CRDs and ClusterRoles installed.
Flux also requires the image repository
ghcr.io/fluxcd to be allowlisted. Ask your platform administrator to do this while enabling the self-managed cluster resources feature.
You need to setup a Git repository that will contain the manifest files. This can be a personal or organization repository. It is strongly recommended that it is a private (as in: not public) repository.
Next you need to generate an SSH key that will be used to communicate with the Git repository. The private key will be used as a Kubernetes Secret in a later step.
ssh-keygen -t rsa -C "flux-deploymentkey" -f <path-to-store-key>
Deploy keys. Copy your public key that you just generated and paste here.
In Kubernetes you will need to:
Create a Namespace for Flux
Create a Git Secret in the Namespace
Create Roles/RoleBindings for Flux.
You need to apply the Custom Resource Definitions (CRDs) required by Flux. This is typically not allowed in a Compliant Kubernetes Environment, but with Flux enabled with the self-managed cluster resources feature, this allows you to apply these yourself.
mkdir crds # Fetches Flux CRDs for v2.1.2 and saves it in the crds directory curl https://raw.githubusercontent.com/fluxcd/flux2/v2.1.2/manifests/crds/kustomization.yaml > crds/kustomization.yaml kubectl apply -k crds
You need create a Namespace where Flux will work. This Namespace should be called
flux-system. Create this sub-namespace under eg.
kubectl hns create -n production flux-system
Next to allow Flux to interact with your Git Repository you need to create a Secret containing the ssh private key created earlier. This can be done with the Flux CLI:
flux create secret git <repo-name>-auth \ --url=ssh://email@example.com/<owner>/<repo-name>.git \ --private-key-file=<path-to-ssh-private-key>
Roles and RoleBindings¶
You need to create the necessary Roles for Flux to function. This needs to be done in every Namespace that you want Flux to work in.
Since Compliant Kubernetes uses the Hierarchical Namespace Controller, the easiest way to achieve this is to place the Roles and RoleBindings in the parent namespace where
flux-system was created from. By doing so, all Namespaces created under the same parent Namespace will inherit the Roles and RoleBindings.
If you have multiple Namespaces that ought to be targets for Flux, you can add the Roles and RoleBindings to more than one "parent" Namespace. For instance, to
staging, to get Flux to work with the
staging Namespace and any Namespace anchored to it.
mkdir roles # Fetches the necessary Roles and saves it in the roles directory curl https://raw.githubusercontent.com/elastisys/compliantkubernetes/main/docs/user-guide/self-managed-services/flux-files/roles/all-controllers-role.yaml > roles/all-controllers-role.yaml curl https://raw.githubusercontent.com/elastisys/compliantkubernetes/main/docs/user-guide/self-managed-services/flux-files/roles/source-controller-role.yaml > roles/source-controller-role.yaml curl https://raw.githubusercontent.com/elastisys/compliantkubernetes/main/docs/user-guide/self-managed-services/flux-files/roles/kustomization.yaml > roles/kustomization.yaml # If you installed Flux in another Namespace other than production, edit the Namespace in roles/kustomization.yaml kubectl apply -k roles
The kustomize and helm controller needs some extra permissions as well since it wants to deploy. The simplest is to add these controller ServiceAccounts to the
extra-workload-admins RoleBinding in the parent Namespace eg.
production. This will grant Flux the maximum permission an application developer can give in the Namespaces where it is configured. Edit the RoleBinding and add the lines below.
kubectl edit rolebindings extra-workload-admins -n production ... roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: admin # Add these lines below subjects: - kind: ServiceAccount name: kustomize-controller namespace: flux-system - kind: ServiceAccount name: helm-controller namespace: flux-system
Installing Flux with
flux bootstrap command does not work when installing in a Compliant Kubernetes environment, please follow our instructions instead.
The script below can be used to generate Flux manifests and a basic cluster folder structure similar to
flux bootstrap. Be sure to configure the environment variables in the script.
# Generate Manifest files # Be sure to edit the variables below! CLUSTER_NAME="<cluster-name>" REPO_NAME="<repo-name>" URL="github.com/<owner>/<repo-name>" mkdir -p clusters/$CLUSTER_NAME/flux-system touch clusters/$CLUSTER_NAME/flux-system/gotk-components.yaml \ clusters/$CLUSTER_NAME/flux-system/gotk-sync.yaml \ clusters/$CLUSTER_NAME/flux-system/kustomization.yaml # Generate flux manifests flux install --export > clusters/$CLUSTER_NAME/flux-system/gotk-components.yaml # Create repo manifest flux create source git $REPO_NAME-repo \ --url=ssh://git@"$URL" \ --branch=main \ --secret-ref=$REPO_NAME-auth \ --export > clusters/$CLUSTER_NAME/flux-system/gotk-sync.yaml # Create Kustomization manifest for Flux flux create kustomization $REPO_NAME-repo \ --namespace=flux-system \ --source=GitRepository/$REPO_NAME-repo.flux-system \ --path="./clusters/$CLUSTER_NAME/flux-system" \ --prune=true \ --interval=1m \ --export >> clusters/$CLUSTER_NAME/flux-system/gotk-sync.yaml # Fetches our patches and saves them in the clusters/$CLUSTER_NAME/flux-system directory curl https://raw.githubusercontent.com/elastisys/compliantkubernetes/main/docs/user-guide/self-managed-services/flux-files/kustomization.yaml > clusters/$CLUSTER_NAME/flux-system/kustomization.yaml
Commit and push the files to the repository.
If you created your SSH keys in the repository folder, make sure you do not push these to the repository.
Simply install by applying the kustomization.
kubectl apply -k clusters/$CLUSTER_NAME/flux-system
Now that it has been installed, you are ready to use Flux on Compliant Kubernetes!
Read the Further Reading, keep in mind the (updated) list of Known Issues, both of which are found below.
Role and RoleBindings does not apply correctly¶
Error from server (NotFound): error when creating "roles/": roles.rbac.authorization.k8s.io "role" not found
There is a known issue with Role and RoleBindings not being able to be applied together using Flux in a GitOps way. For example, if you apply a Role and a RoleBinding that uses the Role, then Flux will fail to apply. If the Role already exists in the cluster then Flux will succeed.
You can workaround this using the Flux Kustomization dependsOn functionality. By splitting the Roles and RoleBindings into separate folders and then creating two Kustomizations for them where the RoleBindings will depend on the Roles. Then the Roles will be applied before the RoleBindings and so the issue will not occur. Refer to the previous link for an example.
For security reasons, Elastisys has not enabled the multi-tenancy model described in Flux documentation. Doing so requires giving Flux implicit ClusterAdmin permissions using impersonation, and to then trust that its use of it is correct and secure.
When using Flux as a self-managed service, it is important to ensure that excessive permissions cannot be gained through exploiting privilege escalation with Flux.
The multitenancy feature flags can still be used by you, however, to, for example, deny cross namespace references.