Flux™ (self-managed)¶

Warning

This page provides a starting point for installing an application. The information is useful, but not kept up-to-date. If you struggle, contact Elastisys Consulting.

This page describes how to install a Customer Application

You are solely responsible for Customer Applications.

If you are an Elastisys Managed Services customer, please review your responsibilities in ToS 5.2.

Specifically, you are responsible for performing due diligence for the project discussed in this page. At the very least, you must:

assess project ownership, governance and licensing;
assess project roadmap and future suitability;
assess project compatibility with your use-case;
assess business continuity, i.e., what will you do if the project is abandoned;
subscribe to security advisories related to the project;
apply security patches and updates, as needed;
regularly test disaster recovery.

This page is a preview for self-managed cluster-wide resources

Welkin restricts Application Developers to manage CustomResourceDefinitions (CRDs) and other cluster-wide resources for security purposes. This means that some applications that require such cluster-wide resources are not possible to install for you as an Application Developer. As a trade-off, we are launching this preview feature that allows for self-management of specific cluster-wide resources required for certain popular applications. It is disabled by default, so please ask your Platform Administrator to enable this preview feature.

For Elastisys Managed Services Customers

You can ask for this feature to be enabled by filing a service ticket. This is a preview feature. For more information, please read ToS 9.1 Preview Features.

Flux is an open-source tool for continuous delivery (CD) and GitOps in Kubernetes. It allows you to automate and manage the deployment of applications and configurations in a Kubernetes cluster using a Git repository as the source of truth.

Flux is a CNCF Graduated project.

This page will help you install Flux in a Welkin environment.

Supported versions

This installation guide has been tested with Flux version 2.1.2.

Initial Prep¶

Dependencies¶

This guide depends on the self-managed cluster resources feature to be enabled. This is so Flux gets the necessary CRDs and ClusterRoles installed.

Flux also requires the image repository ghcr.io/fluxcd to be allowlisted. Ask your Platform Administrator to do this while enabling the self-managed cluster resources feature.

Git¶

You need to setup a Git repository that will contain the manifest files. This can be a personal or organization repository. It is strongly recommended that it is a private (as in: not public) repository.

Next you need to generate an SSH key that will be used to communicate with the Git repository. The private key will be used as a Kubernetes Secret in a later step.

ssh-keygen -t rsa -C "flux-deploymentkey" -f <path-to-store-key>

After you have generated an SSH Key, you want to add it as a deploy key in your Git repository. This can be done through Settings -> Deploy keys. Copy your public key that you just generated and paste here.

Kubernetes¶

In Kubernetes you will need to:

Install CRDs
Create a namespace for Flux
Create a Git Secret in the namespace
Create Roles/RoleBindings for Flux.

CRDs¶

You need to apply the Custom Resource Definitions (CRDs) required by Flux. This is typically not allowed in a Welkin Environment, but with Flux enabled with the self-managed cluster resources feature, this allows you to apply these yourself.

mkdir crds

# Fetches Flux CRDs for v2.1.2 and saves it in the crds directory
curl https://raw.githubusercontent.com/fluxcd/flux2/v2.1.2/manifests/crds/kustomization.yaml > crds/kustomization.yaml

kubectl apply -k crds

Namespace¶

You need to create a namespace where Flux will work. This namespace should be called flux-system. Create this sub-namespace under e.g. production.

kubectl hns create -n production flux-system

Git Secret¶

Next to allow Flux to interact with your Git Repository you need to create a Secret containing the ssh private key created earlier. This can be done with the Flux CLI:

flux create secret git <repo-name>-auth \
    --url=ssh://git@github.com/<owner>/<repo-name>.git \
    --private-key-file=<path-to-ssh-private-key>

Roles and RoleBindings¶

You need to create the necessary Roles for Flux to function. This needs to be done in every namespace that you want Flux to work in.

Since Welkin uses the Hierarchical Namespace Controller, the easiest way to achieve this is to place the Roles and RoleBindings in the parent namespace that flux-system was created from. By doing so, all namespaces created under the same parent namespace will inherit the Roles and RoleBindings.

If you have multiple namespaces that ought to be targets for Flux, you can add the Roles and RoleBindings to more than one "parent" namespace. For instance, to staging, to get Flux to work with the staging namespace and any namespace anchored to it.

mkdir roles

# Fetches the necessary Roles and saves it in the roles directory
curl https://raw.githubusercontent.com/elastisys/welkin/main/docs/user-guide/self-managed-services/flux-files/roles/all-controllers-role.yaml > roles/all-controllers-role.yaml
curl https://raw.githubusercontent.com/elastisys/welkin/main/docs/user-guide/self-managed-services/flux-files/roles/source-controller-role.yaml > roles/source-controller-role.yaml
curl https://raw.githubusercontent.com/elastisys/welkin/main/docs/user-guide/self-managed-services/flux-files/roles/kustomization.yaml > roles/kustomization.yaml

# If you installed Flux in another namespace other than production, edit the namespace in roles/kustomization.yaml

kubectl apply -k roles

The kustomize and helm controller needs some extra permissions as well since it wants to deploy. The simplest is to add these controller ServiceAccounts to the extra-workload-admins RoleBinding in the parent namespace eg. production. This will grant Flux the maximum permission an Application Developer can give in the namespaces where it is configured. Edit the RoleBinding and add the lines below.

kubectl edit rolebindings extra-workload-admins -n production

...
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
# Add these lines below
subjects:
- kind: ServiceAccount
  name: kustomize-controller
  namespace: flux-system
- kind: ServiceAccount
  name: helm-controller
  namespace: flux-system

Setup¶

Generate Manifests¶

Note

Installing Flux with flux bootstrap command does not work when installing in a Welkin environment, please follow our instructions instead.

The script below can be used to generate Flux manifests and a basic cluster folder structure similar to flux bootstrap. Be sure to configure the environment variables in the script.

# Generate Manifest files

# Be sure to edit the variables below!
CLUSTER_NAME="<cluster-name>"
REPO_NAME="<repo-name>"
URL="github.com/<owner>/<repo-name>"

mkdir -p clusters/$CLUSTER_NAME/flux-system
touch clusters/$CLUSTER_NAME/flux-system/gotk-components.yaml \
    clusters/$CLUSTER_NAME/flux-system/gotk-sync.yaml \
    clusters/$CLUSTER_NAME/flux-system/kustomization.yaml

# Generate Flux manifests
flux install --export > clusters/$CLUSTER_NAME/flux-system/gotk-components.yaml

# Create repo manifest
flux create source git $REPO_NAME-repo \
    --url=ssh://git@"$URL" \
    --branch=main \
    --secret-ref=$REPO_NAME-auth \
    --export > clusters/$CLUSTER_NAME/flux-system/gotk-sync.yaml

# Create Kustomization manifest for Flux
flux create kustomization $REPO_NAME-repo \
    --namespace=flux-system \
    --source=GitRepository/$REPO_NAME-repo.flux-system \
    --path="./clusters/$CLUSTER_NAME/flux-system" \
    --prune=true \
    --interval=1m \
    --export >> clusters/$CLUSTER_NAME/flux-system/gotk-sync.yaml

# Fetches our patches and saves them in the clusters/$CLUSTER_NAME/flux-system directory
curl https://raw.githubusercontent.com/elastisys/welkin/main/docs/user-guide/self-managed-services/flux-files/kustomization.yaml > clusters/$CLUSTER_NAME/flux-system/kustomization.yaml

Commit and push the files to the repository.

Warning

If you created your SSH keys in the repository folder, make sure you do not push these to the repository.

Install¶

Simply install by applying the kustomization.

kubectl apply -k clusters/$CLUSTER_NAME/flux-system

Now that it has been installed, you are ready to use Flux on Welkin!

Read the Further Reading, keep in mind the (updated) list of Known Issues, both of which are found below.

Known Issues¶

Role and RoleBindings does not apply correctly¶

Error produced: Error from server (NotFound): error when creating "roles/": roles.rbac.authorization.k8s.io "role" not found

There is a known issue with Role and RoleBindings not being able to be applied together using Flux in a GitOps way. For example, if you apply a Role and a RoleBinding that uses the Role, then Flux will fail to apply. If the Role already exists in the cluster then Flux will succeed.

Flux uses the server-side apply, which requires the ‘bind’ permission to properly apply RoleBindings. And we cannot give you this due to privilege escalation issues with this permission.

You can workaround this using the Flux Kustomization dependsOn functionality. By splitting the Roles and RoleBindings into separate folders and then creating two Kustomizations for them where the RoleBindings will depend on the Roles. Then the Roles will be applied before the RoleBindings and so the issue will not occur. Refer to the previous link for an example.

Notes¶

For security reasons, Elastisys has not enabled the multi-tenancy model described in Flux documentation. Doing so requires giving Flux implicit ClusterAdmin permissions using impersonation, and to then trust that its use of it is correct and secure.

When using Flux as a self-managed service, it is important to ensure that excessive permissions cannot be gained through exploiting privilege escalation with Flux.

The multitenancy feature flags can still be used by you, however, to, for example, deny cross namespace references.