Skip to content

Fluentd Config Schema

Note

This is auto-generated documentation from a JSON schema that is under construction, this will improve over time.

Return to the root config schema

https://raw.githubusercontent.com/elastisys/compliantkubernetes-apps/main/config/schemas/config.yaml#/properties/fluentd

Configuration for Fluentd.

Fluentd automatically collects logs from all containers running in the environment.

In the service cluster audit, application, and platform logs can be shipped to object storage. In the workload cluster audit logs can be shipped to object storage and application and platform logs to OpenSearch running in the service cluster.

Logs are collected using a daemon set, and in the workload cluster two sets are deployed, one for the system nodes and one for the worker nodes. Application developer can modify two ConfigMaps to add additional configuration and plugins to the set running on the worker nodes.

When logs are shipped to object storage a stateful aggregator is deployed that buffers logs with persistence before they are shipped. When logs are shipped to OpenSearch it is done directly from the forwarder daemons.

Shipping audit and service cluster logs requires that objectStorage is configured, and will use the bucket or container set in objectStorage.buckets.audit and objectStorage.buckets.scLogs respectively.

Note

Fluentd is installed in both service cluster and workload cluster, so this configuration applies there with some exceptions.

Abstract Extensible Status Identifiable Custom Properties Additional Properties Access Restrictions Defined In
Can be instantiated No Unknown status No Forbidden Forbidden none config/schemas/config.yaml*

TYPE:

object (Fluentd Config)

PROPERTIES:

Property Type Required Nullable Defined by
enabled boolean Optional cannot be null Welkin Apps Config
audit object Optional cannot be null Welkin Apps Config
scLogs object Optional cannot be null Welkin Apps Config
aggregator object Optional cannot be null Welkin Apps Config
forwarder object Optional cannot be null Welkin Apps Config
user object Optional cannot be null Welkin Apps Config
logManager object Optional cannot be null Welkin Apps Config
extraConfigMaps object Optional cannot be null Welkin Apps Config

enabled

enabled

TYPE:

boolean (Fluentd Enabled)

DEFAULTS:

The default value is:

true

audit

Configure Fluentd audit log collection.

audit

TYPE:

object (Fluentd Audit)

scLogs

Configure Fluentd service cluster log collection.

scLogs

TYPE:

object (Fluentd SC Logs)

aggregator

Configure Fluentd aggregator, used to buffer logs with persistence before they are shipped to object storage.

aggregator

TYPE:

object (Fluentd Aggregator)

forwarder

Configure Fluentd forwarder, used to collect and forward logs on system nodes.

forwarder

TYPE:

object (Fluentd Forwarder)

user

Configure Fluentd forwarder, used to collect and forward logs on worker nodes.

user

TYPE:

object (Fluentd Forwarder User)

logManager

Configure log-manager, used to manage compaction and retention of logs store in object storage.

logManager

TYPE:

object (Log Manager)

extraConfigMaps

Configure extra ConfigMaps for Fluentd.

Note

This is only applicable for Fluentd forwarder running on system nodes in the workload cluster.

extraConfigMaps

TYPE:

object (Fluentd Extra ConfigMaps)

Definitions

logManager

Reference this group by using

{"$ref":"https://raw.githubusercontent.com/elastisys/compliantkubernetes-apps/main/config/schemas/config.yaml#/properties/fluentd/$defs/logManager"}
Property Type Required Nullable Defined by
compaction object Optional cannot be null Welkin Apps Config
retention object Optional cannot be null Welkin Apps Config
jobEnabled boolean Optional cannot be null Welkin Apps Config
jobDays number Optional cannot be null Welkin Apps Config
jobSchedule string Optional cannot be null Welkin Apps Config
jobEphemeralVolumes object Optional cannot be null Welkin Apps Config

compaction

Configure the compaction of logs stored in object storage.

compaction

TYPE:

object (Log Manager Compaction)

retention

Configure the retention of logs stored in object storage.

retention

TYPE:

object (Log Manager Retention)

jobEnabled

jobEnabled

TYPE:

boolean (Log Manager Job Enabled)

DEFAULTS:

The default value is:

true

jobDays

Configure the days to consider for compaction or the days to retain.

jobDays

TYPE:

number (Log Manager Job Enabled)

jobSchedule

Schema for when to run Log Manager Job. Uses the Cron format, see https://en.wikipedia.org/wiki/Cron.

jobSchedule

TYPE:

string (Log Manager Job Schema)

CONSTRAINTS:

pattern: the string must match the following regular expression: 

^(((\*\/)?([0-5]?[0-9])((\,|\-|\/)([0-5]?[0-9]))*|\*)[^\S\r\n]+((\*\/)?((2[0-3]|1[0-9]|[0-9]|00))((\,|\-|\/)(2[0-3]|1[0-9]|[0-9]|00))*|\*)[^\S\r\n]+((\*\/)?([1-9]|[12][0-9]|3[01])((\,|\-|\/)([1-9]|[12][0-9]|3[01]))*|\*)[^\S\r\n]+((\*\/)?([1-9]|1[0-2])((\,|\-|\/)([1-9]|1[0-2]))*|\*|(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec))[^\S\r\n]+((\*\/)?[0-6]((\,|\-|\/)[0-6])*|\*|00|(sun|mon|tue|wed|thu|fri|sat)))$|^@(annually|yearly|monthly|weekly|daily|hourly|reboot)$

try pattern

jobEphemeralVolumes

Configure the job to run with an ephemeral volume if the nodes risk running out of storage.

jobEphemeralVolumes

TYPE:

object (Log Manager Job Ephemeral Volume)

Return to the root config schema

Generated Thu Dec 19 03:51:01 UTC 2024 from elastisys/compliantkubernetes-apps@main