Fluentd Config Schema¶

Note

This is auto-generated documentation from a JSON schema that is under construction, this will improve over time.

https://raw.githubusercontent.com/elastisys/compliantkubernetes-apps/main/config/schemas/config.yaml#/properties/fluentd

Configuration for Fluentd.

Fluentd automatically collects logs from all containers running in the environment.

In the service cluster audit, application, and platform logs can be shipped to object storage. In the workload cluster audit logs can be shipped to object storage and application and platform logs to OpenSearch running in the service cluster.

Logs are collected using a daemon set, and in the workload cluster two sets are deployed, one for the system nodes and one for the worker nodes. Application developer can modify two ConfigMaps to add additional configuration and plugins to the set running on the worker nodes.

When logs are shipped to object storage a stateful aggregator is deployed that buffers logs with persistence before they are shipped. When logs are shipped to OpenSearch it is done directly from the forwarder daemons.

Shipping audit and service cluster logs requires that objectStorage is configured, and will use the bucket or container set in objectStorage.buckets.audit and objectStorage.buckets.scLogs respectively.

Note

Fluentd is installed in both service cluster and workload cluster, so this configuration applies there with some exceptions.

Abstract	Extensible	Status	Identifiable	Custom Properties	Additional Properties	Access Restrictions	Defined In
Can be instantiated	No	Unknown status	No	Forbidden	Forbidden	none	config/schemas/config.yaml*

TYPE:

object (Fluentd Config)

PROPERTIES:

Property	Type	Required	Nullable	Defined by
enabled	`boolean`	Optional	cannot be null	Compliant Kubernetes Apps Config
audit	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
scLogs	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
aggregator	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
forwarder	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
user	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
logManager	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
extraConfigMaps	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config

enabled¶

enabled

is optional
Type: boolean (Fluentd Enabled)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

boolean (Fluentd Enabled)

DEFAULTS:

The default value is:

true

audit¶

Configure Fluentd audit log collection.

audit

is optional
Type: object (Fluentd Audit)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Fluentd Audit)

scLogs¶

Configure Fluentd service cluster log collection.

scLogs

is optional
Type: object (Fluentd SC Logs)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Fluentd SC Logs)

aggregator¶

Configure Fluentd aggregator, used to buffer logs with persistence before they are shipped to object storage.

aggregator

is optional
Type: object (Fluentd Aggregator)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Fluentd Aggregator)

forwarder¶

Configure Fluentd forwarder, used to collect and forward logs on system nodes.

forwarder

is optional
Type: object (Fluentd Forwarder)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Fluentd Forwarder)

user¶

Configure Fluentd forwarder, used to collect and forward logs on worker nodes.

user

is optional
Type: object (Fluentd Forwarder User)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Fluentd Forwarder User)

logManager¶

Configure log-manager, used to manage compaction and retention of logs store in object storage.

logManager

is optional
Type: object (Log Manager)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Log Manager)

extraConfigMaps¶

Configure extra ConfigMaps for Fluentd.

Note

This is only applicable for Fluentd forwarder running on system nodes in the workload cluster.

extraConfigMaps

is optional
Type: object (Fluentd Extra ConfigMaps)
cannot be null
defined in: Compliant Kubernetes Apps Config

TYPE:

object (Fluentd Extra ConfigMaps)

Definitions¶

logManager¶

Reference this group by using

{"$ref":"https://raw.githubusercontent.com/elastisys/compliantkubernetes-apps/main/config/schemas/config.yaml#/properties/fluentd/$defs/logManager"}

Property	Type	Required	Nullable	Defined by
compaction	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
retention	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config
jobEnabled	`boolean`	Optional	cannot be null	Compliant Kubernetes Apps Config
jobDays	`number`	Optional	cannot be null	Compliant Kubernetes Apps Config
jobSchedule	`string`	Optional	cannot be null	Compliant Kubernetes Apps Config
jobEphemeralVolumes	`object`	Optional	cannot be null	Compliant Kubernetes Apps Config