Add an additional output for Fluentd

Add an additional output for Fluentd

If you have a syslog server and want StackLight LMA to send logs to this server, configure an additional output for Fluentd. In this case, Fluentd will push logs both to your syslog server and to Elasticsearch, which is the default target.

To add an additional output for Fluentd:

  1. Download and install the td-agent-additional-plugins package on every host that runs Fluentd:

    apt-get install --only-upgrade td-agent-additional-plugins
    
  2. Open your Git project repository with the Reclass model on the cluster level.

  3. In the classes/cluster/<cluster_name>/init.yml file, perform the following changes:

    1. Comment the system.fluentd.label.default_output.elasticsearch class.

    2. Copy the default_output parameters and rename to elasticsearch_output.

    3. To apply the existing filters for all outputs, copy the default output filter section to the new default output.

    4. Add syslog_output and specify the parameters as required.

    Example:

    classes:
    - system.fluentd
    - system.fluentd.label.default_metric
    - system.fluentd.label.default_metric.prometheus
    ## commented out
    #- system.fluentd.label.default_output.elasticsearch
    - system.fluentd.label.default_output.syslog
    parameters:
      fluentd:
        agent:
          plugin:
            fluent-plugin-remote_syslog:
              deb: ['td-agent-additional-plugins']
          config:
            label:
              ## renamed previous default_output -> elasticsearch_output
              elasticsearch_output:
                match:
                  elasticsearch_output:
                    tag: "**"
                    type: elasticsearch
                    host: ${_param:fluentd_elasticsearch_host}
                    port: ${_param:elasticsearch_port}
              syslog_output:
                match:
                  syslog_output:
                    tag: "**"
                    type: syslog
                    host: 127.0.0.1
                    port: 514
                    ## optional params:
                    # format: xxx
                    # severity: xxx
                    # facility: xxx
                    # protocol: xxx
                    # tls: xxx
                    # ca_file: xxx
                    # verify_mode: xxx
                    # packet_size: xxx
                    # timeout: xxx
                    # timeout_exception: xxx
                    # keep_alive: xxx
                    # keep_alive_idle: xxx
                    # keep_alive_cnt: xxx
                    # keep_alive_intvl: xxx
              default_output:
               ## copy of previous default_output filter section
                # filter: {}
                match:
                  send_to_default:
                    tag: "**"
                    type: copy
                    store:
                      - type: relabel
                        label: syslog_output
                      - type: relabel
                        label: elasticsearch_output
    
  4. Log in to the Salt Master node.

  5. Synchronize Salt modules and refresh Salt pillars:

    salt '*' saltutil.sync_all
    salt '*' saltutil.refresh_pillar
    
  6. Apply the following state:

    salt '*' state.sls fluentd