构建fluentd镜像将收集的日志发送到elasticsearch

绝地灬酷狼 2023-06-09 14:22 13阅读 0赞

构建脚本：

version: "3"
    services:
      fluentd:
        build: ./fluentd
        volumes:
          - ./fluentd/conf:/fluentd/etc
        privileged: true
        ports:
          - "24224:24224"
          - "24224:24224/udp"
        environment:
          - TZ=Asia/Shanghai
        restart: always
        logging:
            driver: "json-file"
            options:
                max-size: 100m
                max-file: "5"

构建用到的Dockerfile

FROM fluent/fluentd
    RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-rdoc", "--no-ri", "--version", "1.9.5"]

conf目录涉及到的配置文件

<source>
      @type forward
      port 24224
      bind 0.0.0.0
    </source>
    <filter nginx>
      @type parser
      key_name log
      <parse>
            @type regexp
            expression (?<remote>[^ ]*) (?<user>[^ ]*) \[(?<localTime>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^\"]*?)(?: +\S*)?)?" (?<code>[^ ]*) (?<size>[^ ]*) (?<requestTime>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)"(?:\s+(?<http_x_forwarded_for>[^ ]+))?)?
            time_format %d/%b/%Y:%H:%M:%S %z
      </parse>
    </filter>
    <match nginx>
      @type copy
      <store>
        @type elasticsearch
        host 172.21.48.48
        port 9200
        logstash_format true
        logstash_prefix nginx
        logstash_dateformat %Y%m%d
        include_tag_key true
        type_name access_log
        flush_interval 1s
        include_tag_key true
        tag_key @log
      </store>
      <store>
        @type stdout
      </store>
    </match>

指定docker-compose  build命令即可。