EFK/ELK搭建与测试

迷南。 2023-09-28 09:18 11阅读 0赞

EFK 是 Elasticsearch Filebeat Kibana 一套自动化收集日志的系统

更多资料见链接：[Elastic 中国社区官方博客][Elastic]

#### EFK/ELK搭建与测试 ####

*  EFK简介
 *  一.Elasticsearch
 *   *  1.安装
     *  2.访问
 *  二.Filebeat
 *   *  1.安装
 *  三.kibana
 *   *  1.安装
     *  2.访问

## EFK简介 ##

旧称：Elasticsearch Logstash Kibana  
全称：Elasticsearch Filebeat Kibana  
用途：自动化日志收集系统  
官网：[The Elastic Stack][]  
下载：[Elasticsearch][]  
[Filebeat][]  
[Kibana][]

## 一.Elasticsearch ##

### 1.安装 ###

# 1.解压并移动至安装目录
    tar zxvf elasticsearch-8.2.2-linux-x86_64.tar.gz
    mv elasticsearch-8.2.2 /usr/local/elasticsearch
    # 2.修改配置文件
    cd /usr/local/elasticsearch/
    vi config/elasticsearch.yml
    # 3.在配置文件末尾添加
    cluster.name: moon_cluster #集群名称
    node.name: node-1 #节点名称
    network.host: 0.0.0.0 #允许外网访问
    http.port: 9200 #服务端口
    cluster.initial_master_nodes: ["node-1"] #初始化新的集群时需要此配置来选举master
    # 4.启动（ES 不能用ROOT用户启动，新建用户）
    useradd appadmin
    chown -R appadmin:appadmin /usr/local/elasticsearch/*
    # 5.修改appadmin用户最大可创建文件数，最大虚拟内存大小
    vi /etc/security/limits.conf
    vi /etc/security/limits.d/20-nproc.conf
    # 6.在limits.conf 末尾追加以下信息并保存（用户打开最大文件数、最大进程数）
    appadmin soft nproc 4096
    appadmin hard nproc 4096
    appadmin soft nofile 65536
    appadmin hard nofile 65536
    #   在20-nproc.conf 末尾追加以下信息并保存
    appadmin soft nofile 65536
    appadmin hard nofile 65536
    # 7.修改最大虚拟内存
    vi /etc/sysctl.conf
    # 8.在sysctl.conf 末尾追加以下信息并保存
    vm.max_map_count=655360
    # 9.刷新
    sysctl -p
    # 10.切换用户并启动服务
    su appadmin
    bin/elasticsearch -d
    # 11.为kibana生成Token
    bin/elasticsearch-create-enrollment-token --scope kibana
    # 12.设置密码
    bin/elasticsearch-setup-passwords interactive

![在这里插入图片描述][68a7a16a89cb4563b754e2c1b30905ea.png]  
Token  
![在这里插入图片描述][8222358bbd8149bf9b21ed9b5a6063ab.png]  
密码

[appadmin@localhost elasticsearch]$ bin/elasticsearch-setup-passwords interactive
    warning: ignoring JAVA_HOME=/usr/local/java; using bundled JDK
    ******************************************************************************
    Note: The 'elasticsearch-setup-passwords' tool has been deprecated. This       command will be removed in a future release.
    ******************************************************************************
    
    Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
    You will be prompted to enter passwords as the process progresses.
    Please confirm that you would like to continue [y/N]y
    
    
    Enter password for [elastic]:
    Reenter password for [elastic]:
    Enter password for [apm_system]:
    Reenter password for [apm_system]:
    Enter password for [kibana_system]:
    Reenter password for [kibana_system]:
    Enter password for [logstash_system]:
    Reenter password for [logstash_system]:
    Enter password for [beats_system]:
    Reenter password for [beats_system]:
    Enter password for [remote_monitoring_user]:
    Reenter password for [remote_monitoring_user]:
    Changed password for user [apm_system]
    Changed password for user [kibana_system]
    Changed password for user [kibana]
    Changed password for user [logstash_system]
    Changed password for user [beats_system]
    Changed password for user [remote_monitoring_user]
    Changed password for user [elastic]
    [appadmin@localhost elasticsearch]$

![在这里插入图片描述][affc1d1361534cfabb225451d739d226.png]

### 2.访问 ###

# https://ip:9200/

![在这里插入图片描述][63cc435f80224fa7bb1b297f142f279e.png]

## 二.Filebeat ##

### 1.安装 ###

# 1.解压和移动
    tar zxvf filebeat-8.2.2-linux-x86_64.tar.gz
    mv filebeat-8.2.2-linux-x86_64 /usr/local/filebeat
    # 2.修改配置
    cd /usr/local/filebeat/
    vi filebeat.yml
    # 3.启动服务
    ./filebeat -e -c filebeat.yml >/tmp/filebeat.log 2>&1 &

配置过滤的文件

- type: filestream
    
      # Unique ID among all inputs, an ID is required.
      id: my-filestream-id
    
      # Change to true to enable this input configuration.
      enabled: false
    
      # Paths that should be crawled and fetched. Glob based paths.
      paths:
        - /var/log/*.log

配置输出到 Elsticsearch（用户名、密码）

output.elasticsearch:
      # Array of hosts to connect to.
      hosts: ["localhost:9200"]
    
      # Protocol - either `http` (default) or `https`.
      #protocol: "https"
    
      # Authentication credentials - either API key or username/password.
      #api_key: "id:api_key"
      username: "elastic"
      password: "changeme"

## 三.kibana ##

### 1.安装 ###

# 1.解压和移动
    tar zxvf kibana-8.2.2-linux-x86_64.tar.gz
    mv kibana-8.2.2 /usr/local/kibana
    # 2.修改配置信息
    cd /usr/local/kibana/
    vi config/kibana.yml
    # 3.在配置文件末尾添加
    server.port: 5601 #服务端口
    server.host: "0.0.0.0" #允许外网访问
    server.name: "moon_kibana" #服务名称
    elasticsearch.hosts: ["http://localhost:9200"] #对应elasticsearch服务地址
    # 4.以ROOT用户权限启动
    bin/kibana --allow-root >/tmp/kibana_start.log &
    # 5.查看输出信息
    cat /tmp/kibana_start.log
    # 6.获取认证码
    bin/kibana-verification-code

![在这里插入图片描述][a60049b99ecf45abb9a51134c5798fc1.png]

### 2.访问 ###

http://192.168.184.129:5601/

初始化  
![在这里插入图片描述][50021b9c7713434a8649279720e12206.png]  
![在这里插入图片描述][b4f120e5ed1b4aeaac1ccea244565e7d.png]  
登录  
![在这里插入图片描述][5fec6287f3584dbc82ebd9c68ca9bc63.png]

[Elastic]: https://elasticstack.blog.csdn.net/?type=blog
[The Elastic Stack]: https://www.elastic.co/cn/elastic-stack/
[Elasticsearch]: https://www.elastic.co/cn/downloads/elasticsearch
[Filebeat]: https://www.elastic.co/cn/downloads/beats/filebeat
[Kibana]: https://www.elastic.co/cn/downloads/kibana
[68a7a16a89cb4563b754e2c1b30905ea.png]: https://img-blog.csdnimg.cn/68a7a16a89cb4563b754e2c1b30905ea.png
[8222358bbd8149bf9b21ed9b5a6063ab.png]: https://img-blog.csdnimg.cn/8222358bbd8149bf9b21ed9b5a6063ab.png
[affc1d1361534cfabb225451d739d226.png]: https://img-blog.csdnimg.cn/affc1d1361534cfabb225451d739d226.png
[63cc435f80224fa7bb1b297f142f279e.png]: https://img-blog.csdnimg.cn/63cc435f80224fa7bb1b297f142f279e.png
[a60049b99ecf45abb9a51134c5798fc1.png]: https://img-blog.csdnimg.cn/a60049b99ecf45abb9a51134c5798fc1.png
[50021b9c7713434a8649279720e12206.png]: https://img-blog.csdnimg.cn/50021b9c7713434a8649279720e12206.png
[b4f120e5ed1b4aeaac1ccea244565e7d.png]: https://img-blog.csdnimg.cn/b4f120e5ed1b4aeaac1ccea244565e7d.png
[5fec6287f3584dbc82ebd9c68ca9bc63.png]: https://img-blog.csdnimg.cn/5fec6287f3584dbc82ebd9c68ca9bc63.png