elk安装

以你之姓@ 2022-05-15 15:56 310阅读 0赞

# 1、准备 #

## 1.1 简介 ##

ELK是Elasticsearch、Logstash、Kibana的简称，这三者是核心套件，但并非全部。

Elasticsearch是实时全文搜索和分析引擎，提供搜集、分析、存储数据三大功能；是一套开放REST和JAVA API等结构提供高效搜索功能，可扩展的分布式系统。它构建于Apache Lucene搜索引擎库之上。

Logstash是一个用来搜集、分析、过滤日志的工具。它支持几乎任何类型的日志，包括系统日志、错误日志和自定义应用程序日志。它可以从许多来源接收日志，这些来源包括 syslog、消息传递（例如 RabbitMQ）和JMX，它能够以多种方式输出数据，包括电子邮件、websockets和Elasticsearch。

Kibana是一个基于Web的图形界面，用于搜索、分析和可视化存储在 Elasticsearch指标中的日志数据。它利用Elasticsearch的REST接口来检索数据，不仅允许用户创建他们自己的数据的定制仪表板视图，还允许他们以特殊的方式查询和过滤数据

## 1.2 jdk安装 ##

wget --no-cookies --no-check-certificate --header "Cookie: gpw_e24=http:%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie" "http://download.oracle.com/otn-pub/java/jdk/8u77-b02/jdk-8u171-linux-x64.rpm"
    rpm -ivh jdk-8u171-linux-x64.rpm
    java -version

# 2、安装elasticsearch、kibana、logstash #

## 2.1 下载 ##

elk下载

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.2.4.rpm
    wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-x86_64.rpm
    wget https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.rpm

filebeat安装

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.4-x86_64.rpm

## 2.2 安装 ##

yum localinstall -y elasticsearch-6.2.4.rpm
    yum localinstall -y kibana-6.2.4-x86_64.rpm
    yum localinstall -y logstash-6.2.4.rpm

## 2.3 配置elk日志和数据目录 ##

mkdir -pv /data/elasticsearch/{data,logs}
    mkdir -pv /data/logstash/{data,logs}
    chown -R elasticsearch.elasticsearch /data/elasticsearch/
    chown -R logstash.logstash /data/logstash/

### 2.3.1 配置elasticsearch ###

vim /etc/elasticsearch/elasticsearch.yml

# ----------------------------------- Paths ------------------------------------
    #
    # Path to directory where to store the data (separate multiple locations by comma):
    #
    path.data: /data/elasticsearch/data
    #
    # Path to log files:
    #
    path.logs: /data/elasticsearch/logs
    # ---------------------------------- Network -----------------------------------
    #
    # Set the bind address to a specific IP (IPv4 or IPv6):
    #
    network.host: 0.0.0.0
    #
    # Set a custom port for HTTP:
    #
    http.port: 9200

### 2.3.2 配置logstash ###

vim /etc/logstash/logstash.yml

# ------------ Data path ------------------
    #
    # Which directory should be used by logstash and its plugins
    # for any persistent needs. Defaults to LOGSTASH_HOME/data
    #
    path.data: /data/logstash/data
    # log.level: info
    path.logs: /data/logstash/logs

vim /etc/logstash/conf.d/logstash.conf

input {
      beats {
        port => 5044
        codec => plain {
              charset => "UTF-8"
        }
      }
    }
    
    output {
      elasticsearch {
        hosts => "127.0.0.1:9200"
        manage_template => false
        index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
        document_type => "%{[@metadata][type]}"
      }
    }

### 2.3.3 配置kibana ###

# Kibana is served by a back end server. This setting specifies the port to use.
    server.port: 5601
    # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
    # The default is 'localhost', which usually means remote machines will not be able to connect.
    # To allow connections from remote users, set this parameter to a non-loopback address.
    server.host: "0.0.0.0"
    # The URL of the Elasticsearch instance to use for all your queries.
    elasticsearch.url: "http://localhost:9200"

**注意：**server.host不可配置外网ip，可以配置内网ip，否则报错。

## 2.4 安装nginx ##

### 2.4.1 安装 ###

yum -y install epel-release
    yum -y install nginx

## 2.4.2 配置nginx ##

cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak
    vim /etc/nginx/nginx.conf

location / {
      proxy_pass http://127.0.0.1:5601; #映射到kibana
    }

## 2.5 启动elk和nginx ##

systemctl daemon-reload  # 重新加载所有配置文件
    systemctl start elasticsearch logstash kibana nginx  # 启动ELK和nginx
    systemctl enable elasticsearch logstash kibana nginx  # 将ELK和nginx加入开机启动
    systemctl status elasticsearch logstash kibana nginx  #查看ELK和nginx启动状态

访问elasticsearch：

#curl http://localhost:9200/
    {
      "name" : "yevq6Lo",
      "cluster_name" : "elasticsearch",
      "cluster_uuid" : "Z996JpOjQW2rij8-7x9_jA",
      "version" : {
        "number" : "6.2.4",
        "build_hash" : "ccec39f",
        "build_date" : "2018-04-12T20:37:28.497551Z",
        "build_snapshot" : false,
        "lucene_version" : "7.2.1",
        "minimum_wire_compatibility_version" : "5.6.0",
        "minimum_index_compatibility_version" : "5.0.0"
      },
      "tagline" : "You Know, for Search"
    }

# 2.6 nginx认证 #

shell > yum -y install httpd-tools  # 安装 htpasswd 工具
    
    shell > cd /etc/nginx
    
    shell > htpasswd -c pass.db admin  # 创建认证用户 admin 并输入密码，添加用户时输入 htpasswd pass.db username
    
    shell > vim nginx.conf
    
    server {
    
            auth_basic "User Authentication";
            auth_basic_user_file /etc/nginx/kibana-user;
    
            location / {
              proxy_pass http://127.0.0.1:5601;
            }
           
        }

# 3 安装filebeat #

## 3.1 安装 ##

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.4-x86_64.rpm
    yum localinstall -y filebeat-6.2.4-x86_64.rpm

## 3.2 配置 ##

vim /etc/filebeat/filebeat.yml

filebeat.prospectors:
    - type: log
      enabled: true
      paths:
        - /data/work/wx/logs/all/*.log #本地日志
    output.logstash:
      # The Logstash hosts
      hosts: ["内网ip:5044"]

由于我们需要将日志输出到logstash，所以注释掉Elasticsearch中的内容即可

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70][]

## 3.3 启动 ##

systemctl start filebeat #启动
    systemctl enable filebeat #开机启动
    systemctl status filebeat #状态

# 4 在kibana中创建filebeat的索引 #

浏览器访问kibana：127.0.0.1（或者外网）

## 4.1 创建索引 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 1][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 2][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 3][]

## 4.2 查看日志 ##

创建完成后在kibana界面的“Discover”中选择“filebeat-\*”可查看filebeat发送过来的日志信息。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 4][]

## 4.3 显示host ##

因为所有的hostname部分都是主机名来代替ip设置，所以先要将logstash的ip和hostname写到 /etc/hosts 里面去。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 5][]

# 5、kibana汉化 #

**注意**：此项目适用于Kibana 5.x-6.x的任意版本，汉化过程不可逆，汉化前请注意备份！ 汉化资源会慢慢更新完善，已汉化过的Kibana可以重复使用此项目汉化更新的资源。除一小部分资源外，大部分资源无需重启Kibana，刷新页面即可看到效果。

git地址：

https://github.com/anbai-inc/Kibana_Hanization

使用方法：

python main.py Kibana安装目录（/usr/share/kibana）

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70]: /images/20220503/12f3291573dc4dba87496143fbe63bdb.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 1]: /images/20220503/61fffa4a48394cc18dee8371a355bc72.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 2]: /images/20220503/ea9d96a1299447f9b6725ed8486d135a.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 3]: /images/20220503/87d24424f9a64edcbf1663d1d6e90259.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 4]: /images/20220503/e216f99e1a374a538b0b6056910ba2eb.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpeGluZ3poZTI_size_16_color_FFFFFF_t_70 5]: /images/20220503/e6639c5ced4d4867a0b269691dedc23a.png