ELK logstash-7.5收集交换机日志

古城微笑少年丶 2022-09-12 08:56 346阅读 0赞

问题：有人反馈说7.x版本收集不了交换机日志，在此记录一次logstash-7.5收集华为交换机的日志记录。

前提：ELK环境已经安装完成，具体操作查看另外篇文章

## 一、交换机配置 ##

添加：info-center loghost 192.168.14.210，IP地址是logstash服务器，华为交换机默认是UDP514端口发送数据

1、查看交换机版本
    [SW30]display  version 
    Huawei Versatile Routing Platform Software
    VRP (R) software, Version 5.70 (S2700 V100R006C05)
    Copyright (C) 2003-2013 HUAWEI TECH CO., LTD
    Quidway S2700-9TP-SI-AC Routing Switch uptime is 23 weeks, 5 days, 7 hours, 28 minutes
    
    E8FED 0(Master) : uptime is 23 weeks, 5 days, 7 hours, 27 minutes
    64M bytes DDR Memory
    16M bytes FLASH
    Pcb      Version :  VER E
    Basic  BOOTROM  Version :  149 Compiled at Mar 15 2013, 11:02:25
    Software Version : VRP (R) Software, Version 5.70 (V100R006C05)
    
    2、配置内容
    [SW30]display  current-configuration  | in info
     info-center loghost 192.168.14.210
     snmp-agent sys-info version all

## 二、logstash7.5安装 ##

1、安装JDK

[root@localhost ~]# tar -zxvf jdk-11.0.5_linux-x64_bin.tar.gz -C /usr/local/
    [root@localhost ~]# vim /etc/profile
    export JAVA_HOME=/usr/local/jdk-11.0.5/
    export PATH=$PATH:$JAVA_HOME/bin
    export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar:$CLASSPATH
     
    [root@localhost ~]# source  /etc/profile

2、解压logstash二进制包

[root@localhost ~]#tar -zxvf logstash-7.5.0.tar.gz  -C /usr/local/

3、添加环境变量

[root@localhost ~]# vi /etc/profile
    export PATH=$PATH:/usr/local/logstash-7.5.0/bin
    [root@localhost ~]# source /etc/profile

## 三、logstash7.5配置 ##

1、关闭rsyslog服务，因为这个会占用514端口

[root@localhost ~]# systemctl stop rsyslog

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16][]

2、添加logstash配置文件，根据监听交换机端口区分不通网络设备型号(直接复制可用，修改下IP地址)

[root@localhost ~]# vi  /usr/local/logstash-7.5.0/config/switch.conf 
    [root@localhost ~]# cat /usr/local/logstash-7.5.0/config/switch.conf 
    input{
        tcp { port => 5002 
        type => "Cisco"}
        udp { port => 514
        type => "HUAWEI"}
        udp { port => 5002
        type => "Cisco"}
        udp { port => 5003
        type => "H3C"}
    }
    filter {
        if [type] == "Cisco" {
        grok {
        match => { "message" => "<%{BASE10NUM:syslog_pri}>%{NUMBER:log_sequence}: .%{SYSLOGTIMESTAMP:timestamp}: %%{DATA:facility}-%{POSINT:severity}-%{CISCO_REASON:mnemonic}: %{GREEDYDATA:message}" }
        match => { "message" => "<%{BASE10NUM:syslog_pri}>%{NUMBER:log_sequence}: %{SYSLOGTIMESTAMP:timestamp}: %%{DATA:facility}-%{POSINT:severity}-%{CISCO_REASON:mnemonic}: %{GREEDYDATA:message}" }
        add_field => {"severity_code" => "%{severity}"}
        overwrite => ["message"]
        }
    }
        elseif [type] == "H3C" {
        grok {
        match => { "message" => "<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{YEAR:year} %{DATA:hostname} %%%{DATA:vvmodule}/%{POSINT:severity}/%{DATA:digest}: %{GREEDYDATA:message}" }
        remove_field => [ "year" ]
        add_field => {"severity_code" => "%{severity}"}
        overwrite => ["message"]
        }
    }
    	elseif [type] == "HUAWEI" {
        grok {
           match => { "message" => "<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{DATA:hostname} %%%{DATA:ddModuleName}/%{POSINT:severity}/%{DATA:Brief}:%{GREEDYDATA:message}"}
           match => { "message" => "<%{BASE10NUM:syslog_pri}>%{SYSLOGTIMESTAMP:timestamp} %{DATA:hostname} %{DATA:ddModuleName}/%{POSINT:severity}/%{DATA:Brief}:%{GREEDYDATA:message}"}
           remove_field => [ "timestamp" ]
        add_field => {"severity_code" => "%{severity}"}
        overwrite => ["message"]
        }
    }
    #mutate {
    #        gsub => [
    #        "severity", "0", "Emergency",
    #        "severity", "1", "Alert",
    #        "severity", "2", "Critical",
    #        "severity", "3", "Error",
    #        "severity", "4", "Warning",
    #        "severity", "5", "Notice",
    #        "severity", "6", "Informational",
    #        "severity", "7", "Debug"	
    #        ]
    #    }
    }
    output{
        stdout {
           codec => rubydebug
    }
        elasticsearch {
            index =>
            "syslog-%{+YYYY.MM.dd}"
            hosts => ["192.168.14.211:9200"]
            user => "elastic"
            password => "password"
        }
    }

这里为了方便查看，直接输出到终端显示了，工作环境可以删除stdout的配置。并且添加了用户名和密码认证

3、启动，在终端可以查看到数据

[root@localhost ~]# logstash -f /usr/local/logstash-7.5.0/config/switch.conf

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 1][]

##  四、kibana查看交换机日志 ##

1、打开Management

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 2][]

2、添加索引

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 3][]

3、搜索在配置文件里面自定义的索引名称

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 4][]

4、点击创建

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 5][]

5、回到首页查看日志

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 6][]

##  五、错误记录 ##

1、因为elk7的elasticsearch增加了认证功能，如果logstash配置文件没有添加用户名和密码就有如下连接错误提示

![watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 7][]

[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16]: /images/20220828/b1c6f42957c54687b54bf2b844a1b43d.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 1]: /images/20220828/9c89075a2231453eb76f4d4267a8d8e5.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 2]: /images/20220828/8bc645a0654a4cc99f2eef81fceb785d.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 3]: /images/20220828/21365bbc646f41c0abac2890a9b50377.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 4]: /images/20220828/bf016ff899394631ae759935b93817a8.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 5]: /images/20220828/d019030f05a0495db6d63b364729a142.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 6]: /images/20220828/5841cae3d14a46778c7e4b4f16d2e769.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5Y-L5Lq6YeeslOiusA_size_20_color_FFFFFF_t_70_g_se_x_16 7]: /images/20220828/75b2d267655045daa15588994a0b0ec5.png