Elasticsearch 字段动态/显示映射，kibana数据可视化

旧城等待， 2022-11-11 05:11 389阅读 0赞

### 文章目录 ###

*  1, 动态映射：Dynamic mapping (动态确定字段类型，以第一个值为参照）
 *  2, 显示映射：Explicit mapping (预先规定字段类型）
 *  3, 使用kibana：可视化查看elasticsearch数据
 *  4, Elasticsearch/Kibana 设置用户登录

# 1, 动态映射：Dynamic mapping (动态确定字段类型，以第一个值为参照） #

Elasticsearch 默认不需要（事先建好索引，确定字段名和字段类型，然后才能添加数据），而是可以直接添加数据，并根据数值来动态确定字段类型[https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html][https_www.elastic.co_guide_en_elasticsearch_reference_current_dynamic-field-mapping.html]

*  json int --> long
 *  json double --> float
 *  json boolean --> boolean
 *  json array --> 根据元素类型确定类型
 *  json object --> 嵌套的对象类型（内部字段类型按对应规则确定）
 *  json-string date-- > date : 默认格式为 yyyy-MM-dd 或 yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch\_millis

示例如下：

# 1，添加一条数据（完成的动作：创建索引名，并根据kv对来确定字段和数值类型，然后添加数据）
    curl -X POST "localhost:9200/test_dynamic_mapidx/_doc/1?pretty" -H 'Content-Type: application/json' -d' { "id": 1, "age": 23, "name": "name1", "is_rich_man":false, "salary": 13210.23, "birth_year": "1993-01-02", "company": {"location":"bj", "house":"123123" }, "hobby": ["books","running","movie"], "family_member":[{ "name":"lisi", "role":"son" }, { "name":"xiaoli", "role":"wife", "age":33 }] } '
    # 2，查看索引的字段隐射结果
    [root@hadoop01 ~]# curl -X GET "192.168.56.1:9200/test_dynamic_mapidx/_mapping?pretty"
    { 
      "test_dynamic_mapidx" : { 
        "mappings" : { 
          "properties" : { 
            "birth_year" : { 
              "type" : "date"
            },
            "company" : { 
              "properties" : { 
                "house" : { 
                  "type" : "text",
                  "fields" : { 
                    "keyword" : { 
                      "type" : "keyword",
                      "ignore_above" : 256
                    }
                  }
                },
              }
            },
    ....

# 2, 显示映射：Explicit mapping (预先规定字段类型） #

设计索引（名称+ 字段类型）：  
创建index: [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html][https_www.elastic.co_guide_en_elasticsearch_reference_current_indices-create-index.html]  
更新index-settings: [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html][https_www.elastic.co_guide_en_elasticsearch_reference_current_indices-update-settings.html]  
更新index-mapping: [https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html][https_www.elastic.co_guide_en_elasticsearch_reference_current_indices-put-mapping.html]

# 1，创建索引名
    #curl -X PUT "localhost:9200/test_idx1?pretty"
    curl -X PUT "localhost:9200/test_idx1?pretty" -H 'Content-Type: application/json' -d' { "settings": { "number_of_shards": 1 }, "mappings": { "properties": { "field1": { "type": "text" } } } } '
    
    
    # 2，更新已存在索引，添加字段和映射类型: date类型默认是yyyy-MM-dd 或 yyyy/MM/dd HH:mm:ss||yyyy/MM/dd||epoch_millis
    # text 类型适合用于 full-text搜索，
    # keyword 类型即不分词的string类型适合用于排序/聚合： While text fields work well for full-text search, keyword fields are not analyzed and may work better for sorting or aggregations.
    # 
    curl -X PUT "localhost:9200/test_idx1/_mapping?pretty" -H 'Content-Type: application/json' -d' { "properties": { "id": { "type": "long"}, "name": { "type": "text"}, "age": { "type": "long"}, "birth": { "type": "date", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis", "ignore_malformed": false, "null_value": null } } } '
    
    # 3，查看字段映射规则
    curl -X GET "localhost:9200/test_idx1/_mapping?pretty"
    
    # 4，添加数据
    curl -X POST "localhost:9200/test_idx1/_doc/1?pretty" -H 'Content-Type: application/json' -d' { "id": 1, "name": "name1", "age": 23, "birth": "1992-01-02 09:01:10" } '

# 3, 使用kibana：可视化查看elasticsearch数据 #

kibana下载：[https://www.elastic.co/cn/downloads/kibana][https_www.elastic.co_cn_downloads_kibana]

*  kibana 可以自动给后续添加的elasticsearch索引添加上，而对安装前已存在的es索引要手动配对
 *  配置kibana.yml，并启动

server.host: "0.0.0.0"
    elasticsearch.hosts: ["http://localhost:9200"]
    # Supported languages are the following: English - en , by default , Chinese - zh-CN .
    #i18n.locale: "en"

使用kinana查看elasticsearch数据 [https://www.elastic.co/guide/en/kibana/current/discover.html][https_www.elastic.co_guide_en_kibana_current_discover.html]

Home --> Management ：Stack Management  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70]

Home --> Analytics：Discover

#kql语法示例：
    #查询：地址中包含mill 或 lane, 并且工资 大于 30000: 
    	match语法
    		address : [mill lane]     and balance >= 30000
    		address : (mill  or lane) and balance >= 30000
    		
    	match_phrase语法
    		address : ("mill" or "lane" ) and balance >= 30000

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70 1]

# 4, Elasticsearch/Kibana 设置用户登录 #

[https://www.elastic.co/guide/en/elasticsearch/reference/current/get-started-enable-security.html][https_www.elastic.co_guide_en_elasticsearch_reference_current_get-started-enable-security.html]  
elasticsearch.yml

xpack.security.enabled: true
    discovery.type: single-node
    
    #重启Elasticsearch, 并设置内置的默认账户密码
    D:\download\elk-stack\elasticsearch-7.11.2\bin
    λ .\elasticsearch-setup-passwords.bat interactive
    Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.
    You will be prompted to enter passwords as the process progresses.
    Please confirm that you would like to continue [y/N]y
    
    Enter password for [elastic]:
    Reenter password for [elastic]:
    Enter password for [apm_system]:
    Reenter password for [apm_system]:
    Enter password for [kibana_system]:
    Reenter password for [kibana_system]:
    Enter password for [logstash_system]:
    Reenter password for [logstash_system]:
    Enter password for [beats_system]:
    Reenter password for [beats_system]:
    Enter password for [remote_monitoring_user]:
    Reenter password for [remote_monitoring_user]:
    ...
    Changed password for user [kibana]
    Changed password for user [elastic]

kibana.yml

elasticsearch.username: "kibana_system"
    elasticsearch.password: "123456"

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70 2]

[https_www.elastic.co_guide_en_elasticsearch_reference_current_dynamic-field-mapping.html]: https://www.elastic.co/guide/en/elasticsearch/reference/current/dynamic-field-mapping.html
[https_www.elastic.co_guide_en_elasticsearch_reference_current_indices-create-index.html]: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html
[https_www.elastic.co_guide_en_elasticsearch_reference_current_indices-update-settings.html]: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html
[https_www.elastic.co_guide_en_elasticsearch_reference_current_indices-put-mapping.html]: https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html
[https_www.elastic.co_cn_downloads_kibana]: https://www.elastic.co/cn/downloads/kibana
[https_www.elastic.co_guide_en_kibana_current_discover.html]: https://www.elastic.co/guide/en/kibana/current/discover.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70]: /images/20221022/b34ab9baba5842cbab139ff8aa90fb98.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70 1]: /images/20221022/ede0f5570fe249488b9e1361eb7374fd.png
[https_www.elastic.co_guide_en_elasticsearch_reference_current_get-started-enable-security.html]: https://www.elastic.co/guide/en/elasticsearch/reference/current/get-started-enable-security.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20210319144505435.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2V5ZW9mZWFnbGU=,size_16,color_FFFFFF,t_70