【ElasticSearch】（八）—— 自动补全

爱被打了一巴掌 2023-09-30 14:18 2阅读 0赞

**目录**

一、拼音分词器

二、自定义分词器

三、自动补全查询

四、实现酒店搜索框自动补全

1.修改酒店映射结构

2.HotelDoc实体

3.自动补全查询的JavaAPI

--------------------

当用户在搜索框输入字符时，我们应该提示出与该字符有关的搜索项，如图：

![1367322116a74c3dbfc4bed9592ffe2c.png][]

这种根据用户输入的字母，提示完整词条的功能，就是自动补全了。

因为需要根据拼音字母来推断，因此要用到拼音分词功能。

## 一、拼音分词器 ##

要实现根据字母做补全，就必须对文档按照拼音分词。在GitHub上恰好有elasticsearch的拼音分词插件。地址：[https://github.com/medcl/elasticsearch-analysis-pinyin][https_github.com_medcl_elasticsearch-analysis-pinyin]

或者gitCode上镜像 [mirrors / medcl / elasticsearch-analysis-pinyin · GitCode][mirrors _ medcl _ elasticsearch-analysis-pinyin _ GitCode]

![71141f666a904917b4494432c58b9b1e.png][]

安装方式与IK分词器一样，分三步：

①解压

②上传到虚拟机中，elasticsearch的plugin目录

③重启elasticsearch

④测试

详细安装步骤可以参考IK分词器的安装过程。

[【ElasticSearch】（二）—— 安装elasticsearch\_机智兵的博客-CSDN博客\_安装elasticsearch][ElasticSearch_ _elasticsearch_-CSDN_elasticsearch]

测试语法如下：

> POST /\_analyze  
> \{  
> "text": "我在学习拼音分词器",  
> "analyzer": "pinyin"  
> \}

{
      "tokens" : [
        {
          "token" : "wo",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 0
        },
        {
          "token" : "wzxxpyfcq",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 0
        },
        {
          "token" : "zai",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 1
        },
        {
          "token" : "xue",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 2
        },
        {
          "token" : "xi",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 3
        },
        {
          "token" : "pin",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 4
        },
        {
          "token" : "yin",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 5
        },
        {
          "token" : "fen",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 6
        },
        {
          "token" : "ci",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 7
        },
        {
          "token" : "qi",
          "start_offset" : 0,
          "end_offset" : 0,
          "type" : "word",
          "position" : 8
        }
      ]
    }

## 二、自定义分词器 ##

默认的拼音分词器会将每个汉字单独分为拼音，而我们希望的是每个词条形成一组拼音，需要对拼音分词器做个性化定制，形成自定义分词器。

> elasticsearch中分词器（analyzer）的组成包含三部分：
> 
>  *  character filters：在tokenizer之前对文本进行处理。例如删除字符、替换字符
>  *  tokenizer：将文本按照一定的规则切割成词条（term）。例如keyword，就是不分词；还有ik\_smart
>  *  tokenizer filter：将tokenizer输出的词条做进一步处理。例如大小写转换、同义词处理、拼音处理等

文档分词时会依次由这三部分来处理文档：

![114833448e1e42a1bad7d5a0462cdfa6.png][] 声明自定义分词器的语法如下：

> PUT /test  
> \{  
> "settings": \{  
> "analysis": \{  
> "analyzer": \{ // 自定义分词器  
> "my\_analyzer": \{ // 分词器名称  
> "tokenizer": "ik\_max\_word",  
> "filter": "py"  
> \}  
> \},  
> "filter": \{ // 自定义tokenizer filter  
> "py": \{ // 过滤器名称  
> "type": "pinyin", // 过滤器类型，这里是pinyin  
> "keep\_full\_pinyin": false,  
> "keep\_joined\_full\_pinyin": true,  
> "keep\_original": true,  
> "limit\_first\_letter\_length": 16,  
> "remove\_duplicated\_term": true,  
> "none\_chinese\_pinyin\_tokenize": false  
> \}  
> \}  
> \}  
> \},  
> "mappings": \{  
> "properties": \{  
> "name": \{  
> "type": "text",  
> "analyzer": "my\_analyzer",  
> "search\_analyzer": "ik\_smart"  
> \}  
> \}  
> \}  
> \}

测试：

> GET /test0/\_analyze  
> \{  
> "text": \["今天天气真不错"\],  
> "analyzer": "my\_analyzer"  
> \}

结果：

![309df0d6c32147cf80743b991313bdf8.png][]

**小结**

> 如何使用拼音分词器？
> 
>  *  ①下载pinyin分词器
>  *  ②解压并放到elasticsearch的plugin目录
>  *  ③重启即可
> 
> 如何自定义分词器？
> 
>  *  ①创建索引库时，在settings中配置，可以包含三部分
>  *  ②character filter
>  *  ③tokenizer
>  *  ④filter
> 
> 拼音分词器注意事项？
> 
>  *  为了避免搜索到同音字，搜索时不要使用拼音分词器

## 三、自动补全查询 ##

elasticsearch提供了[Completion Suggester][]查询来实现自动补全功能。这个查询会匹配以用户输入内容开头的词条并返回。为了提高补全查询的效率，对于文档中字段的类型有一些约束：

*  参与补全查询的字段必须是completion类型。
 *  字段的内容一般是用来补全的多个词条形成的数组。

比如，一个这样的索引库：

> // 创建索引库  
> PUT test  
> \{  
> "mappings": \{  
> "properties": \{  
> "title":\{  
> "type": "completion"  
> \}  
> \}  
> \}  
> \}

然后插入下面的数据：

> // 示例数据  
> POST test/\_doc  
> \{  
> "title": \["Sony", "WH-1000XM3"\]  
> \}  
> POST test/\_doc  
> \{  
> "title": \["SK-II", "PITERA"\]  
> \}  
> POST test/\_doc  
> \{  
> "title": \["Nintendo", "switch"\]  
> \}

查询的DSL语句如下：

> // 自动补全查询  
> GET /test/\_search  
> \{  
> "suggest": \{  
> "title\_suggest": \{  
> "text": "s", // 关键字  
> "completion": \{  
> "field": "title", // 补全查询的字段  
> "skip\_duplicates": true, // 跳过重复的  
> "size": 10 // 获取前10条结果  
> \}  
> \}  
> \}  
> \}

## 四、实现酒店搜索框自动补全 ##

现在，我们的hotel索引库还没有设置拼音分词器，需要修改索引库中的配置。但是我们知道索引库是无法修改的，只能删除然后重新创建。

另外，我们需要添加一个字段，用来做自动补全，将brand、suggestion、city等都放进去，作为自动补全的提示。

> 因此，总结一下，我们需要做的事情包括：
> 
> 1.  新建hotel索引库结构，设置自定义拼音分词器
> 2.  修改索引库的name、all字段，使用自定义分词器
> 3.  索引库添加一个新字段suggestion，类型为completion类型，使用自定义的分词器
> 4.  给HotelDoc类添加suggestion字段，内容包含brand、business
> 5.  重新导入数据到hotel库

### 1.修改酒店映射结构 ###

> // 酒店数据索引库  
> PUT /hotel  
> \{  
> "settings": \{  
> "analysis": \{  
> "analyzer": \{  
> "text\_anlyzer": \{  
> "tokenizer": "ik\_max\_word",  
> "filter": "py"  
> \},  
> "completion\_analyzer": \{  
> "tokenizer": "keyword",  
> "filter": "py"  
> \}  
> \},  
> "filter": \{  
> "py": \{  
> "type": "pinyin",  
> "keep\_full\_pinyin": false,  
> "keep\_joined\_full\_pinyin": true,  
> "keep\_original": true,  
> "limit\_first\_letter\_length": 16,  
> "remove\_duplicated\_term": true,  
> "none\_chinese\_pinyin\_tokenize": false  
> \}  
> \}  
> \}  
> \},  
> "mappings": \{  
> "properties": \{  
> "id":\{  
> "type": "keyword"  
> \},  
> "name":\{  
> "type": "text",  
> "analyzer": "text\_anlyzer",  
> "search\_analyzer": "ik\_smart",  
> "copy\_to": "all"  
> \},  
> "address":\{  
> "type": "keyword",  
> "index": false  
> \},  
> "price":\{  
> "type": "integer"  
> \},  
> "score":\{  
> "type": "integer"  
> \},  
> "brand":\{  
> "type": "keyword",  
> "copy\_to": "all"  
> \},  
> "city":\{  
> "type": "keyword"  
> \},  
> "starName":\{  
> "type": "keyword"  
> \},  
> "business":\{  
> "type": "keyword",  
> "copy\_to": "all"  
> \},  
> "location":\{  
> "type": "geo\_point"  
> \},  
> "pic":\{  
> "type": "keyword",  
> "index": false  
> \},  
> "all":\{  
> "type": "text",  
> "analyzer": "text\_anlyzer",  
> "search\_analyzer": "ik\_smart"  
> \},  
> "suggestion":\{  
> "type": "completion",  
> "analyzer": "completion\_analyzer"  
> \}  
> \}  
> \}  
> \}

### 2.HotelDoc实体 ###

>     package com.elasticsearch.hotel.pojo;
>     
>     import lombok.Data;
>     import lombok.NoArgsConstructor;
>     
>     import java.util.ArrayList;
>     import java.util.Arrays;
>     import java.util.Collections;
>     import java.util.List;
>     
>     @Data
>     @NoArgsConstructor
>     public class HotelDoc {
>         private Long id;
>         private String name;
>         private String address;
>         private Integer price;
>         private Integer score;
>         private String brand;
>         private String city;
>         private String starName;
>         private String business;
>         private String location;
>         private String pic;
>         private Object distance;
>         private Boolean isAD;
>         private List<String> suggestion;
>     
>         public HotelDoc(Hotel hotel) {
>             this.id = hotel.getId();
>             this.name = hotel.getName();
>             this.address = hotel.getAddress();
>             this.price = hotel.getPrice();
>             this.score = hotel.getScore();
>             this.brand = hotel.getBrand();
>             this.city = hotel.getCity();
>             this.starName = hotel.getStarName();
>             this.business = hotel.getBusiness();
>             this.location = hotel.getLatitude() + ", " + hotel.getLongitude();
>             this.pic = hotel.getPic();
>             // 组装suggestion
>             if(this.business.contains("/")){
>                 // business有多个值，需要切割
>                 String[] arr = this.business.split("/");
>                 // 添加元素
>                 this.suggestion = new ArrayList<>();
>                 this.suggestion.add(this.brand);
>                 Collections.addAll(this.suggestion, arr);
>             }else {
>                 this.suggestion = Arrays.asList(this.brand, this.business);
>             }
>         }
>     }

### 3.自动补全查询的JavaAPI ###

![1515094c6a024c52a81ad161471f77f2.png][]

核心实现方法：

@Override
    public List<String> getSuggestions(String prefix) {
        try {
            // 1.准备Request
            SearchRequest request = new SearchRequest("hotel");
            // 2.准备DSL
            request.source().suggest(new SuggestBuilder().addSuggestion(
                "suggestions",
                SuggestBuilders.completionSuggestion("suggestion")
                .prefix(prefix)
                .skipDuplicates(true)
                .size(10
            ));
            // 3.发起请求
            SearchResponse response = client.search(request, RequestOptions.DEFAULT);
            // 4.解析结果
            Suggest suggest = response.getSuggest();
            // 4.1.根据补全查询名称，获取补全结果
            CompletionSuggestion suggestions = suggest.getSuggestion("suggestions");
            // 4.2.获取options
            List<CompletionSuggestion.Entry.Option> options = suggestions.getOptions();
            // 4.3.遍历
            List<String> list = new ArrayList<>(options.size());
            for (CompletionSuggestion.Entry.Option option : options) {
                String text = option.getText().toString();
                list.add(text);
            }
            return list;
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

[1367322116a74c3dbfc4bed9592ffe2c.png]: https://img-blog.csdnimg.cn/1367322116a74c3dbfc4bed9592ffe2c.png
[https_github.com_medcl_elasticsearch-analysis-pinyin]: https://github.com/medcl/elasticsearch-analysis-pinyin
[mirrors _ medcl _ elasticsearch-analysis-pinyin _ GitCode]: https://gitcode.net/mirrors/medcl/elasticsearch-analysis-pinyin
[71141f666a904917b4494432c58b9b1e.png]: https://img-blog.csdnimg.cn/71141f666a904917b4494432c58b9b1e.png
[ElasticSearch_ _elasticsearch_-CSDN_elasticsearch]: https://blog.csdn.net/a6470831/article/details/124787155
[114833448e1e42a1bad7d5a0462cdfa6.png]: https://img-blog.csdnimg.cn/114833448e1e42a1bad7d5a0462cdfa6.png
[309df0d6c32147cf80743b991313bdf8.png]: https://img-blog.csdnimg.cn/309df0d6c32147cf80743b991313bdf8.png
[Completion Suggester]: https://www.elastic.co/guide/en/elasticsearch/reference/7.6/search-suggesters.html
[1515094c6a024c52a81ad161471f77f2.png]: https://img-blog.csdnimg.cn/1515094c6a024c52a81ad161471f77f2.png