ES总结（三）停词，同义词配置

川长思鸟来 2021-10-14 04:30 674阅读 0赞

**目录**

前言

搜索

停词配置

同义词

如何实现动态配置同义词呢？

意义

--------------------

# 前言 #

上次总结了es的搜索匹配算法等等，这次总结一下在es搜索的时候是将词语进行分词，那么我们如何让es更好的分词？

# 搜索 #

[上一篇实践搜索优化][Link 1]

# 停词配置 #

设置了之后不会出现类型的词语

一般采用ik插件进行分词，具体百度，就是下载下来之后解压到es plugins/ik目录下

看下IKAnalyzer.cfg

<?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
    <properties>
    	<comment>IK Analyzer 扩展配置</comment>
    	
    	<entry key="ext_dict"></entry>
    	 
    	<entry key="ext_stopwords"></entry>
    	
    	
    	
    	
    </properties>

在上面配置即可，但是如果要更新的文本的话，需要重启es。所以网上有些就去改源码，进行数据库动态配置，[看这里][Link 2]

# 同义词 #

这个很有意义，比如我搜索美妆，可能需要关联粉底液，等等一系列东西，可以使用同义词去配置，具体也可以看上面链接。

[https://github.com/ginobefun/elasticsearch-dynamic-synonym][https_github.com_ginobefun_elasticsearch-dynamic-synonym] 这个插件

按照里面步骤走下来即可，**注意点**：版本问题：es我使用5.6.15，上面插件采用最新的，可以使用，然后就是插件的配置了，这个根据具体情况去配置

plugin-descriptor.properties

# 'description': simple summary of the plugin
    description=ElasticSearch Plugin for Dynaic Synonym Token Filter.
    #
    # 'version': plugin's version
    version=5.2.2
    #
    # 'name': the plugin name
    name=dynamic-synonym
    
    ### mandatory elements for site plugins:
    #
    # 'site': set to true to indicate contents of the _site/
    #  directory in the root of the plugin should be served.
    site=true
    #
    ### mandatory elements for jvm plugins :
    #
    # 'jvm': true if the 'classname' class should be loaded
    #  from jar files in the root directory of the plugin.
    #  Note that only jar files in the root directory are
    #  added to the classpath for the plugin! If you need
    #  other resources, package them into a resources jar.
    jvm=true
    #
    # 'classname': the name of the class to load, fully-qualified.
    classname=com.ginobefunny.elasticsearch.plugins.synonym.DynamicSynonymPlugin
    #
    # 'java.version' version of java the code is built against
    # use the system property java.specification.version
    # version string must be a sequence of nonnegative decimal integers
    # separated by "."'s and may have leading zeros
    java.version=1.8
    #
    # 'elasticsearch.version' version of elasticsearch compiled against
    # You will have to release a new version of the plugin for each new
    # elasticsearch release. This version is checked when the plugin
    # is loaded so Elasticsearch will refuse to start in the presence of
    # plugins with the incorrect elasticsearch.version.
    elasticsearch.version=5.6.15
    #
    ### deprecated elements for jvm plugins :
    #
    # 'isolated': true if the plugin should have its own classloader.
    # passing false is deprecated, and only intended to support plugins
    # that have hard dependencies against each other. If this is
    # not specified, then the plugin is isolated by default.
    isolated=true
    #

1.我采用postman去创建index

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODMzNjY1OA_size_16_color_FFFFFF_t_70][]

{
        "settings": {
            "analysis": {
                "analyzer": {
                    "analyzer_with_dynamic_synonym": {
                        "type": "custom",
                        "tokenizer": "whitespace",
                        "filter": [
                            "my_synonym"
                        ]
                    }
                },
                "filter": {
                    "my_synonym": {
                        "type": "dynamic-synonym",
                        "expand": true,
                        "ignore_case": true,
                        "tokenizer": "whitespace",
                        "db_url": "jdbc:mysql://localhost:3306/test?user=root&password=ys123456&useUnicode=true&characterEncoding=UTF8"
                    }
                }
            }
        }
    }

里面的数据库链接自己配置

2.创建mapping

POST /index_synonym/product/_mapping
    {
        "product": {
            "properties": {
                "productName": {
                    "type": "text",
                    "analyzer": "analyzer_with_dynamic_synonym"
                }
            }
        }
    }

3.查询搜索同义词是否成功

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODMzNjY1OA_size_16_color_FFFFFF_t_70 1][]

搜索结果

{
        "took": 5,
        "timed_out": false,
        "_shards": {
            "total": 5,
            "successful": 5,
            "skipped": 0,
            "failed": 0
        },
        "hits": {
            "total": 2,
            "max_score": 0.9547894,
            "hits": [
                {
                    "_index": "index_synonym",
                    "_type": "product",
                    "_id": "4",
                    "_score": 0.9547894,
                    "_source": {
                        "productName": "This is a adidas sports jacket"
                    }
                },
                {
                    "_index": "index_synonym",
                    "_type": "product",
                    "_id": "3",
                    "_score": 0.2824934,
                    "_source": {
                        "productName": "This is a adidas shoes"
                    }
                }
            ]
        }
    }

可以看到同义词起作用了。

# 如何实现动态配置同义词呢？ #

网上很多也是使用txt去配置的，这里也需要改源码，[看这一篇][Link 2]，不愧是大佬呀。

在es高版本的话不支持手动去配置，只能api去请求，就是createIndex(xx.class,setting)后面是setting将上面的数据转成map进行注入

# 意义 #

可以让搜索更加准确，不然只能上算法跟机器学习了，比如我搜索眼影，会附带眼影盘这些出来。同义词的出现可以让我们搜索更加精确

[Link 1]: https://blog.csdn.net/weixin_38336658/article/details/99607745
[Link 2]: https://blog.csdn.net/zq199419951001/article/details/89884461
[https_github.com_ginobefun_elasticsearch-dynamic-synonym]: https://github.com/ginobefun/elasticsearch-dynamic-synonym
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODMzNjY1OA_size_16_color_FFFFFF_t_70]: /images/20211013/14ca2b6ff87b463684798996d0a625fc.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl8zODMzNjY1OA_size_16_color_FFFFFF_t_70 1]: /images/20211013/db4a61fba13c4cefa61194ed43e89047.png