ik 分词器远程扩展字典、停用字典配置，热更新扩展字典、停用字典

朱雀 2022-11-18 04:38 318阅读 0赞

ik 分词器官网:[https://github.com/medcl/elasticsearch-analysis-ik][https_github.com_medcl_elasticsearch-analysis-ik]

ik 分词器扩展字典本地配置:[https://blog.csdn.net/shfqbluestone/article/details/115524821?spm=1001.2014.3001.5501][https_blog.csdn.net_shfqbluestone_article_details_115524821_spm_1001.2014.3001.5501]

1.  首先编辑 ik 的配置文件 /xxx/elasticsearch-7.12.0/plugins/ik/config/IKAnalyzer.cfg.xml :

<?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
    <properties>
    	<comment>IK Analyzer 扩展配置</comment>
    	
    	<entry key="ext_dict">shfq_ext_dic.dic</entry>
    	 
    	<entry key="ext_stopwords"></entry>
    	
    	<entry key="remote_ext_dict">http://localhost:8162/test/downloadExtDic?fileName=extWords.dic</entry>
    	
    	<entry key="remote_ext_stopwords">http://localhost:8162/test/downloadExtDic?fileName=stopWords.dic</entry>
    </properties>

1.  在下载扩展、停用词典的接口里返回 Last-Modified 或 ETag responder header 。

package com.huitongjy.recommend.exercise.controller;
    
    import com.huitongjy.common.util.LogUtils;
    import io.swagger.annotations.Api;
    import io.swagger.models.HttpMethod;
    import org.apache.commons.lang3.StringUtils;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import org.springframework.core.io.UrlResource;
    import org.springframework.http.HttpHeaders;
    import org.springframework.http.MediaType;
    import org.springframework.http.ResponseEntity;
    import org.springframework.web.bind.annotation.RequestMapping;
    import org.springframework.web.bind.annotation.RestController;
    import javax.servlet.http.HttpServletRequest;
    import java.io.File;
    import java.io.IOException;
    import java.net.MalformedURLException;
    import java.nio.file.Files;
    import java.nio.file.Path;
    import java.nio.file.Paths;
    import java.nio.file.attribute.BasicFileAttributes;
    
    @RestController
    @Api(value = "PrivateController", tags = "测试")
    
    public class TestController { 
        private static final Logger LOG = LoggerFactory.getLogger(PrivateController.class);
    
        @RequestMapping(value = "/test/downloadExtDic")
        public ResponseEntity<org.springframework.core.io.Resource> downloadFileFromLocal(HttpServletRequest request, String fileName) { 
            if (StringUtils.isEmpty(fileName)) { 
                return null;
            }
            // 可以把字典的配置文件写到配置中心
            String location = "";
            if (!location.endsWith(File.separator)) { 
                location += File.separator;
            }
            String method = request.getMethod();
            boolean isHead = HttpMethod.HEAD.name().equals(method);
            String pathStr = location + fileName;
            File file = new File(pathStr);
            if (!file.exists()) { 
                LogUtils.error(LOG, "字典不存在", "path", pathStr);
                return null;
            }
    
            Path path = Paths.get(pathStr);
            org.springframework.core.io.Resource resource = null;
    
            try { 
                resource = new UrlResource(path.toUri());
            } catch (MalformedURLException e) { 
                LogUtils.error(LOG, "加载字典异常", e);
                return null;
            }
            HttpHeaders httpHeaders = new HttpHeaders();
            httpHeaders.add(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"");
            httpHeaders.add(HttpHeaders.ETAG, generateLastUpdateDate(path) + "");
            if (isHead) { 
                resource = null;
            }
            return ResponseEntity.ok()
                    .contentType(MediaType.TEXT_PLAIN)
                    .headers(httpHeaders)
                    .body(resource);
        }
    
        private long generateLastUpdateDate(Path path) { 
            try { 
                BasicFileAttributes attr = Files.readAttributes(path, BasicFileAttributes.class);
                Long l = attr.lastModifiedTime().toMillis();
                return l;
            } catch (IOException e) { 
                return -1;
            }
    
        }
    
    }

1.  添加扩展字典、停用字典。

ik 分词器会每隔 1 分钟调用一次 head 方法根据 etag 来判断字典是否更新了，如果更新了的话才会调用 get 方法去下载字典。先调用 head 方法确认更新后再调用 get下载，这样就节省了带宽流量，也省的去更新词典了。

[https_github.com_medcl_elasticsearch-analysis-ik]: https://github.com/medcl/elasticsearch-analysis-ik
[https_blog.csdn.net_shfqbluestone_article_details_115524821_spm_1001.2014.3001.5501]: https://blog.csdn.net/shfqbluestone/article/details/115524821?spm=1001.2014.3001.5501