Elasticsearch 7.2.0 搜索时报 all shards failed 错误

快来打我* 2023-07-22 07:00 7阅读 0赞

**问题描述：**

通过ElasticSearch搜索时，当搜索的内容过长（大概25个汉字以上）时，会莫名其妙的报错，Elasticsearch exception \[type=search\_phase\_execution\_exception, reason=all shards failed\]。

**解决过程：**

到网上搜索一番后，尝试了很多种方案，终于找到了一个解决方案，如下：

searchSourceBuilder.trackScores(true)
    .sort(SortBuilders.fieldSort("_score").order(SortOrder.DESC))
    .sort(SortBuilders.fieldSort("timestamp").order(SortOrder.ASC).unmappedType("date"));

发现增加了根据timestamp排序，该问题就不会出现。但是感觉很奇怪，这和排序有什么关系呢？！

后来，一次在重启elasticsearch后，查看elasticsearch时，发现了一段错误，如下：

org.elasticsearch.transport.RemoteTransportException: [node-1][10.1.2.3:9300][indices:data/read/search[phase/query]]
    Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query]
            at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:306) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:114) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.2.0.jar:7.2.0]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221]
            at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221]
    Caused by: java.lang.NullPointerException
            at org.apache.lucene.search.DisjunctionMaxScorer.setMinCompetitiveScore(DisjunctionMaxScorer.java:116) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.ScoreCachingWrappingScorer.setMinCompetitiveScore(ScoreCachingWrappingScorer.java:59) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.TopScoreDocCollector.updateMinCompetitiveScore(TopScoreDocCollector.java:242) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:78) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.elasticsearch.common.lucene.MinimumScoreCollector.collect(MinimumScoreCollector.java:57) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:263) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:214) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:652) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.XIndexSearcher.search(XIndexSearcher.java:44) ~[elasticsearch-7.2.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:177) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:271) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:114) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.2.0.jar:7.2.0]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_221]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_221]
            at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_221]
    [2020-04-03T16:23:26,871][DEBUG][o.e.a.s.TransportSearchAction] [node-1] All shards failed for phase: [query]
    org.elasticsearch.ElasticsearchException$1: null
            at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:639) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:137) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:259) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:105) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.InitialSearchPhase.access$200(InitialSearchPhase.java:50) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:273) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:59) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:441) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1111) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1223) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1197) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:60) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:56) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:70) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:64) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.2.0.jar:7.2.0]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_221]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_221]
            at java.lang.Thread.run(Thread.java:748) [?:1.8.0_221]
    Caused by: java.lang.NullPointerException
            at org.apache.lucene.search.DisjunctionMaxScorer.setMinCompetitiveScore(DisjunctionMaxScorer.java:116) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.ScoreCachingWrappingScorer.setMinCompetitiveScore(ScoreCachingWrappingScorer.java:59) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.TopScoreDocCollector.updateMinCompetitiveScore(TopScoreDocCollector.java:242) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:78) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.elasticsearch.common.lucene.MinimumScoreCollector.collect(MinimumScoreCollector.java:57) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:263) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:214) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:652) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.apache.lucene.search.XIndexSearcher.search(XIndexSearcher.java:44) ~[elasticsearch-7.2.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:177) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.0.0.jar:8.0.0 2ae4746365c1ee72a0047ced7610b2096e438979 - jimczi - 2019-03-08 11:58:55]
            at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:271) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:114) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.2.0.jar:7.2.0]
            at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.2.0.jar:7.2.0]
            ... 8 more

而这段错误就是在查询了很多字之后报出的！在Caused by后面，看到了空指针异常，带着这些关键词去网上一搜，很多找到了答案：这是由于设置最小分数值引起的，而这又是es自身的bug！通过描述，说是该错误会在7.2.2版本修复，而我在网上只看到了7.2.1的版本，在尝试将maven中的依赖改为7.2.1版本后，再次尝试，发现世界都安静了！

**问题反思：**

碰到该类问题，最先看的就应该是es本身的日志，那里才会有问题的根源！

参考文章：

[https://github.com/elastic/elasticsearch/issues/43497][https_github.com_elastic_elasticsearch_issues_43497]

[https://github.com/elastic/elasticsearch/pull/43527][https_github.com_elastic_elasticsearch_pull_43527]

[https_github.com_elastic_elasticsearch_issues_43497]: https://github.com/elastic/elasticsearch/issues/43497
[https_github.com_elastic_elasticsearch_pull_43527]: https://github.com/elastic/elasticsearch/pull/43527