'Exceeded memory limit for $group, but didn't allow external sort. Pass allowDiskUse 解决 java 实例-蒲公英云

'Exceeded memory limit for $group, but didn't allow external sort. Pass allowDiskUse 解决 java 实例

蔚落 2023-06-30 11:58 12阅读 0赞

报错

ERROR[taskExecutorSchedule-13]o.s.s.s.TaskUtils$LoggingErrorHandler.handleError:95 -Unexpected error occurred in scheduled task
org.springframework.data.mongodb.UncategorizedMongoDbException: Command failed with error 16945 (Location16945): ‘Exceeded memory limit for $group, but didn’t allow external sort. Pass allowDiskUse:true to opt in.’ on server 192.168.31.237:29050. The full response is {“ok”: 0.0, “errmsg”: “Exceeded memory limit for $group, but didn’t allow external sort. Pass allowDiskUse:true to opt in.”, “code”: 16945, “codeName”: “Location16945”, “$clusterTime”: {“clusterTime”: {“$timestamp”: {“t”: 1579246265, “i”: 84}}, “signature”: {“hash”: {“$binary”: “CRqmnSmfpTbHTIi9A6kWwxq6vWE=”, “$type”: “00”}, “keyId”: {“$numberLong”: “6756819692251250707”}}}, “operationTime”: {“$timestamp”: {“t”: 1579246265, “i”: 1}}}; nested exception is com.mongodb.MongoCommandException: Command failed with error 16945 (Location16945): ‘Exceeded memory limit for $group, but didn’t allow external sort. Pass allowDiskUse:true to opt in.’ on server 192.168.31.237:29050. The full response is {“ok”: 0.0, “errmsg”: “Exceeded memory limit for $group, but didn’t allow external sort. Pass allowDiskUse:true to opt in.”, “code”: 16945, “codeName”: “Location16945”, “$clusterTime”: {“clusterTime”: {“$timestamp”: {“t”: 1579246265, “i”: 84}}, “signature”: {“hash”: {“$binary”: “CRqmnSmfpTbHTIi9A6kWwxq6vWE=”, “$type”: “00”}, “keyId”: {“$numberLong”: “6756819692251250707”}}}, “operationTime”: {“$timestamp”: {“t”: 1579246265, “i”: 1}}}
at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:138)
at org.springframework.data.mongodb.core.MongoTemplate.potentiallyConvertRuntimeException(MongoTemplate.java:2902)
at org.springframework.data.mongodb.core.MongoTemplate.execute(MongoTemplate.java:587)
at org.springframework.data.mongodb.core.MongoTemplate.doAggregate(MongoTemplate.java:2172)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2141)
at org.springframework.data.mongodb.core.MongoTemplate.aggregate(MongoTemplate.java:2031)
at com.meerkat.common.scan.mongo.MongoDBDaoImpl.aggregate(MongoDBDaoImpl.java:148)
at com.meerkat.common.scan.mongo.MongoDBDaoImplFastClassBySpringCGLIBa8e0568a.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:769)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:747)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)

原因：

查询结果超过内存，需要设置 allowDiskUse(true)

public Long countRecord(Long startTime, Long endTime) throws Exception {
        Criteria criteria = Criteria.where("uptime").gte(startTime).lte(endTime);
        AggregationOptions aggregationOptions = AggregationOptions.builder().allowDiskUse(true).build();
        TypedAggregation<返回对象> agg = Aggregation.newAggregation(返回对象.class,
                Aggregation.match(criteria),
                Aggregation.group("imsi").count().as("count")
                        .first("imsi").as("imsi")
        ).withOptions(aggregationOptions);
        AggregationResults<返回对象> aggregate = mongoDBDao.aggregate(agg, 查询对象.class, 返回对象.class);
        if (aggregate.getMappedResults().size() > 0) {
            return Long.valueOf(aggregate.getMappedResults().size());
        }
        return 0L;
    }