httpclient的两个重要的参数maxPerRoute及MaxTotal

Bertha 。 2021-06-10 20:40 931阅读 0赞

httpclient封装了java中进行http网络请求的底层实现，是一个被广泛使用的组件。

httpclient是支持池化机制的，这两个参数就是表示池化设置的。

服务之间发送http请求常用的有Apache的Fluent以及spring的restTemplate等

Apache的Fluent，以及spring的restTemplate都是对httpclient进行了封装

以Apache的Fluent为例说明

其Executor类

/**
     * An Executor for fluent requests
     * <p/>
     * A {@link PoolingHttpClientConnectionManager} with maximum 100 connections per route and
     * a total maximum of 200 connections is used internally.
     */

最大100 connections per route 以及 最大200个 connection

CONNMGR = new PoolingHttpClientConnectionManager(sfr);
    CONNMGR.setDefaultMaxPerRoute(100);
    CONNMGR.setMaxTotal(200);
    CLIENT = HttpClientBuilder.create().setConnectionManager(CONNMGR).build();

其代码中是写死的

# 这两个参数的含义是什么呢？ #

下面用测试代码说明一下

测试端

public class HttpFluentUtil {
        private Logger logger = LoggerFactory.getLogger(HttpFluentUtil.class);
        private final static int MaxPerRoute = 2;
        private final static int MaxTotal = 4;
     
        final static PoolingHttpClientConnectionManager CONNMGR;
        final static HttpClient CLIENT;
        final static Executor EXECUTOR;
     
     
        static {
            LayeredConnectionSocketFactory ssl = null;
            try {
                ssl = SSLConnectionSocketFactory.getSystemSocketFactory();
            } catch (final SSLInitializationException ex) {
                final SSLContext sslcontext;
                try {
     
                    sslcontext = SSLContext.getInstance(SSLConnectionSocketFactory.TLS);
                    sslcontext.init(null, null, null);
                    ssl = new SSLConnectionSocketFactory(sslcontext);
                } catch (final SecurityException ignore) {
     
                } catch (final KeyManagementException ignore) {
     
                } catch (final NoSuchAlgorithmException ignore) {
    
                }
     
            }
    
            final Registry<ConnectionSocketFactory> sfr = RegistryBuilder.<ConnectionSocketFactory>create()
                    .register("http", PlainConnectionSocketFactory.getSocketFactory())
                    .register("https", ssl != null ? ssl : SSLConnectionSocketFactory.getSocketFactory()).build();
            CONNMGR = new PoolingHttpClientConnectionManager(sfr);
            CONNMGR.setDefaultMaxPerRoute(MaxPerRoute);
            CONNMGR.setMaxTotal(MaxTotal);
            CLIENT = HttpClientBuilder.create().setConnectionManager(CONNMGR).build();
            EXECUTOR = Executor.newInstance(CLIENT);
        }
     
     
        public static String Get(String uri, int connectTimeout, int socketTimeout) throws IOException {
            return EXECUTOR.execute(Request.Get(uri).connectTimeout(connectTimeout).socketTimeout(socketTimeout))
                    .returnContent().asString();
        }
     
     
        public static String Post(String uri, StringEntity stringEntity, int connectTimeout, int socketTimeout)
                throws IOException {
            return EXECUTOR.execute(Request.Post(uri).socketTimeout(socketTimeout)
                    .addHeader("Content-Type", "application/json").body(stringEntity)).returnContent().asString();
        }
     
        public static void main(String[] args) {
            HttpUtil httpUtil = new HttpUtil();
            String url = "http://localhost:9064/app/test"; // 服务端sleep 5秒再返回
            for (int i = 0; i < 5; i++) { // MaxPerRoute若设置为2，则5线程分3组返回（2、2、1），共15秒
     
                new Thread(new Runnable() {
                    @Override
                    public void run() {
                        try {
                            String result = HttpFluentUtil.Get(url, 2000, 2000);
                            System.out.println(result);
                        } catch (IOException e) {
                            e.printStackTrace();
                        }
     
                    }
                }).start();
     
            }
        }
    }

服务器端

很简单的springmvc

@GetMapping(value="test")
    public String test() throws InterruptedException {
        Thread.sleep(1000);
        return "1";
    }

测试1：测试端MaxPerRoute=5 MaxTotal=4

服务器端结果

![20190705085157189.png][]

可以看到先接收4个请求，处理完成后，再接收下一次剩余的1个请求。即其一次最多接收MaxTotal次请求

测试2：测试端MaxPerRoute=2 MaxTotal=5

服务器端结果

![20190705085220192.png][]

可以看到接收2个请求，2个请求，1个请求，即说明maxPerRoute意思是某一个服务每次能并行接收的请求数量

# 什么场景下要设置？ #

知道了两个参数的含义，那么在什么情况下要对这两个参数进行设置呢？

比如说下面的场景

服务1要通过Fluent调用服务2的接口。服务1发送了400个请求，但由于Fluent默认只支持maxPerRoute=100，MaxTotal=200，比如接口执行时间为500ms，由于maxPerRoute=100，所以要分为100,100,100,100分四批来执行，全部执行完成需要2000ms。而如果maxPerRoute设置为400，全部执行完需要500ms。在这种情况下（提供并发能力时）就要对这两个参数进行设置了。

# 设置的方法 #

Apache Fluent可以使用上面测试的HttpFluentUtil工具类来执行请求

RestTemplate类似使用下面的方式

@Bean
    public HttpClient httpClient() {
       Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("http", PlainConnectionSocketFactory.getSocketFactory())
                .register("https", SSLConnectionSocketFactory.getSocketFactory())
                .build();
        PoolingHttpClientConnectionManager connectionManager = new PoolingHttpClientConnectionManager(registry);
        connectionManager.setMaxTotal(restTemplateProperties.getMaxTotal());
        connectionManager.
                       setDefaultMaxPerRoute(restTemplateProperties.getDefaultMaxPerRoute());
        connectionManager.
             setValidateAfterInactivity(restTemplateProperties.getValidateAfterInactivity());
     
        RequestConfig requestConfig = RequestConfig.custom()
                .setSocketTimeout(restTemplateProperties.getSocketTimeout())
                .setConnectTimeout(restTemplateProperties.getConnectTimeout())
               .setConnectionRequestTimeout(restTemplateProperties.getConnectionRequestTimeout()).build();
     
        return HttpClientBuilder.create()
                .setDefaultRequestConfig(requestConfig)
                .setConnectionManager(connectionManager)
                .build();
    }
    
    
    @Bean
    public ClientHttpRequestFactory httpRequestFactory() {
        return new HttpComponentsClientHttpRequestFactory(httpClient());
    }
    
    
    @Bean
    public RestTemplate restTemplate() {
        return new RestTemplate(httpRequestFactory());
    }

其中RestTemplateProperties通过配置文件来配置

max-total

default-max-per-route

connect-timeout  获取连接超时

connection-request-timeout  请求超时

socket-timeout  读超时

[20190705085157189.png]: /images/20210527/7cfa5a0ef1f140eaa44ea95a1b2b670d.png
[20190705085220192.png]: /images/20210527/2bab73753a364978a04c6d5fe82f60cc.png