nginx upstream 使用 keepalive 保持后端连接池方案

骑猪看日落 2022-10-18 01:28 550阅读 0赞

测试环境

/usr/local/nginx http 80

/usr/local/nginx.2 http 81 代理到 80端口

1. 基准性能

/usr/local/nginx/conf/nginx.conf配置

worker_processes  1;
    events {
        worker_connections  1024;
    }
    http {
        include       mime.types;
        default_type  application/octet-stream;
        sendfile        on;
        keepalive_timeout  65;
        server {
            listen       80;
            server_name  localhost;
            location / {
                root   html;
                index  index.html index.htm;
            }
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
        }
    }

Apache Bench测试结果

yeqiang@yeqiang-PC:/usr/local/nginx$ ab -n 100000 -c 30 http://127.0.0.1/
    This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
    Licensed to The Apache Software Foundation, http://www.apache.org/
    
    Benchmarking 127.0.0.1 (be patient)
    Completed 10000 requests
    Completed 20000 requests
    Completed 30000 requests
    Completed 40000 requests
    Completed 50000 requests
    Completed 60000 requests
    Completed 70000 requests
    Completed 80000 requests
    Completed 90000 requests
    Completed 100000 requests
    Finished 100000 requests
    
    
    Server Software:        nginx/1.21.0
    Server Hostname:        127.0.0.1
    Server Port:            80
    
    Document Path:          /
    Document Length:        612 bytes
    
    Concurrency Level:      30
    Time taken for tests:   2.843 seconds
    Complete requests:      100000
    Failed requests:        0
    Total transferred:      84500000 bytes
    HTML transferred:       61200000 bytes
    Requests per second:    35174.43 [#/sec] (mean)
    Time per request:       0.853 [ms] (mean)
    Time per request:       0.028 [ms] (mean, across all concurrent requests)
    Transfer rate:          29025.78 [Kbytes/sec] received
    
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0    0   0.0      0       2
    Processing:     0    1   0.1      1       3
    Waiting:        0    1   0.1      1       3
    Total:          1    1   0.1      1       3
    
    Percentage of the requests served within a certain time (ms)
      50%      1
      66%      1
      75%      1
      80%      1
      90%      1
      95%      1
      98%      1
      99%      1
     100%      3 (longest request)

/usr/local/nginx.2/conf/nginx.conf

worker_processes  1;
    events {
        worker_connections  1024;
    }
    http {
        include       mime.types;
        default_type  application/octet-stream;
        sendfile        on;
        keepalive_timeout  65;
    upstream backend-server {
            server 127.0.0.1;
    }
        server {
            listen       81;
            server_name  localhost;
            location / {
                    proxy_pass http://backend-server/;
            }
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
        }
    }

Apache Bench 测试结果

yeqiang@yeqiang-PC:/usr/local/nginx.2$ ab -n 100000 -c 30 http://127.0.0.1:81/
    This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
    Licensed to The Apache Software Foundation, http://www.apache.org/
    
    Benchmarking 127.0.0.1 (be patient)
    Completed 10000 requests
    Completed 20000 requests
    Completed 30000 requests
    Completed 40000 requests
    Completed 50000 requests
    Completed 60000 requests
    Completed 70000 requests
    Completed 80000 requests
    Completed 90000 requests
    Completed 100000 requests
    Finished 100000 requests
    
    
    Server Software:        nginx/1.21.0
    Server Hostname:        127.0.0.1
    Server Port:            81
    
    Document Path:          /
    Document Length:        612 bytes
    
    Concurrency Level:      30
    Time taken for tests:   5.794 seconds
    Complete requests:      100000
    Failed requests:        0
    Total transferred:      84500000 bytes
    HTML transferred:       61200000 bytes
    Requests per second:    17259.96 [#/sec] (mean)
    Time per request:       1.738 [ms] (mean)
    Time per request:       0.058 [ms] (mean, across all concurrent requests)
    Transfer rate:          14242.84 [Kbytes/sec] received
    
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0    0   0.0      0       1
    Processing:     0    2   0.1      2       5
    Waiting:        0    2   0.1      2       5
    Total:          1    2   0.1      2       5
    
    Percentage of the requests served within a certain time (ms)
      50%      2
      66%      2
      75%      2
      80%      2
      90%      2
      95%      2
      98%      2
      99%      2
     100%      5 (longest request)

可以看到，经过一层代理后，性能下降接近一半。

其主要原因是TCP链接的创建资源消耗

采用keepalive参数实现后端的连接池

/usr/local/nginx.2/nginx.conf

worker_processes  1;
    events {
        worker_connections  1024;
    }
    http {
        include       mime.types;
        default_type  application/octet-stream;
        sendfile        on;
        keepalive_timeout  65;
    upstream backend-server {
            server 127.0.0.1;
            keepalive 100;
    }
        server {
            listen       81;
            server_name  localhost;
            location / {
                    proxy_pass http://backend-server/;
                    proxy_http_version 1.1;
                    proxy_set_header Connection "keep-alive";
            }
            error_page   500 502 503 504  /50x.html;
            location = /50x.html {
                root   html;
            }
        }
    }

Apache Bench测试结果

yeqiang@yeqiang-PC:/usr/local/nginx.2$ ab -n 100000 -c 30 http://127.0.0.1:81/
    This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
    Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
    Licensed to The Apache Software Foundation, http://www.apache.org/
    
    Benchmarking 127.0.0.1 (be patient)
    Completed 10000 requests
    Completed 20000 requests
    Completed 30000 requests
    Completed 40000 requests
    Completed 50000 requests
    Completed 60000 requests
    Completed 70000 requests
    Completed 80000 requests
    Completed 90000 requests
    Completed 100000 requests
    Finished 100000 requests
    
    
    Server Software:        nginx/1.21.0
    Server Hostname:        127.0.0.1
    Server Port:            81
    
    Document Path:          /
    Document Length:        612 bytes
    
    Concurrency Level:      30
    Time taken for tests:   3.090 seconds
    Complete requests:      100000
    Failed requests:        0
    Total transferred:      84500000 bytes
    HTML transferred:       61200000 bytes
    Requests per second:    32363.44 [#/sec] (mean)
    Time per request:       0.927 [ms] (mean)
    Time per request:       0.031 [ms] (mean, across all concurrent requests)
    Transfer rate:          26706.16 [Kbytes/sec] received
    
    Connection Times (ms)
                  min  mean[+/-sd] median   max
    Connect:        0    0   0.0      0       1
    Processing:     0    1   0.1      1       5
    Waiting:        0    1   0.1      1       5
    Total:          0    1   0.1      1       5
    
    Percentage of the requests served within a certain time (ms)
      50%      1
      66%      1
      75%      1
      80%      1
      90%      1
      95%      1
      98%      1
      99%      1
     100%      5 (longest request)

通过netstat指令可以看出到80端口的连接数明细减少，同时80 端口TCP TIME\_WAIT状态也明细下降。

总结

常规代理，吞吐量为代理前的49.07%

连接池方案代理，吞吐量为代理前的92.00%，比常规代理高出42.93%