限制 kubernetes 跑的业务容器所起的进程数

Bertha 。 2022-06-01 12:29 92阅读 0赞

#### 现象描述 ####

发现业务容器，不停地起线程，现象如下:

[root@master ~]# kubectl logs appserv-2336744132-qhd2d -n xiaojiang
    ... 
    WARNING: Bean creation exception on non-lazy FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'testService': Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: unable to create new native thread
    Jan 29, 2018 10:25:40 AM org.springframework.beans.factory.support.DefaultListableBeanFactory getTypeForFactoryBean
    WARNING: Bean creation exception on non-lazy FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'testService': Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: unable to create new native thread
    Jan 29, 2018 10:25:40 AM org.springframework.beans.factory.support.DefaultListableBeanFactory getTypeForFactoryBean
    WARNING: Bean creation exception on non-lazy FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'testService': Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: unable to create new native thread
    Jan 29, 2018 10:25:40 AM org.springframework.beans.factory.support.DefaultListableBeanFactory getTypeForFactoryBean
    WARNING: Bean creation exception on non-lazy FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'testService': Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: unable to create new native thread

这种情况下，会把所在的节点资源耗光，从而触发各种问题的产生

##### 处理方法 #####

限制docker 容器所起的进程数，执行以下命令就可以解决此类问题

#!/bin/bash 
    MAX_PIDS=1000
    CGROUP_PIDS=/sys/fs/cgroup/pids
    SYSTEM_SLICE=/sys/fs/cgroup/pids/system.slice/
    
    DOCKER=/sys/fs/cgroup/pids/docker/*
    KUBEPODS=/sys/fs/cgroup/pids/kubepods
    
    echo "set @ `date`">/tmp/pids_setting.latest
    
    if [ ! -d $CGROUP_PIDS ];then
        mkdir -p $CGROUP_PIDS
        mount -t cgroup -o pids none $CGROUP_PIDS
    fi
    
    for i in `find $DOCKER -type d`;
    do
        echo $MAX_PIDS > $i/pids.max
    done
    
    for i in `find $KUBEPODS -name "pod*" -type d`;
    do
        echo $MAX_PIDS > $i/pids.max
    done

更多精彩分享博客:  
[http://www.lijiaocn.com/tags/kubernetes.html][http_www.lijiaocn.com_tags_kubernetes.html]

[http_www.lijiaocn.com_tags_kubernetes.html]: http://www.lijiaocn.com/tags/kubernetes.html