CDH 版本Hadoop Yarn配置容量调度器（默认是公平调度器）

蔚落 2022-09-11 05:25 371阅读 0赞

`配置`中进行如下调整：  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5a2m5Lqu57yW56iL5omL6K6w_size_20_color_FFFFFF_t_70_g_se_x_16]

其实就是在在yarn-site.xm中添加`yarn.resourcemanager.scheduler.class`，修改成容量调度器：

<property>
        <name>yarn.resourcemanager.scheduler.class</name>
        <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
    </property>

配置中搜索`yarn.scheduler.capacity.root.queues`，以`XML格式查看`，导入以下配置后重启yarn：  
![在这里插入图片描述][watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5a2m5Lqu57yW56iL5omL6K6w_size_20_color_FFFFFF_t_70_g_se_x_16 1]

修改后的配置：

<configuration>
    <property>
        <name>yarn.scheduler.capacity.maximum-applications</name>
        <value>10000</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.maximum-am-resource-percent</name>
        <value>0.1</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.resource-calculator</name>
        <value>org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.queues</name>
        <value>default,dmgeo</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.capacity</name>
        <value>40</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.capacity</name>
        <value>60</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.user-limit-factor</name>
        <value>1</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.user-limit-factor</name>
        <value>1</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.maximum-capacity</name>
        <value>100</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.maximum-capacity</name>
        <value>100</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.state</name>
        <value>RUNNING</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.state</name>
        <value>RUNNING</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
        <value>*</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.acl_submit_applications</name>
        <value>*</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
        <value>*</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.acl_administer_queue</name>
        <value>*</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.default.acl_application_max_priority</name>
        <value>*</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.root.dmgeo.acl_application_max_priority</name>
        <value>*</value>
      </property>
       <property>
         <name>yarn.scheduler.capacity.root.default.maximum-application-lifetime </name>
         <value>-1</value>
       </property>
       <property>
         <name>yarn.scheduler.capacity.root.dmgeo.maximum-application-lifetime </name>
         <value>-1</value>
       </property>
       <property>
         <name>yarn.scheduler.capacity.root.default.default-application-lifetime </name>
         <value>-1</value>
       </property>
       <property>
         <name>yarn.scheduler.capacity.root.dmgeo.default-application-lifetime </name>
         <value>-1</value>
       </property>
      <property>
        <name>yarn.scheduler.capacity.node-locality-delay</name>
        <value>40</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.rack-locality-additional-delay</name>
        <value>-1</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.queue-mappings</name>
        <value></value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.queue-mappings-override.enable</name>
        <value>false</value>
      </property>
      <property>
        <name>yarn.scheduler.capacity.per-node-heartbeat.maximum-offswitch-assignments</name>
        <value>1</value>
      </property>
    </configuration>

默认的配置：

<configuration><property><name>yarn.scheduler.capacity.root.queues</name><value>default</value></property><property><name>yarn.scheduler.capacity.root.capacity</name><value>100</value></property><property><name>yarn.scheduler.capacity.root.default.capacity</name><value>100</value></property></configuration>

参考链接：https://www.cnblogs.com/hongfeng2019/p/11477583.html

[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5a2m5Lqu57yW56iL5omL6K6w_size_20_color_FFFFFF_t_70_g_se_x_16]: /images/20220828/ae5018c7dc0b45c4984d7622532be72b.png
[watermark_type_ZHJvaWRzYW5zZmFsbGJhY2s_shadow_50_text_Q1NETiBA5a2m5Lqu57yW56iL5omL6K6w_size_20_color_FFFFFF_t_70_g_se_x_16 1]: /images/20220828/44a43ae3b20f4758b0626fa447b1c0d8.png