Yarn 常用命令

￡神魔★判官ぃ 2022-01-26 15:59 725阅读 0赞

1.  **yarn命令**
    
    **用法:**
    
        yarn [--config confdir] COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>resourcemanager -format-state-store</td> 
       <td>格式化RMStateStore,只有当历史作业不再需要时，才需要格式化</td> 
      </tr> 
      <tr> 
       <td>resourcemanager</td> 
       <td>启动 ResourceManager。用 -format-state-store for 删除 RMStateStore； 使用 -remove-application-from-state-store 从RMStateStore中删除应用程序</td> 
      </tr> 
      <tr> 
       <td>nodemanager</td> 
       <td>在每个节点启动 NodeManager</td> 
      </tr> 
      <tr> 
       <td>timelineserver</td> 
       <td>启动 timeline server</td> 
      </tr> 
      <tr> 
       <td>rmadmin</td> 
       <td>管理工具</td> 
      </tr> 
      <tr> 
       <td>version</td> 
       <td>版本信息</td> 
      </tr> 
      <tr> 
       <td>jar </td> 
       <td>运行 jar 文件</td> 
      </tr> 
      <tr> 
       <td>application</td> 
       <td>查看应用程序状态，或终止应用程序</td> 
      </tr> 
      <tr> 
       <td>applicationattempt</td> 
       <td>打印应用程序尝试报告</td> 
      </tr> 
      <tr> 
       <td>container</td> 
       <td>打印 Container 尝试报告</td> 
      </tr> 
      <tr> 
       <td>node</td> 
       <td>打印节点尝试报告</td> 
      </tr> 
      <tr> 
       <td>queue</td> 
       <td>打印对列信息，只有一个参数</td> 
      </tr> 
      <tr> 
       <td>logs</td> 
       <td>主要用于收集作业日志，注意要等程序结束才能查看日志</td> 
      </tr> 
      <tr> 
       <td>classpath</td> 
       <td>打印 Hadoop 相关依赖和其他依赖库</td> 
      </tr> 
      <tr> 
       <td>daemonlog</td> 
       <td>针对指定的守护进程获取/设置日志优先级</td> 
      </tr> 
      <tr> 
       <td>top</td> 
       <td>启动集群使用工具</td> 
      </tr> 
      <tr> 
       <td>CLASSNAME</td> 
       <td>启动 CLASSNAME 类</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn version
        Hadoop 2.6.0-cdh5.12.1
        Subversion http://github.com/cloudera/hadoop -r 520d8b072e666e9f21d645ca6a5219fc37535a52 
        Compiled by jenkins on 2017-08-24T16:43Z
        Compiled with protoc 2.5.0
        From source with checksum de51bf9693ab9426379a1cd28142cea0
        This command was run using /opt/cloudera/parcels/CDH-5.12.1-1.cdh5.12.1.p0.3/jars/hadoop-common-2.6.0-cdh5.12.1.jar
2.  **yarn rmadmin 命令**
    
    **用法:**
    
        Usage: yarn rmadmin COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-refreshQueues</td> 
       <td>重载队列的ACL、状态和调度器特定的属性，ResourceManager将重新加载mapred-queues配置文件</td> 
      </tr> 
      <tr> 
       <td>-refreshNodes [-g [timeout in seconds] -client</td> 
       <td>server]</td> 
      </tr> 
      <tr> 
       <td>-refreshSuperUserGroupsConfiguration</td> 
       <td>刷新用户组的配置</td> 
      </tr> 
      <tr> 
       <td>-refreshUserToGroupsMappings</td> 
       <td>刷新用户到组的映射</td> 
      </tr> 
      <tr> 
       <td>-refreshAdminAcls</td> 
       <td>刷新 ResourceManager 的 ACL 管理</td> 
      </tr> 
      <tr> 
       <td>-refreshServiceAcl</td> 
       <td>ResourceManager 重载服务级别的授权文件</td> 
      </tr> 
      <tr> 
       <td>-getGroups [username]</td> 
       <td>获取指定用户所属的组</td> 
      </tr> 
      <tr> 
       <td>-help [cmd]</td> 
       <td>显示指定命令的帮助，没有指定，则显示全部指令帮助</td> 
      </tr> 
      <tr> 
       <td>-addToClusterNodeLabels [label1,label2,label3] (label splitted by “,”)</td> 
       <td>添加标签</td> 
      </tr> 
      <tr> 
       <td>-removeFromClusterNodeLabels [label1,label2,label3] (label splitted by “,”)</td> 
       <td>移除标签</td> 
      </tr> 
      <tr> 
       <td>-replaceLabelsOnNode [label1,label2,label3] (label splitted by “,”)</td> 
       <td>替换标签</td> 
      </tr> 
      <tr> 
       <td>directlyAccessNodeLabelStore</td> 
       <td></td> 
      </tr> 
      <tr> 
       <td>transitionToActive [–forceactive]</td> 
       <td></td> 
      </tr> 
      <tr> 
       <td>transitionToStandby </td> 
       <td></td> 
      </tr> 
      <tr> 
       <td>failover [–forcefence] [–forceactive] </td> 
       <td></td> 
      </tr> 
      <tr> 
       <td>getServiceState </td> 
       <td></td> 
      </tr> 
      <tr> 
       <td>checkHealth </td> 
       <td></td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn rmadmin -getGroups hdfs
        hdfs : hdfs hadoop supergroup
3.  **yarn application 命令**
    
    **用法:**
    
        Usage: yarn application COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-appStates </td> 
       <td>与 -list 配合使用，基于应用程序的状态来过滤,多个状态用逗号分隔状态。有效的状态: ALL,NEW,NEW_SAVING,SUBMITTED,ACCEPTED,RUNNING,FINISHED,FAILED,KILLED</td> 
      </tr> 
      <tr> 
       <td>-appTypes </td> 
       <td>与 -list 配合使用，基于应用程序的类型来过滤，多个应用程序用逗号分隔</td> 
      </tr> 
      <tr> 
       <td>-help</td> 
       <td>帮助列表</td> 
      </tr> 
      <tr> 
       <td>-kill </td> 
       <td>杀死指定的应用程序</td> 
      </tr> 
      <tr> 
       <td>-list</td> 
       <td>从 ResourceManager 获取应用程序列表. 使用 -appTypes 基于应用程序的类型来过滤 使用 -appStates 基于应用程序的状态来过滤</td> 
      </tr> 
      <tr> 
       <td>-movetoqueue </td> 
       <td>将应用程序移动到不同的队列</td> 
      </tr> 
      <tr> 
       <td>-queue </td> 
       <td>与 movetoqueue配合使用，移到指定队列</td> 
      </tr> 
      <tr> 
       <td>-status </td> 
       <td>获取应用程序的状态</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn application -status application_1557221110563_0246
        ...
        Application Report : 
        	Application-Id : application_1557221110563_0246
        	Application-Name : INSERT OVERWRITE TABLE hdsc...hdsc.base_user(Stage-1)
        	Application-Type : MAPREDUCE
        	User : admin
        	Queue : default
        	Start-Time : 1557366149859
        	Finish-Time : 1557366154321
        	Progress : 100%
        	State : KILLED
        	Final-State : KILLED
        	Tracking-URL : null
        	RPC Port : -1
        	AM Host : N/A
        	Aggregate Resource Allocation : 3111 MB-seconds, 2 vcore-seconds
        	Log Aggregation Status : NOT_START
        	Diagnostics : Application killed by user.
4.  **yarn applicationattempt 命令**
    
    **用法:**
    
        Usage: yarn applicationattempt COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-help</td> 
       <td>帮助列表</td> 
      </tr> 
      <tr> 
       <td>-list </td> 
       <td>获取应用程序尝试的列表</td> 
      </tr> 
      <tr> 
       <td>-status </td> 
       <td>获取应用程序尝试的状态</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn applicationattempt -list application_1557221110563_0246
        ...
        Total number of application attempts :1
        		 ApplicationAttempt-Id	               State	                    AM-Container-Id	                       Tracking-URL
        appattempt_1557221110563_0246_000001	              KILLED	container_e52_1557221110563_0246_01_000002	http://slave1:8088/proxy/application_1557221110563_0246/
        
        [root@cdh01:~]# yarn applicationattempt -status appattempt_1557221110563_0246_000001
        Application Attempt Report : 
        ApplicationAttempt-Id : appattempt_1557221110563_0246_000001
        State : KILLED
        AMContainer : container_e52_1557221110563_0246_01_000002
        Tracking-URL : http://slave1:8088/proxy/application_1557221110563_0246/
        RPC Port : -1
        AM Host : N/A
        Diagnostics : Attempt recovered after RM restart
5.  **yarn container 命令**
    
    **用法:**
    
        Usage: yarn container COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-help</td> 
       <td>帮助列表</td> 
      </tr> 
      <tr> 
       <td>-list </td> 
       <td>获取 Container 列表</td> 
      </tr> 
      <tr> 
       <td>-status </td> 
       <td>获取 Container 的状态</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn container -list appattempt_1557221110563_0369_000001
        ...
        Total number of containers :0
        				  Container-Id	          Start Time	         Finish Time	               State	                Host	                            LOG-URL
6.  **yarn node 命令**
    
    **用法:**
    
        yarn node COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-all</td> 
       <td>与 -list 打印所有节点的报告</td> 
      </tr> 
      <tr> 
       <td>-list</td> 
       <td>获取所有 RUNNING 状态的节点。 支持 -states 选项过滤指定的状态。节点状态包括 NEW,RUNNING,UNHEALTH,DECOMMISSIONED,LOST 和 REBOOTED</td> 
      </tr> 
      <tr> 
       <td>-states </td> 
       <td>与 -list 配合使用，用逗号分隔节点状态，只显示这些状态的节点信息</td> 
      </tr> 
      <tr> 
       <td>-status </td> 
       <td>获取指点节点的状态</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn node -all -list
        ...
        Total Nodes:5
           Node-Id	     Node-State	Node-Http-Address	Number-of-Running-Containers
        cdh01:8041	        RUNNING	cdh01:8042	                  0
        cdh02:8041	        RUNNING	cdh02:8042	                  0
        cdh03:8041	        RUNNING	cdh03:8042	                  0
        cdh04:8041	        RUNNING	cdh04:8042	                  0
        cdh05:8041	        RUNNING	cdh05:8042	                  0
        
        [root@cdh01:~]# yarn node -status cdh01:8041
        Node Report : 
        Node-Id : cdh01:8041
        Rack : /default
        Node-State : RUNNING
        Node-Http-Address : cdh01:8042
        Last-Health-Update : Thu 30/May/19 08:14:31:270CST
        Health-Report : 
        Containers : 0
        Memory-Used : 0MB
        Memory-Capacity : 12288MB
        CPU-Used : 0 vcores
        CPU-Capacity : 4 vcores
        Node-Labels :
7.  **yarn queue 命令**
    
    **用法:**
    
        yarn queue COMMAND
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-help</td> 
       <td>帮助列表</td> 
      </tr> 
      <tr> 
       <td>-status </td> 
       <td>获取队列的状态</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn queue -status default
        ...
        Queue Information : 
        Queue Name : root.default
        	State : RUNNING
        	Capacity : .0%
        	Current Capacity : .0%
        	Maximum Capacity : -100.0%
        	Default Node Label expression : 
        	Accessible Node Labels :
8.  **yarn logs 命令**
    
    **用法:**
    
        yarn logs -applicationId <application ID> [OPTIONS]
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-appOwner </td> 
       <td>指定应用程序的所有者</td> 
      </tr> 
      <tr> 
       <td>-containerId </td> 
       <td>指定运行任务的 Container ID</td> 
      </tr> 
      <tr> 
       <td>-nodeAddress </td> 
       <td>指定运行任务的节点，格式为 host:name</td> 
      </tr> 
     </tbody> 
    </table>
9.  **yarn daemonlog 命令**
    
    **用法:**
    
        yarn daemonlog -getlevel <host:port> <classname> [-protocol (http|https)]
        或
        yarn daemonlog -setlevel <host:port> <classname> <level> [-protocol (http|https)]
    
    <table> 
     <thead> 
      <tr> 
       <th>参数</th> 
       <th>描述</th> 
      </tr> 
     </thead> 
     <tbody> 
      <tr> 
       <td>-getlevel host:port </td> 
       <td>针对指定的守护进程获取日志优先级</td> 
      </tr> 
      <tr> 
       <td>-setlevel host:port [-protocol (http</td> 
       <td>https)</td> 
      </tr> 
     </tbody> 
    </table>
    
    **示例：**
    
        [root@cdh01:~]# yarn daemonlog -getlevel cdh01:50075 org.apache.hadoop.hdfs.server.datanode.DataNode
        root@bigdata-sit-54:~# yarn daemonlog -getlevel bigdata-sit-54:50075 org.apache.hadoop.hdfs.server.datanode.DataNode
        Connecting to http://cdh01:50075/logLevel?log=org.apache.hadoop.hdfs.server.datanode.DataNode
        Submitted Log Name: org.apache.hadoop.hdfs.server.datanode.DataNode
        Log Class: org.apache.commons.logging.impl.Log4JLogger
        Effective Level: INFO
        [root@cdh01:~]# yarn daemonlog -getlevel cdh01:8088 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl
        Connecting to http://cdh01:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl
        Submitted Log Name: org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl
        Log Class: org.apache.commons.logging.impl.Log4JLogger
        Effective Level: INFO
        [root@cdh01:~]# yarn daemonlog -getlevel cdh01:19888 org.apache.hadoop.mapreduce.v2.hs.JobHistory
        Connecting to http://cdh01:19888/logLevel?log=org.apache.hadoop.mapreduce.v2.hs.JobHistory
        Submitted Log Name: org.apache.hadoop.mapreduce.v2.hs.JobHistory
        Log Class: org.apache.commons.logging.impl.Log4JLogger
        Effective Level: INFO