hadoop 配置完善（二）

╰半夏微凉° 2024-02-18 19:52 24阅读 0赞

**hadoop主机配置**  
优先顺序：  
1)share目录下默认的配置文件（最低)  
2)etc目录下配置文件（其次）  
3）命令执行(最高）  
执行案例之前必须启动6个进程哦！  
![20180426160229312][]  
案例：删除uber模式启动

\#bin/hadoop jar share/hadoop/mapreduce/hadoop-mapredude-examples-2.7.6.jar wordcount -D mapreduce.job.ubertask.enable=false /words /test01

![2018042616051867][]

执行后查看：

![20180426160503902][]

在启动时候查看：

![2018042616030124][]

*将localhsot修改成主机名，【0.0.0.0】也修改成主机名：*  
 （1）NameNode/SecondNameNode

http://hadoop.apache.org/docs/r2.7.6/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

![20180426160647154][]

dfs.namenode.secondary.http-address=hlx:50090  
 dfs.namenode.secondary.https-address  这两个配置一个就OK！

**hdfs-site.xml文件**

<property>
            <name>dfs.namenode.secondary.http-address</name>
            <value>hlx:50090</value>
        </property>
    	
    	 <property>
            <name>dfs.namenode.http-address</name>
            <value>hlx:50070</value>
        </property>

(2)DataNode/NodeManager

在slaves修改

![2018042616082955][]

(3)JobHistoryServer

http://hadoop.apache.org/docs/r2.7.6/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

![20180426160847721][]

**mapred-site.xml文件**

<property>
            <name>mapreduce.jobhistory.webapp.address</name>
            <value>hlx:19888</value>
        </property>
    	
    	<property>
            <name>mapreduce.jobhistory.address</name>
            <value>hlx:10020</value>
        </property>

(4)ResourceManager

http://hadoop.apache.org/docs/r2.7.6/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

![20180426160957114][]

**yarn-site.xml文件**

<property>
            <name>yarn.resourcemanager.hostname</name>
            <value>hlx</value>
        </property>
    	
    	 <property>
            <name>yarn.resourcemanager.webapp.address</name>
            <value>hlx:8088</value>
        </property>
    	
    	 <property>
            <name>yarn.resourcemanager.address</name>
            <value>hlx:8032</value>
        </property>

(5)webapplication如何启动 (代理进程,防止攻击，系统防火墙）  
 \#sbin/yarn-daemon.sh start proxyserver

\#jps

![20180426161053411][]

没有启动,查看日志分析  
 \#ls logs/

![20180426161106155][]

\#more logs/yarn-root-proxyserver-hadoop.log

![20180426161220583][]

按Enter查看,有个错误！

![20180426161156865][]

http://hadoop.apache.org/docs/r2.7.6/hadoop-yarn/hadoop-yarn-common/yarn-default.xml查找

![2018042616130673][]

**yarn-site.xml文件**

<property>
            <name>yarn.web-proxy.address</name>
            <value>hlx:8888</value>
        </property>

最后查看:  
（1）先停止dfs 再重新启动dfs  
（2）先停止yarn 再重新启动yarn

（3）先停止jobhistory 再重新启动jobhistory

![20180426161442660][]

再依次启动查看，现在所有的都在主机hlx 中

![20180426161509885][]

![20180426161606282][]

(4) \#sbin/yarn-daemon.sh start proxyserver

![20180426161629307][]

[20180426160229312]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/ef6676a08d0f4090a7f0c27f24959a4d.png
[2018042616051867]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/ea26e49beb3541a48f73baf23d6ab66d.png
[20180426160503902]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/8371dc88167745589ce9233155aa948c.png
[2018042616030124]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/0f6e0506277f451dbdfa438f59c44e9f.png
[20180426160647154]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/e45ec62621e64ade922fb2988f623845.png
[2018042616082955]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/977aebac26734f4c8f064aa8d8399a18.png
[20180426160847721]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/c0299ad92f624374bcad4e5adce64fa7.png
[20180426160957114]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/2fa0972ac77a4a4c8870943d595fb8c7.png
[20180426161053411]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/31fc0de661a54f6e8d8f5032209d3622.png
[20180426161106155]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/26f21981198b4e739a809b2b4e06ba98.png
[20180426161220583]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/625b8f506a194831a6ab91d5b0853e87.png
[20180426161156865]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/78ca75686c49458b99c53c85f3162ad1.png
[2018042616130673]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/2ccaf95451e9445b84ed9cf33a84cb01.png
[20180426161442660]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/8ae6a59cc2304900a67a117ae7e070db.png
[20180426161509885]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/1a00e5a9e4f04abfb59e43003bd9bd84.png
[20180426161606282]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/ec1e3486444d49b289ead238bfcad286.png
[20180426161629307]: https://image.dandelioncloud.cn/pgy_files/images/2024/01/29/f2f21c816b01496d88f5bfee2f7f3604.png