redhat6.下安装配置hadoop环境--单实例版本

妖狐艹你老母 2022-01-17 17:03 260阅读 0赞

redhat6.4下（用作hadoop master ：记录所有数据分布情况--namenode）--安装单实例计算机  
1、修改hostname  
\[root@hadoop01 \]\# vim /etc/hosts  
\[root@hadoop01 \]\# hostname  
hadoop01  
2、关闭防火墙  
\[root@hadoop01\]\# service iptables stop  
iptables: Setting chains to policy ACCEPT: filter \[ OK \]  
iptables: Flushing firewall rules: \[ OK \]  
iptables: Unloading modules: \[ OK \]  
\[root@hadoop01 \]\# chkconfig iptables off  
\[root@hadoop01 \]\# vi /etc/selinux/config  
3、配置java环境  
\[root@hadoop01 \]\# vi /etc/profile  
export JAVA\_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86\_64/  
export JRE\_HOME=$JAVA\_HOME/jre  
export CLASSPARH=$JACA\_HOME/lib:$JRE\_HOME/lib:$CLASSPATH  
export PATH=$JAVA\_HOME/bin:$JRE\_HOME/bin:$PATH  
\[root@hadoop01 \]\# source /etc/profile //环境生效  
\[root@hadoop01 \]\# mv hadoop-2.7.6 /opt  
4、新建hadoop文件夹  
mkdir /root/hadoop  
mkdir /root/hadoop/tmp  
mkdir /root/hadoop/var  
mkdir /root/hadoop/dfs  
mkdir /root/hadoop/dfs/name  
mkdir /root/hadoop/dfs/data  
5、配置hadoop  
\[root@hadoop01 hadoop\]\# pwd  
/opt/hadoop-2.7.6/etc/hadoop  
\[root@hadoop01 hadoop\]vim hadoop-env.sh //修改JAVA\_HOME  
export HADOOP\_OPTS="-Djava.library.path=$\{HADOOP\_HOME\}/lib/native"  
JAVA\_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86\_64/

\[root@hadoop01 hadoop\]vim yarn-env.sh //修改JAVA\_HOME  
JAVA\_HOME=/usr/lib/jvm/jre-1.7.0-openjdk.x86\_64/

\[root@hadoop01 hadoop\]\# vim core-site.xml  
  
<property>  
<name>hadoop.tmp.dir</name>  
<value>/root/hadoop/tmp</value>  
<description>Abase for other temporary directories.</description>  
</property>  
<property>  
<name>io.file.bufefer.size</name>  
<value>131072</value>  
</property>  
  
<property>  
<name>fs.default.name</name>  
<value>hdfs://hadoop01:9000</value>  
</property>

\[root@hadoop01 hadoop\]\#vim hdfs-site.xml  
<property>  
<name>dfs.namenode.secondary.http-address</name>  
<value>hdfs://hadoop01:9001</value>  
</property>  
<property>  
<name>dfs.namenode.dir</name>  
<value>/root/hadoop/dfs/name</value>  
<description>Path on the local filesystem where theNameNode stores the namespace and transactions logs persistently.</description>  
</property>  
<property>  
<name>dfs.datanode.dir</name>  
<value>/root/hadoop/dfs/data</value>  
<description>Comma separated list of paths on the localfilesystem of a DataNode where it should store its blocks.</description>  
</property>  
  
<property>  
<name>dfs.replication</name>  
<value>3</value>  
</property>  
<property>  
<name>dfs.webhdfs.enabled</name>  
<value>true</value>  
</property>

\[root@hadoop01 hadoop\# vim mapred-site.xml  
<!– 第三个就是制定当前Hadoop的并行运行计算架构，这里为yarn,第四个就是Job运行的历史记录Server,第五个就是历史运行记录的Web服务器。 –>  
<property>  
<name>mapred.job.tracker</name>  
<value>hadoop01:9001</value>  
</property>  
<property>  
<name>mapred.local.dir</name>  
<value>/root/hadoop/var</value>  
</property>  
  
<property>  
<name>mapreduce.framework.name</name>  
<value>yarn</value>  
</property>  
  
<property>  
<name>mapreduce.jobhistory.address</name>  
<value>hadoop01:10020</value>  
</property>  
  
<property>  
<name>mapreduce.jobhistory.webapp.address</name>  
<value>hadoop01:19888</value>  
</property>

\[root@hadoop01 hadoop\]\# vim yarn-site.xml  
  
<property>  
<name>yarn.nodemanager.aux-services</name>  
<value>mapreduce\_shuffle</value>  
</property>  
  
<property>  
<name>yarn.resourcemanager.hostname</name>  
<value>hadoop01</value>  
</property>  
 <property>  
<name>yarn.resourcemanager.webapp.address</name>  
<value>hadoop01:8088</value>  
</property>  
  
<property>  
<name>yarn.log-aggregation-enable</name>  
<value>true</value>  
</property>  
  
<property>  
<name>yarn.log-aggregation.retain-seconds</name>  
<value>86400</value>  
</property>

\[root@hadoop01 hadoop\# vi /etc/profile  
export HADOOP\_HOME=/opt/hadoop-2.7.6  
export HADOOP\_COMMON\_LIB\_NATIVE\_DIR=$HADOOP\_HOME/lib/native  
export HADOOP\_OPTS="-Djava.library.path=$HADOOP\_HOME/lib"  
export PATH=.:$\{JAVA\_HOME\}/bin:$\{HADOOP\_HOME\}/bin:$PATH:$HADOOP\_HOME/sbin  
6、初始化并启动HADOOP  
\[root@hadoop01 bin\]\# ./hadoop namenode -format //初始化  
\[root@hadoop01 current\]\# pwd  
/root/hadoop/dfs/name/current  
\[root@hadoop01 current\]\# ls //验证初始化成功  
fsimage\_0000000000000000000 fsimage\_0000000000000000000.md5 seen\_txid VERSION  
\[root@hadoop01 sbin\]\# pwd  
/opt/hadoop-2.7.6/sbin  
\[root@hadoop01 sbin\]\# ./start-dfs.sh //启动HDFS  
Starting namenodes on \[hadoop01\]  
The authenticity of host 'hadoop01 (172.19.15.151)' can't be established.  
RSA key fingerprint is 00:57:cb:a6:0d:28:05:17:58:0b:5f:8c:9d:2e:c5:1f.  
Are you sure you want to continue connecting (yes/no)? yes  
hadoop01: Warning: Permanently added 'hadoop01,172.19.15.151' (RSA) to the list of known hosts.  
root@hadoop01's password:  
hadoop01: starting namenode, logging to /opt/hadoop-2.7.6/logs/hadoop-root-namenode-hadoop01.out  
root@localhost's password:  
localhost: starting datanode, logging to /opt/hadoop-2.7.6/logs/hadoop-root-datanode-hadoop01.out  
Starting secondary namenodes \[0.0.0.0\]  
The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.  
RSA key fingerprint is 00:57:cb:a6:0d:28:05:17:58:0b:5f:8c:9d:2e:c5:1f.  
Are you sure you want to continue connecting (yes/no)? yes  
0.0.0.0: Warning: Permanently added '0.0.0.0' (RSA) to the list of known hosts.  
root@0.0.0.0's password:  
0.0.0.0: starting secondarynamenode, logging to /opt/hadoop-2.7.6/logs/hadoop-root-secondarynamenode-hadoop01.out  
\[root@hadoop01 sbin\]\# ./start-yarn.sh //启动yarn  
\[root@hadoop01 sbin\]\# start-yarn.sh  
\-bash: start-yarn.sh: command not found  
\[root@hadoop01 sbin\]\# ./start-yarn.sh  
starting yarn daemons  
starting resourcemanager, logging to /opt/hadoop-2.7.6/logs/yarn-root-resourcemanager-hadoop01.out  
root@localhost's password:  
localhost: starting nodemanager, logging to /opt/hadoop-2.7.6/logs/yarn-root-nodemanager-hadoop01.out  
7、查看是否启动成功  
\[root@hadoop01 sbin\]\# jps  
27916 DataNode  
28629 Jps  
28239 ResourceManager  
27779 NameNode  
28083 SecondaryNameNode  
28531 NodeManager  
8、网页访问  
[http://172.19.15.151:8088/cluster][http_172.19.15.151_8088_cluster]  
[http://172.19.15.151:50070][http_172.19.15.151_50070]

转载于:https://blog.51cto.com/14074978/2372596

[http_172.19.15.151_8088_cluster]: http://172.19.15.151:8088/cluster
[http_172.19.15.151_50070]: http://172.19.15.151:50070