Hadoop2.6.0 mac 安装

末蓝、 2022-08-02 04:44 103阅读 0赞

1. 单机模式

(1) ssh设置远程登录：

xiaolan:~ xiaolan$@ssh-keygen -t rsa -P ""

xiaolan:~ xiaolan$cat ~/.ssh/id\_rsa.pub >> ~/.ssh/authorized\_keys

登录测试

xiaolan:~ xiaolan$ssh localhost

xiaolan:~ xiaolan$exit

(2) 安装jdk(忽略)

(3) 配置环境变量

xiaolan:~ xiaolan$vim ~/.bash\_profile

输入内容：

export HADOOP\_HOME=/Users/xiaolan/Applications/hadoop-2.6.0

export YARN\_HOME=/Users/xiaolan/Applications/hadoop-2.6.0

export HADOOP\_CONF\_DIR=$HADOOP\_HOME/etc/hadoop

export YARN\_CONF\_DIR=$YARN\_HOME/etc/hadoop

:wq

让环境变量生效

xiaolan:~ xiaolan$source ~/.bash\_profile

(4) 配置hadoop

进入Applications/hadoop-2.6.0/etc/hadoop

xiaolan:~ xiaolan$vim hadoop-env.sh

输入内容：  
export JAVA\_HOME=/Library/Java/JavaVirtualMachines/jdk1.7.0\_71.jdk/Contents/Home

export HADOOP\_HOME=/Users/xiaolan/Applications/hadoop-2.6.0

export PATH=$PATH:/Users/xiaolan/Applications/hadoop-2.6.0/bin

:wq

让环境变量生效

xiaolan:~ xiaolan$source hadoop-env.sh

单机模式安装成功

(5) 测试

xiaolan:~ xiaolan$mkdir input

xiaolan:~ xiaolan$ cp \*\*/\*\* input

注：随便拷贝点文件

xiaolan:~ xiaolan$hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount input output

xiaolan:~ xiaolan$ cat output/\*

注：成功看到结果

2. 伪分布模式

(1) 配置core-site.xml

<configuration></configuration>之间增加如下内容：

<name>fs.default.name</name>

<value>hdfs://localhost:9000</value>

</property>

(2) 配置yarn-site.xml

<configuration></configuration>之间增加如下内容：

<name>yarn.noCHdemanager.aux-services</name>

<value>mapreduce\_shuffle</value>

</property>

<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name><value>org.apache.hadoop.mapred.ShuffleHandler</value>

</property>

(3) 创建和配置mapred-site.xml

默认/etc/hadoop文件夹下有mapred.xml.template文件，复制更改用户名mapred.xml

<configuration></configuration>之间增加如下内容：

<name>mapreduce.framework.name</name>

</property>

(4) 配置hdfs-site.xml

指定主机上作为namenode和datanode的目录：

/Users/xiaolan/Applications/hadoop-2.6.0/hdfs/name

/Users/xiaolan/Applications/hadoop-2.6.0/hdfs/data

<configuration></configuration>之间增加如下内容：

<name>dfs.replication</name>

</property>

<name>dfs.namenode.name.dir</name>

<value>file:/Users/xiaolan/Applications/hadoop-2.6.0/hdfs/name</value>

</property>

<name>dfs.datanode.data.dir</name>

<value>file:/Users/xiaolan/Applications/hadoop-2.6.0/hdfs/data</value>

</property>

(5) 格式化hdfs

xiaolan:~ xiaolan$hdfs namenode -format

(6) 启动hadoop

xiaolan:~ xiaolan$cd Applications/hadoop-2.6.0/sbin

xiaolan:sbin xiaolan$./start-dfs.sh

xiaolan:sbin xiaolan$./start-yarn.sh

浏览器打开 http://localhost:50070/，会看到hdfs管理页面

浏览器打开 http://localhost:8088/，会看到hadoop进程管理页面

(7) WordCount验证

创建input目录

xiaolan:hadoop-2.6.0 xiaolan$hadooop fs -mkdir input

fs:hadoop下的文件命令，和普通命令基本一致

拷贝文件

xiaolan:hadoop-2.6.0 xiaolan$hadooop fs -copyFromLocal README.txt input

xiaolan:hadoop-2.6.0 xiaolan$hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount input output

xiaolan:hadoop-2.6.0 xiaolan$cat output/\*

注：成功看到结果

linux：图文教程，版本较老：

http://www.linuxidc.com/Linux/2013-01/77681p4.htm

http://www.cnblogs.com/kinglau/p/3794433.html

官网安装教程：http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/SingleCluster.html

Eclipse配置：

http://andy-ghg.iteye.com/blog/1165453

注：

据说可以忽略：WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable：

http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-error-on-centos

转帖：http://wollt1992.blog.163.com/blog/static/57013456201411244394378

Hadoop2.6.0 mac 安装

发表评论取消回复

还没有评论，来说两句吧...

相关阅读

相关 Hadoop安装 --- 简易安装Hadoop

相关 Hadoop安装(二) --- Hadoop安装

相关 Mac下单机安装部署Hadoop3.2.1

相关 Hadoop之Hadoop安装

相关 Mac搭建Hadoop

相关 260 Single Number III

相关 Hadoop2.6.0 mac 安装

相关 260. Single Number III

相关 Mac 安装 hadoop

相关 Mac 下编译 Hadoop