Hadoop-本地模式搭建

约定不等于承诺〃 2023-06-27 12:39 99阅读 0赞

![format_png][]

小编先跳过Hadoop的体系结构这段,后续会补上,先将环境搭建写出来,便于大家能够先对Hadoop有个初步认识做准备,今天主要是写本地模式搭建.本地模式不涉及HDFS(存储),只是对数据的分析,小编会在本地模式搭建完成后,通过Hadoop中自带的WordCount来对文件的单词进行统计.

在搭建环境前，需要先关闭防火墙

**1.环境所需的软件**

jdk-8u144-linux-x64.tar.gz

hadoop-2.7.3.tar.gz

**2.上传到Linux环境**

小编是通过SSH Secure连接Linux系统，进行上传，约定好上传的目录为/root/tools

![format_png 1][]

**3.解压压缩包**

解压的路径也约定为 /root/training

\[root@bigdata11 tools\]\# tar -zxvf jdk-8u144-linux-x64.tar.gz -C /root/training/

\[root@bigdata11 tools\]\# tar -zxvf hadoop-2.7.3.tar.gz -C /root/training/

![format_png 2][]

**4.JDK环境变量配置 **

\[root@bigdata11 jdk1.8.0\_144\]\# vi ~/.bash\_profile

![format_png 3][]

**5.Hadoop环境变量配置**

\[root@bigdata11 hadoop-2.7.3\]\# vi ~/.bash\_profile

![format_png 4][]

**6.生效环境变量**

\[root@bigdata11 hadoop-2.7.3\]. ~/.bash\_profile或者source ~/.bash\_profile

注意：若使用前面的语句 .和~中间有个空格。

**7.修改hadoop下的hadoop-env.sh文件**

![format_png 5][]

\[root@bigdata11 hadoop\]\# vi hadoop-env.sh

![format_png 6][]

**8.演示MapReduce程序**

![format_png 7][]

\[root@bigdata11 mapreduce\]\# hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /root/temp/data.txt /root/temp/dataout

执行后，进入/root/temp/dataout目录中查看结果：

![format_png 8][]

\---------------华丽分割线---------------

演示程序中的data.txt需要自己创建，其中的内容为：

I love Beijing

I love China

Beijing is the capital of China

此篇文章主要是环境的搭建,不涉及到结构体系的讲解,大家可以先搭建环境,后续会补充这些相关的理论知识,如果大家学习的大数据知识超过文章的进度,可以与小编留言一起学习讨论.

在公众号 回复 3：获取最新的大数据学习路线，当前获取的大数据学习路线，还在完善中。

[format_png]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X2dpZi9veEFIZXdhTDJyaWFoenFhUXpBYjZEeFZiWjJ5REpqdDZFVnZKdzhpYWFOb1F1UVJNTlRBQkh3NXZDaWIwTDhkUk5vek1MUkZpYWp0QU1sbHgyMEU1aWExczF3LzY0MA?x-oss-process=image/format,png
[format_png 1]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTBTa29kN04wUTVLOWRwSWZmUEx2RVJMNDA2NVd0c1Q3RWtpYXptM2ljMTJhVWZ5SnpYd3NGTHV3dy82NDA?x-oss-process=image/format,png
[format_png 2]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTA2clk5QnZDeTdmRjNraWF0R1B4VzJLUG5kekduSzlodzYwemlidmplRFNhVEZYbVZHclJKaWNyTUEvNjQw?x-oss-process=image/format,png
[format_png 3]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTA3NG5iaFJJQUhPNnByYTRaNUhCQzlhblVoVE1zN3h5dHZrNTFHamhXSW9tWkt3QmY2THBJY0EvNjQw?x-oss-process=image/format,png
[format_png 4]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTBEeWZCcHJpY0tuMjZaYkhZdFMxbTRHSlFhdnp3NnVTaWJpYXNFeWFNaWMwVmlhSVgzT2pIWVNTTWdUZy82NDA?x-oss-process=image/format,png
[format_png 5]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTBXbVVrOUU5aWNDNkUzcGlhZWtmU3F2MndRQ1FtQzB2dTR1SlFEQzlsQTlzcVlIS3I5MUVoS0tVUS82NDA?x-oss-process=image/format,png
[format_png 6]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTBpYzc1OUJVWjNLSnQ3Z3Q1aldwa1hpYnNRemxJUk1LRVhPY01rdk9RaWN1ZzhuTHJJaWE5OHppYnEwQS82NDA?x-oss-process=image/format,png
[format_png 7]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTA0TXhGWEVMY0lpYzFPcWVHSGVpYnl3QXVGY2psWjMxNzBpY1gwSWlhOXRvMDI3NDFpYjRVNmliYTRJZ0EvNjQw?x-oss-process=image/format,png
[format_png 8]: https://imgconvert.csdnimg.cn/aHR0cHM6Ly9tbWJpei5xcGljLmNuL21tYml6X3BuZy9veEFIZXdhTDJyZ1MyYnJ1ZklvcWlhemZLR3pLRFl2RTBrMURlWEZjV25FQVhWN09lNmxvVnRRYThzMzA4dzNzazVHYXhQdzF3cWppYzBSOXFiY1ZVWEJRLzY0MA?x-oss-process=image/format,png