Hadoop2.5.2完全分布式运行wordcount

心已赠人 2022-08-10 03:43 254阅读 0赞

**\[hadoop@lrts216 bin\]$./hdfs dfs -mkdir -p /lrts/zhangzk**

13/12/19 22:20:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
**\[hadoop@lrts216 bin\]$./hdfs dfs -ls /**  
13/12/19 22:20:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
Found 1 items  
drwxr-xr-x   - hadoop supergroup          0 2013-12-19 22:20 /lrts  
**\[hadoop@lrts216 bin\]$./hdfs dfs -put hdfs.cmd /lrts/zhangzk/**  
13/12/19 22:21:16 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
**\[hadoop@lrts216 bin\]$ll**  
total 252  
\-rwxr--r--. 1 hadoop hadoop 78809 Oct  7 14:38 container-executor  
\-rwxr--r--. 1 hadoop hadoop  4857 Oct  7 14:38 hadoop  
\-rwxr--r--. 1 hadoop hadoop  7530 Oct  7 14:38 hadoop.cmd  
\-rwxr--r--. 1 hadoop hadoop  7954 Oct  7 14:38 hdfs  
\-rwxr--r--. 1 hadoop hadoop  5138 Oct  7 14:38 hdfs.cmd  
\-rwxr--r--. 1 hadoop hadoop  4989 Oct  7 14:38 mapred  
\-rwxr--r--. 1 hadoop hadoop  5560 Oct  7 14:38 mapred.cmd  
\-rwxr--r--. 1 hadoop hadoop  1776 Oct  7 14:38 rcc  
\-rwxr--r--. 1 hadoop hadoop 95102 Oct  7 14:38 test-container-executor  
\-rwxr--r--. 1 hadoop hadoop  8548 Oct  7 14:38 yarn  
\-rwxr--r--. 1 hadoop hadoop  8322 Oct  7 14:38 yarn.cmd  
**\[hadoop@lrts216 bin\]$./hadoop jar /home/aboutyun/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.2.jar wordcount /lrts/zhangzk /lrts/ou**t

13/12/19 22:24:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

13/12/19 22:24:56 INFO client.RMProxy: Connecting to ResourceManager at lrts216/192.168.1.216:8032  
13/12/19 22:24:57 INFO input.FileInputFormat: Total input paths to process : 1  
13/12/19 22:24:57 INFO mapreduce.JobSubmitter: number of splits:1  
13/12/19 22:24:57 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class  
13/12/19 22:24:57 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class  
13/12/19 22:24:57 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name  
13/12/19 22:24:57 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class  
13/12/19 22:24:57 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir  
13/12/19 22:24:57 INFO mapreduce.JobSubmitter: Submitting tokens for job: job\_1387462237922\_0001  
13/12/19 22:24:58 INFO impl.YarnClientImpl: Submitted application application\_1387462237922\_0001 to ResourceManager at lrts216/192.168.1.216:8032  
13/12/19 22:24:58 INFO mapreduce.Job: The url to track the job: http://lrts216:8088/proxy/application\_1387462237922\_0001/  
13/12/19 22:24:58 INFO mapreduce.Job: Running job: job\_1387462237922\_0001  
13/12/19 22:25:06 INFO mapreduce.Job: Job job\_1387462237922\_0001 running in uber mode : false  
13/12/19 22:25:06 INFO mapreduce.Job:  map 0% reduce 0%  
13/12/19 22:25:11 INFO mapreduce.Job:  map 100% reduce 0%  
13/12/19 22:25:17 INFO mapreduce.Job:  map 100% reduce 100%  
13/12/19 22:25:17 INFO mapreduce.Job: Job job\_1387462237922\_0001 completed successfully  
13/12/19 22:25:18 INFO mapreduce.Job: Counters: 43  
        File System Counters  
                FILE: Number of bytes read=4715  
                FILE: Number of bytes written=167555  
                FILE: Number of read operations=0  
                FILE: Number of large read operations=0  
                FILE: Number of write operations=0  
                HDFS: Number of bytes read=5244  
                HDFS: Number of bytes written=3660  
                HDFS: Number of read operations=6  
                HDFS: Number of large read operations=0  
                HDFS: Number of write operations=2  
        Job Counters   
                Launched map tasks=1  
                Launched reduce tasks=1  
                Rack-local map tasks=1  
                Total time spent by all maps in occupied slots (ms)=2410  
                Total time spent by all reduces in occupied slots (ms)=2941  
        Map-Reduce Framework  
                Map input records=171  
                Map output records=523  
                Map output bytes=6735  
                Map output materialized bytes=4715  
                Input split bytes=106  
                Combine input records=523  
                Combine output records=264  
                Reduce input groups=264  
                Reduce shuffle bytes=4715  
                Reduce input records=264  
                Reduce output records=264  
                Spilled Records=528  
                Shuffled Maps =1  
                Failed Shuffles=0  
                Merged Map outputs=1  
                GC time elapsed (ms)=35  
                CPU time spent (ms)=750  
                Physical memory (bytes) snapshot=398831616  
                Virtual memory (bytes) snapshot=1765736448  
                Total committed heap usage (bytes)=354942976  
        Shuffle Errors  
                BAD\_ID=0  
                CONNECTION=0  
                IO\_ERROR=0  
                WRONG\_LENGTH=0  
                WRONG\_MAP=0  
                WRONG\_REDUCE=0  
        File Input Format Counters   
                Bytes Read=5138  
        File Output Format Counters   
                Bytes Written=3660  
**\[hadoop@lrts216 bin\]$./hdfs dfs -ls /lrts/ou**t  
13/12/19 22:26:17 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
Found 2 items  
\-rw-r--r--   2 hadoop supergroup          0 2013-12-19 22:25 /lrts/out/\_SUCCESS  
\-rw-r--r--   2 hadoop supergroup       3660 2013-12-19 22:25 /lrts/out/part-r-00000  
**\[hadoop@lrts216 bin\]$./hdfs dfs -cat /lrts/out/part-r-0000**0  
13/12/19 22:27:30 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable  
"%1"    2  
"%HADOOP\_BIN\_PATH:~-1%" 1  
"--config"      2  
"AS     1  
"License");     1  
"\\"     1  
%\*      2  
%1      1  
%1,     1

%2      1