hadoop源码导入到eclipse

迷南。 2022-07-13 11:00 192阅读 0赞

# 1.版本说明 #

操作系统：ubantu16.04.1 Desktop

hadoop：hadoop-2.7.3-src

eclipse：java-neon

**其他需要安装软件说明：**

**jdk1.8 maven3.0.5 ProtocalBuffer-2.5.0**

**安装教程请参考我的另一篇博客：[http://blog.csdn.net/blue\_it/article/details/54133407][http_blog.csdn.net_blue_it_article_details_54133407]**

# 2.导入前准备 #

将hadoop-2.7.3-src.tar.gz解压到eclipse的workspace目录，如下图：

![Center][]

cd进入hadoop-2.7.3-src/hadoop-maven-plugins/目录，输入如下命令：

mvn install

如下图：

![Center 1][]

执行成功会提示如下信息：

![Center 2][]

cd 进入hadoop-2.7.3-src目录，输入如下命令：

mvn eclipse:eclipse -DskipTests

执行完成会提示build success。

# 3 导入 #

打开eclipse，点击File ---> Import --->Existing Maven Projects

![Center 3][]

点击，找到workspace下的hadoop-2.7.3-src文件夹，选中，点击finish，即开始导入hadoop源码

![Center 4][]

导入过程中会弹出对话框选择Resolve all later，然后点击finish即完成项目的导入。

# 4 错误处理 #

源码导入后，会看到大概有五十多个项目，其中部分出现红叉，有一个出现红色感叹号。红叉大概有两类原因引起，pom.xml错误和类库无法找到，红色感叹号由资源文件未找到引起。

*  解决pom.xml错误

在eclipse菜单栏选取Windows ---> Preferences ---> Maven ---> Lifecycle Mappings，如下图：

![Center 5][]

点击Open Workspace Lifecycle Mappings Metadata，即可看到eclipse中打开了lifecycle-mappings-metadata.xml文件，关闭Prefences窗口，编辑lifecycle-mappings-metadata.xml文件，将如下xml内容，复制到lifecycle-mappings-metadata.xml中对原来的文件内容进行覆盖。

<?xml version="1.0" encoding="UTF-8"?>
    <lifecycleMappingMetadata>
      <pluginExecutions>
        <pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-antrun-plugin</artifactId>
            <versionRange>1.7</versionRange>
            <goals>
              <goal>run</goal>
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
        <pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-jar-plugin</artifactId>
            <versionRange>2.5</versionRange>
            <goals>
              <goal>jar</goal>
              <goal>test-jar</goal>
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
        <pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-plugin-plugin</artifactId>
            <versionRange>3.4</versionRange>
            <goals>
              <goal>descriptor</goal>
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
      
      	<pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-maven-plugins</artifactId>
            <versionRange>2.7.3</versionRange>
            <goals>
              <goal>protoc</goal>
              <goal>version-info</goal>       
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
        
        <pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-remote-resources-plugin</artifactId>
            <versionRange>1.5</versionRange>
            <goals>
              <goal>bundle</goal>
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
        <pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-maven-plugins</artifactId>
            <versionRange>1.5</versionRange>
            <goals>
              <goal>bundle</goal>
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
        <pluginExecution>
          <pluginExecutionFilter>
            <groupId>org.apache.avro</groupId>
            <artifactId>avro-maven-plugin</artifactId>
            <versionRange>1.7.4</versionRange>
            <goals>
              <goal>schema</goal>
              <goal>protocol</goal>
            </goals>
          </pluginExecutionFilter>
          <action>
            <ignore />
          </action>
        </pluginExecution>
      </pluginExecutions>
    </lifecycleMappingMetadata>

编辑文件完成后，保存，再次打开Windows ---> Preferences ---> Maven ---> Lifecycle Mappings，一次点击Rload Workspace Lifecycle Mappings Metadata、 Apply、 OK。

在eclipse中点开每个项目，凡是pom.xml文件出现红叉的，选中该项目名称，右键在弹出的菜单中选取

![Center 6][]

选择Update Project …即可修正pom.xml出现红叉的错误。

*  解决红叉错误： org.apache.hadoop.ipc.protobuf.xxxx找不到错误

进入hadoop-2.7.3-src/hadoop-common-project/hadoop-common/src/test/proto目录

在命令行中输入如下命令：

protoc --java\_out=../java \*.proto

![Center 7][]

执行完成后，选中项目，F5刷新项目，即可解决错误。

*  解决红叉错误：AvroRecord cannot be resolved to a type TestAvroSerialization.java

下载下载 avro-tools-1.7.4.jar，放在hadoop-2.7.3-src/hadoop-common-project/hadoop-common/src/test/avro/目录下  
然后执行下述命令：  
java -jar avro-tools-1.7.4.jar compile schema avroRecord.avsc ../java

![Center 8][]

执行完成后，选中项目，F5刷新项目，即可解决错误。

*  解决红色感叹号：Project 'hadoop-streaming' is missing required source……

选定项目右键，properties->Java Build Path->Source->选定带红叉的项->Remove，然后关闭窗口即可。

![Center 9][]

[http_blog.csdn.net_blue_it_article_details_54133407]: http://blog.csdn.net/blue_it/article/details/54133407
[Center]: /images/20220713/fbf6c09e4dd34abca4c4df972aafa19d.png
[Center 1]: /images/20220713/eeb6dc8ee3ea4c1a8ad4d1dd5d5169b0.png
[Center 2]: /images/20220713/cecc5a6453524458a9f47f572df0d66b.png
[Center 3]: /images/20220713/e002ee06d8af4ad48af545e75605d601.png
[Center 4]: /images/20220713/49bba25d881446d28cc318a70d49509e.png
[Center 5]: /images/20220713/39c7cc956bba444f93c1133c6c9c53cb.png
[Center 6]: /images/20220713/626e5da4c8f24822aa6e6e33067aff58.png
[Center 7]: /images/20220713/14b467c136ea4b728a772bc41a2dd97b.png
[Center 8]: /images/20220713/ebf02e0a589d4d69a5852a6cf4ccaef3.png
[Center 9]: /images/20220713/f757c3103faf4c83b371bab5a2b750cc.png