Hadoop Java对应版本号

布满荆棘的人生 2023-02-12 05:58 1阅读 0赞

Hadoop和Java的版本要一一对应，否则到时候会出现不支持的函数出现。下面这个是官方给出来的对应信息。

## Java 版本支持 ##

*  Apache Hadoop 3.x 版本 现在只支持 Java 8
 *  Apache Hadoop 从2.7.x 到 2.x 版本支持Java 7 and 8
 *  Apache Hadoop 最新更新的2.7版本需要Java 7. 它在 OpenJDK 和Oracle (HotSpot)'s JDK/JRE下都已经通过编译和检测，早期的版本 (2.6 或更早版本) 则支持Java 6.
 *  Java 11 现阶段部分支持:
    
     *  trunk (3.3.0-SNAPSHOT) 支持 Java 11 运行时:  [HADOOP-15338][] - Java 11 运行时支持RESOLVED
     *  Hadoop在使用 Java 11编译时不支持:  [HADOOP-16795][] - Java 11 编译时支持 OPEN

##  JDKs/JVMs支持 ##

*  现Apache Hadoop 社区用 OpenJDK 来作为 build/test/release 的环境, 下面是 OpenJDK 为什么得到社区支持的原因 .
    
     *  [https://github.com/apache/hadoop/blob/rel/release-3.2.1/dev-support/docker/Dockerfile\#L92][https_github.com_apache_hadoop_blob_rel_release-3.2.1_dev-support_docker_Dockerfile_L92]
 *  其他的jdks/jvms 应该也蛮好使的. 如果你觉得他们不怎么好使, 请查找 JIRA相关文档.

## 通过检验的JDK ##

下表是已经已知 在使用的 或已经通过检验的JDKs 版本:

<table> 
 <tbody> 
 <tr> 
 <td style="width:305px;"> 版本号 </td> 
 <td style="width:252px;"> 状态 </td> 
 <td> 报告来源 </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.7.0_15 </td> 
 <td style="width:252px;"> Good </td> 
 <td> Cloudera </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.7.0_21 </td> 
 <td style="width:252px;"> Good (4) </td> 
 <td> Hortonworks </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.7.0_45 </td> 
 <td style="width:252px;"> Good </td> 
 <td> Pivotal </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> openjdk 1.7.0_09-icedtea </td> 
 <td style="width:252px;"> Good (5) </td> 
 <td> Hortonworks </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_16 </td> 
 <td style="width:252px;"> Avoid (1) </td> 
 <td> Cloudera </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_18 </td> 
 <td style="width:252px;"> Avoid </td> 
 <td> Many </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_19 </td> 
 <td style="width:252px;"> Avoid </td> 
 <td> Many </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_20 </td> 
 <td style="width:252px;"> Good (2) </td> 
 <td> LinkedIn, Cloudera </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle&nbsp;<a href="http://www.oracle.com/technetwork/java/javase/downloads/jdk6-jsp-136632.html" rel="nofollow">1.6.0_21</a> </td> 
 <td style="width:252px;"> Good (2) </td> 
 <td> Yahoo!, Cloudera </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_24 </td> 
 <td style="width:252px;"> Good </td> 
 <td> Cloudera </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_26 </td> 
 <td style="width:252px;"> Good(2) </td> 
 <td> Hortonworks, Cloudera </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_28 </td> 
 <td style="width:252px;"> Good </td> 
 <td> LinkedIn </td> 
 </tr> 
 <tr> 
 <td style="width:305px;"> oracle 1.6.0_31 </td> 
 <td style="width:252px;"> Good(3, 4) </td> 
 <td> Cloudera, Hortonworks </td> 
 </tr> 
 </tbody> 
</table>

如果是新项目的话，在没有项目包袱的情况下，建议尽量选用Hadoop3.X以上的版本会更加适合。如果是老项目的话，想进行项目升级的话，要注意了，因为Java版本的一些变化导致代码有些不兼容。

下面是来自官网的JDK1.8版本修改导致的不兼容问题。

## Java 不兼容的变更 ##

下面表格文档供升级的Hadoop集群的Java版本的用户使用 . 它记录了影响Apache Hadoop的Java的变更.

### Java 8 ###

<table> 
 <thead> 
 <tr> 
 <th> 版本号 </th> 
 <th style="width:492px;"> 不兼容的变更 </th> 
 <th colspan="1" style="width:94px;"> 相关JDK bug系统 票证 </th> 
 <th colspan="1" style="width:311px;"> 关联的JIRAs </th> 
 </tr> 
 </thead> 
 <tbody> 
 <tr> 
 <td colspan="1">1.8.0_242</td> 
 <td colspan="1" style="width:492px;"> The visibility of sun.nio.ch.SocketAdaptor is changed from public to package-private.&nbsp;TestIPC#testRTEDuringConnectionSetup is affected.&nbsp; </td> 
 <td colspan="1" style="width:94px;"><a href="https://bugs.openjdk.java.net/browse/JDK-8237177" rel="nofollow">JDK-8237177</a></td> 
 <td colspan="1" style="width:311px;"> <a href="https://issues.apache.org/jira/browse/HADOOP-15787" rel="nofollow">HADOOP-15787</a>&nbsp;-&nbsp;[JDK11] TestIPC.testRTEDuringConnectionSetup fails&nbsp;RESOLVED </td> 
 </tr> 
 <tr> 
 <td colspan="1">1.8.0_242</td> 
 <td colspan="1" style="width:492px;"> Kerberos Java client will fail by "Message stream modified (41)" when the client requests a renewable ticket and the KDC returns a non-renewable ticket. If your principal is not allowed to obtain a renewable ticket, you must remove "renew_lifetime" setting from your krb5.conf. </td> 
 <td colspan="1" style="width:94px;"><a href="https://bugs.openjdk.java.net/browse/JDK-8131051" rel="nofollow">JDK-8131051</a></td> 
 <td colspan="1" style="width:311px;">&nbsp;</td> 
 </tr> 
 <tr> 
 <td>1.8.0_171</td> 
 <td style="width:492px;"> In Apache Hadoop 2.7.0 to 2.7.6, 2.8.0 to 2.8.4, 2.9.0 to 2.9.1, 3.0.0 to 3.0.2, and 3.1.0, KMS fails by java.security. UnrecoverableKeyException due to&nbsp;<a href="https://www.oracle.com/technetwork/java/javase/8u171-relnotes-4308888.html#JDK-8189997" rel="nofollow">Enhanced KeyStore Mechanisms</a>. You need to set the system property "jceks.key.serialFilter" to the following value to avoid this error: <pre>java.lang.Enum;java.security.KeyRep;java.security.
KeyRep$Type;javax.crypto.spec.SecretKeySpec;
org.apache.hadoop.crypto.key.JavaKeyStoreProvider$KeyMetadata;!*"</pre> </td> 
 <td colspan="1" style="width:94px;">&nbsp;</td> 
 <td colspan="1" style="width:311px;"> <a href="https://issues.apache.org/jira/browse/HADOOP-15473" rel="nofollow">HADOOP-15473</a>&nbsp;-&nbsp;Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997&nbsp;RESOLVED </td> 
 </tr> 
 <tr> 
 <td>1.8.0_191</td> 
 <td style="width:492px;"> All DES cipher suites were disabled. If you are explicitly using DEC cipher suites, you need to change cipher suite to a strong one. </td> 
 <td colspan="1" style="width:94px;">&nbsp;</td> 
 <td colspan="1" style="width:311px;"> <a href="https://issues.apache.org/jira/browse/HADOOP-16016" rel="nofollow">HADOOP-16016</a>&nbsp;-&nbsp;TestSSLFactory#testServerWeakCiphers sporadically fails in precommit builds&nbsp;RESOLVED </td> 
 </tr> 
 </tbody> 
</table>

[HADOOP-15338]: https://issues.apache.org/jira/browse/HADOOP-15338
[HADOOP-16795]: https://issues.apache.org/jira/browse/HADOOP-16795
[https_github.com_apache_hadoop_blob_rel_release-3.2.1_dev-support_docker_Dockerfile_L92]: https://github.com/apache/hadoop/blob/rel/release-3.2.1/dev-support/docker/Dockerfile#L92