java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

﹏ヽ暗。殇╰゛Y 2022-06-02 03:49 107阅读 0赞

在已经搭建好的集群环境Centos6.6+Hadoop2.7+Hbase0.98+Spark1.3.1下，在Win7系统Intellij开发工具中调试Spark读取Hbase。运行直接报错：

<table> 
 <tbody> 
  <tr> 
   <td> 
    <div>
      1 
    </div> 
    <div>
      2 
    </div> 
    <div>
      3 
    </div> 
    <div>
      4 
    </div> 
    <div>
      5 
    </div> 
    <div>
      6 
    </div> 
    <div>
      7 
    </div> 
    <div>
      8 
    </div> 
    <div>
      9 
    </div> 
    <div>
      10 
    </div> 
    <div>
      11 
    </div> 
    <div>
      12 
    </div> 
    <div>
      13 
    </div> 
    <div>
      14 
    </div> 
    <div>
      15 
    </div> 
    <div>
      16 
    </div> 
    <div>
      17 
    </div> 
    <div>
      18 
    </div> 
    <div>
      19 
    </div> 
    <div>
      20 
    </div> 
    <div>
      21 
    </div> 
    <div>
      22 
    </div> 
    <div>
      23 
    </div> 
    <div>
      24 
    </div> 
    <div>
      25 
    </div> 
    <div>
      26 
    </div> 
    <div>
      27 
    </div> 
    <div>
      28 
    </div> 
    <div>
      29 
    </div> </td> 
   <td> 
    <div> 
     <div> 
      <code>15</code> 
      <code>/</code> 
      <code>06</code> 
      <code>/</code> 
      <code>11</code> 
      <code>15</code> 
      <code>:</code> 
      <code>35</code> 
      <code>:</code> 
      <code>50</code> 
      <code>ERROR Shell: Failed to locate the winutils binary in the hadoop binary path</code> 
     </div> 
     <div> 
      <code>java.io.IOException: Could not locate executable </code> 
      <code>null</code> 
      <code>\bin\winutils.exe in the Hadoop binaries.</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:</code> 
      <code>356</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:</code> 
      <code>371</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.util.Shell.&lt;clinit&gt;(Shell.java:</code> 
      <code>364</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.util.StringUtils.&lt;clinit&gt;(StringUtils.java:</code> 
      <code>80</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.security.SecurityUtil.getAuthenticationMethod(SecurityUtil.java:</code> 
      <code>611</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:</code> 
      <code>272</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:</code> 
      <code>260</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:</code> 
      <code>790</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:</code> 
      <code>760</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:</code> 
      <code>633</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$</code> 
      <code>1</code> 
      <code>.apply(Utils.scala:</code> 
      <code>2001</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$</code> 
      <code>1</code> 
      <code>.apply(Utils.scala:</code> 
      <code>2001</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at scala.Option.getOrElse(Option.scala:</code> 
      <code>120</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:</code> 
      <code>2001</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.SecurityManager.&lt;init&gt;(SecurityManager.scala:</code> 
      <code>207</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.SparkEnv$.create(SparkEnv.scala:</code> 
      <code>218</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:</code> 
      <code>163</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:</code> 
      <code>269</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:</code> 
      <code>272</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at org.apache.spark.SparkContext.&lt;init&gt;(SparkContext.scala:</code> 
      <code>154</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at SparkFromHbase$.main(SparkFromHbase.scala:</code> 
      <code>15</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at SparkFromHbase.main(SparkFromHbase.scala)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:</code> 
      <code>57</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:</code> 
      <code>43</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at java.lang.reflect.Method.invoke(Method.java:</code> 
      <code>606</code> 
      <code>)</code> 
     </div> 
     <div> 
      <code>&nbsp;&nbsp;&nbsp;&nbsp;</code> 
      <code>at com.intellij.rt.execution.application.AppMain.main(AppMain.java:</code> 
      <code>134</code> 
      <code>)</code> 
     </div> 
    </div> </td> 
  </tr> 
 </tbody> 
</table>

查看hadoop源码发现里有这么一段：

很明显应该是HADOOP\_HOME的问题。如果HADOOP\_HOME为空，必然fullExeName为null\\bin\\winutils.exe。解决方法很简单，配置环境变量，不想重启电脑可以在程序里加上：

<table> 
 <tbody> 
  <tr> 
   <td> 
    <div>
      1 
    </div> </td> 
   <td> 
    <div> 
     <div> 
      <code>System.setProperty(</code> 
      <code>"hadoop.home.dir"</code> 
      <code>, </code> 
      <code>"E:\\Program Files\\hadoop-2.7.0"</code> 
      <code>);</code> 
     </div> 
    </div> </td> 
  </tr> 
 </tbody> 
</table>

注：E:\\\\Program Files\\\\hadoop-2.7.0是我本机解压的hadoop的路径。

稍后再执行，你可能还是会出现同样的错误，这个时候你可能会要怪我了。其实一开始我是拒绝的，因为你进入你的hadoop-x.x.x/bin目录下看，你会发现你压根就没有winutils.exe这个东东。

于是我告诉你，你可以去github下载一个，地球人都知道的地址发你一个。

地址：[https://github.com/srccodes/hadoop-common-2.2.0-bin][https_github.com_srccodes_hadoop-common-2.2.0-bin]

不要顾虑它的版本，不用怕，因为我用的最新的hadoop-2.7.0都没问题！下载好后，把winutils.exe加入你的hadoop-x.x.x/bin下。

原创：https://www.cnblogs.com/hyl8218/p/5492450.html

[https_github.com_srccodes_hadoop-common-2.2.0-bin]: https://github.com/srccodes/hadoop-common-2.2.0-bin

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

发表评论取消回复

还没有评论，来说两句吧...

相关阅读

相关【异常】FileNotFoundException: Could not locate Hadoop executable: E:\Hadoop\hadoop\bin\winutils.exe

相关 java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

相关 IDEA启动hadoop报Could not locate executable null\bin\winutils.exe in the Hadoop binaries.错误的解决办法

相关 scala运行异常Could not locate executable null\bin\winutils.exe in the Hadoop binaries

相关 java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

相关 Failed to locate the winutils binary in the hadoop binary path java.io.IOException: Could not locate

相关解决Could not locate executable null\bin\winutils.exe in the Hadoop binaries.

相关 Could not locate executable null\bin\winutils.exe in the Hadoop binaries

相关 Hadoop报错：Failed to locate the winutils binary in the hadoop binary path

相关 win7运行mapreduce报错Could not locate executable null\bin\winutils.exe in the Hadoop binaries.