Spark连接hive出错，table not found

向右看齐 2022-02-24 03:14 572阅读 0赞

运行通过maven构建好的jar出错：  
Exception in thread “main” org.apache.spark.sql.catalyst.analysis.NoSuchTableException: Table or view ‘zgw’ not found in database ‘default’;

Exception in thread "main" org.apache.spark.sql.catalyst.analysis.NoSuchTableException: Table or view 'zgw' not found in database 'default';
    at org.apache.spark.sql.hive.client.HiveClient$$anonfun$getTable$1.apply(HiveClient.scala:76)
    at org.apache.spark.sql.hive.client.HiveClient$$anonfun$getTable$1.apply(HiveClient.scala:76)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.sql.hive.client.HiveClient$class.getTable(HiveClient.scala:76)
    at org.apache.spark.sql.hive.client.HiveClientImpl.getTable(HiveClientImpl.scala:78)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$org$apache$spark$sql$hive$HiveExternalCatalog$$getRawTable$1.apply(HiveExternalCatalog.scala:110)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$org$apache$spark$sql$hive$HiveExternalCatalog$$getRawTable$1.apply(HiveExternalCatalog.scala:110)
    at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:95)
    at org.apache.spark.sql.hive.HiveExternalCatalog.org$apache$spark$sql$hive$HiveExternalCatalog$$getRawTable(HiveExternalCatalog.scala:109)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$getTable$1.apply(HiveExternalCatalog.scala:601)
    at org.apache.spark.sql.hive.HiveExternalCatalog$$anonfun$getTable$1.apply(HiveExternalCatalog.scala:601)
    at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:95)
    at org.apache.spark.sql.hive.HiveExternalCatalog.getTable(HiveExternalCatalog.scala:600)
    at org.apache.spark.sql.hive.HiveMetastoreCatalog.lookupRelation(HiveMetastoreCatalog.scala:106)
    at org.apache.spark.sql.hive.HiveSessionCatalog.lookupRelation(HiveSessionCatalog.scala:69)
    at org.apache.spark.sql.SparkSession.table(SparkSession.scala:578)
    at org.apache.spark.sql.SparkSession.table(SparkSession.scala:574)
    at org.apache.spark.sql.SQLContext.table(SQLContext.scala:708)
    at Sparksql.HiveContextApp$.main(HiveContextApp.scala:21)
    at Sparksql.HiveContextApp.main(HiveContextApp.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:738)
    at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
    at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)

根据异常信息提示的很明显，就是说找不到这个表在default这个数据库中，我去hive中查询了一下,  
![keyi][]  
可以看到是存在这个hive表的。  
解决方法：  
**将hive的hive-site.xml拷贝到放入$SPARK-HOME/conf目录下即可。**

[keyi]: /images/20220224/10ef76a0a46a427eb8a7d9d7791863f5.png