Hadoo系列之Hive安装和使用（三）

太过爱你忘了你带给我的痛 2023-06-18 12:54 17阅读 0赞

hive下载地址：[http://www.apache.org/dyn/closer.cgi/hive/][http_www.apache.org_dyn_closer.cgi_hive]

# hive的安装 #

tar -zxvf apache-hive-3.1.2-bin.tar.gz \#解压

mv hive \#修改文件名

chown -R hadoop:hadoop hive \#修改文件权限

mkdir -p hive/warehouse \#创建元数据存储文件夹

chmod a+rwx -hive/warehouse \#修改文件权限

配置hive环境变量

export HIVE\_HOME=/usr/local/soft/hive

export PATH=$PATH:$HIVE\_HOME/bin

启动hive，要在hadoop用户下执行启动命令：hive

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70][]

# 使用Derby数据库的方式 #

配置hive-site.xml，若无此文件，可以新建一个，此处配置的是本地模式的hive

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<name>hive.metastore.warehouse.dir</name>

<value>/usr/local/soft/hive/warehouse</value>

<description>location of default database for the warehouse</description>

</property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:derby:;databaseName=/usr/local/soft/hive/metastore\_db;create=true</value>

<description>JDBC connect string for a JDBC metastore</description>

</property>

</configuration>

# 使用MySQL数据库的方式 #

安装mysql参考：[https://blog.csdn.net/shiqijiamengjie/article/details/52192398][https_blog.csdn.net_shiqijiamengjie_article_details_52192398]

mysql -uroot -p \#登录mysql数据库

insert into mysql.user (Host,User,Password) values ("localhost","hive",password (" hive")); 2创建hive用户

若上面的创建hive用户语句出现错误，请参考问题3，改用下面的语句创建hive用户

create user 'hive'@'%' identified by 'hive';   //第一个hive是用户名，'%'表示所有ip都可访问，第二个hive是密码

create database hive; \# 创建hive数据库

grant all on hive.\* to hive@'%' identified by 'hive'; \#赋予权限

grant all on hive.\* to hive@'localhost' identified by 'hive';

flush privileges; \#刷新

exit \#退出mysql

mysql -u hive -p hive \#验证hive用户

show databases;

配置hive-site.xml文件

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<name>hive.metastore.warehouse.dir</name>

<value>/usr/local/soft/hive/warehouse</value>

<description>location of default database for the warehouse</description>

</property>

<name>javax.jdo.option.ConnectionURL</name>

<value>jdbc:mysql://localhost:3306/hive?createDatebaseIfNotExist=true</value>

<description>JDBC connect string for a JDBC metastore</description>

</property>

<name>javax.jdo.option.ConnectionDriverName</name>

<value>com.mysql.jdbc.Driver</value>

<description>Driver class name for a JDBC metastore</description>

</property>

<name>javax.jdo.option.ConnectionUserName</name>

<description>Username to use against metastore database</description>

</property>

<name>javax.jdo.option.ConnectionPassword </name>

</property>

<name>datanucleus.schema.autoCreateAll</name>

</property>

</configuration>

将jdbc驱动（mysql-connector-java-x.x.x-bin.jar）放在hive/lib下面

将hive/lib/jline-2.12.jar 放到hadoop/share/hadoop/yarn/lib下面

mkdir -p tmp \#在hive下面创建临时文件夹

chmod a+rwx tmp \#修改文件权限

测试

启动hadoop，执行hive命令，输入show databases;进行测试

# 问题和解决方法： #

1、bash: hive: command not found

解决：hive环境变量没有生效，请检查hive环境变量配置，或重新启动hive环境变量配置

2、Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 1][]

解决（参考文章：[http://www.bubuko.com/infodetail-3286965.html][http_www.bubuko.com_infodetail-3286965.html]）

从报错信息来看：

一，java.lang.NoSuchMethodError

原因：1.系统找不到相关jar包

2.同一类型的 jar 包有不同版本存在，系统无法决定使用哪一个

二，com.google.common.base.Preconditions.checkArgument

根据百度可知，该类来自于guava.jar

三，查看该jar包在hadoop和hive中的版本信息

hadoop-3.2.1（路径：hadoop\\share\\hadoop\\common\\lib）中该jar包为  guava-27.0-jre.jar

![20191202163441554.png][]

hive-2.3.6(路径：hive/lib)中该jar包为guava-14.0.1.jar

![20191202163501883.png][]

四，解决方案

删除hive中低版本的guava-14.0.1.jar包，将hadoop中的guava-27.0-jre.jar复制到hive的lib目录下即可。

五，成功运行hive

3、问题：创建hive用户时，出现错误ERROR 1364 (HY000): Field 'ssl\_cipher' doesn't have a default value

![20191202163535236.png][]

原因：创建用户后没有赋予权限的原因

解决方法（参考文章[https://www.cnblogs.com/m97i/p/10700657.html][https_www.cnblogs.com_m97i_p_10700657.html]）：

mysql5.1之后的版本，插入用户表的语句为

create user 'hive'@'%' identified by 'hive';   //第一个hive是用户名，'%'表示所有ip都可访问，第二个hive是密码

grant usage on \*.\* to 'hive'@'%' with grant option; \#赋予权限

grant select,insert,update,delete,create,drop on \*.\* to 'hive'@'%' with grant option;        //给了hive账号 所有数据库中所有表的增删查改和建表删表的权限

flush privileges;    //刷新，使操作成功

4、当删除创建的用户后，重新创建，报错：ERROR 1396 (HY000): Operation CREATE USER failed for 'hive'@'%'

![20191202163625835.png][]

原因：删除用户不彻底

解决方法（参考文章[https://blog.csdn.net/u011575570/article/details/51438841][https_blog.csdn.net_u011575570_article_details_51438841]）：

select user from user; \#查看用户是否还存在

若没有此用户，删除之后，有无刷新权限；若无，则刷新权限

flush privileges; \#刷新权限

再次执行创建用户的命令，若还是不行，则再删除一次用户，并刷新权限

drop user 'hive'@'%'; \#删除用户

flush privileges;

执行创建用户命令，成功了。

5、navicat如果连接mysql不成功，则需要关闭服务器防火墙

6、问题：在hive下执行show databases；时，报错

FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

![20191202163708661.png][]

解决方法：

1）hive --service metastore &

2）然后Ctrl+C

3）再hive,进去

参考文章（[https://blog.csdn.net/qq\_35078688/article/details/86137440][https_blog.csdn.net_qq_35078688_article_details_86137440]）

在执行hive --service metastore &命令的时候，报错：MetaException(message:Version information not found in metastore. )

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 2][]

解决方法：

在hive-site.xml中配置

<name>hive.metastore.schema.verification</name>

<value>false</value>

</property>

参考文章（[https://blog.csdn.net/zhangyingchengqi/article/details/97645353][https_blog.csdn.net_zhangyingchengqi_article_details_97645353]）

在执行hive --service metastore &命令的时候，若单纯的只有下面的日志，则不用管，可以继续下面的操作

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 3][]

7、执行create database if not exists hive;命令时，报错Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=root, access=WRITE, inode="/":hadoop:supergroup:drwxr-xr-x

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 4][]

解决方法：

在hadoop的hdfs-site.xml中添加配置，并重启hadoop

<name>dfs.permissions</name>

<value>false</value>

</property>

参考文章（[https://my.oschina.net/whitejavadog/blog/896645][https_my.oschina.net_whitejavadog_blog_896645]）

[http_www.apache.org_dyn_closer.cgi_hive]: http://www.apache.org/dyn/closer.cgi/hive/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/2019120216305641.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM=,size_16,color_FFFFFF,t_70
[https_blog.csdn.net_shiqijiamengjie_article_details_52192398]: https://blog.csdn.net/shiqijiamengjie/article/details/52192398
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20191202163335991.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM=,size_16,color_FFFFFF,t_70
[http_www.bubuko.com_infodetail-3286965.html]: http://www.bubuko.com/infodetail-3286965.html
[20191202163441554.png]: https://img-blog.csdnimg.cn/20191202163441554.png
[20191202163501883.png]: https://img-blog.csdnimg.cn/20191202163501883.png
[20191202163535236.png]: https://img-blog.csdnimg.cn/20191202163535236.png
[https_www.cnblogs.com_m97i_p_10700657.html]: https://www.cnblogs.com/m97i/p/10700657.html
[20191202163625835.png]: https://img-blog.csdnimg.cn/20191202163625835.png
[https_blog.csdn.net_u011575570_article_details_51438841]: https://blog.csdn.net/u011575570/article/details/51438841
[20191202163708661.png]: https://img-blog.csdnimg.cn/20191202163708661.png
[https_blog.csdn.net_qq_35078688_article_details_86137440]: https://blog.csdn.net/qq_35078688/article/details/86137440
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/2019120216372861.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM=,size_16,color_FFFFFF,t_70
[https_blog.csdn.net_zhangyingchengqi_article_details_97645353]: https://blog.csdn.net/zhangyingchengqi/article/details/97645353
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20191202163748355.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM=,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20191202163813922.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTAxMjE4ODM=,size_16,color_FFFFFF,t_70
[https_my.oschina.net_whitejavadog_blog_896645]: https://my.oschina.net/whitejavadog/blog/896645