使用在hdfs上的文件导入到hive中

古城微笑少年丶 2022-06-11 05:42 561阅读 0赞

hive要打开，另一边root@hadoop-virtual-machine:/\# hdfs dfs -ls /user/hive查看hdfs上都有些什么，

然后把数据文件导入到hdfs上 ，root@hadoop-virtual-machine:/\# hdfs dfs -put /opt/os.txt /user/hive/

hive那边用show databases;查看当前都有什么数据库

然后创建表（此表是用来装导入进来的数据的）

hive> create table os  
> (pathid string,nodeweight int,pathweight string,nodeid int,shutdown string)  
> ROW FORMAT DELIMITED FIELDS TERMINATED BY ' ' LINES TERMINATED BY '\\n'  
> STORED AS TEXTFILE;

要保证创建的表的格式要和数据文件中的格式一样，例如字段间的间隔符是什么行与行间的间隔符都要进行限定，

导入数据 hive> load data inpath '/user/hive/os.txt' into table os;

hive> select \* from os;  
OK  
\["12->22"\]9638\[52\]12false  
\["19->78"\]11895\[51\]19true  
\["16->17"\]8608\[58\]16true  
\["16->17"\]10240\[54\]16false  
\["12->22"\]9638\[52\]12true  
\["2->22","2->3"\]9638\[52,44\]2true  
\["19->78"\]11895\[51\]19false  
\["16->17"\]8608\[58\]16false  
\["16->17"\]10240\[54\]16true  
\["1->65"\]13271\[53\]1 true  
\["11->17","11->13"\]10383\[50,56\]11false  
\["6->21","6->22","6->19","6->7","6->44","6->10","6->9","6->11"\]5398\[54,58,54,55,50,59,55,55\]6false  
\["0->52","0->91","0->53"\]9814\[52,58,54\]0true  
\["1->65"\]13271\[53\]1 false  
\["11->17","11->13"\]10383\[50,56\]11true  
\["6->21","6->22","6->19","6->7","6->44","6->10","6->9","6->11"\]5398\[54,58,54,55,50,59,55,55\]6true  
\["0->52","0->91","0->53"\]9814\[52,58,54\]0false  
Time taken: 0.114 seconds, Fetched: 17 row(s)

进来了

其实我在导入的时候下面显示Loading data to table default.os  
Failed with exception Unable to alter table. For direct MetaStore DB connections, we don't support retries at the client level.  
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

但是之后我想查看一下表中到底有没有数据，数据是导入进来的，但是为什么低下有failed那几行我也不太清楚，如果有知道的可以告诉我，学习知识大家互相讨论互相进步嘛