JSON數據讀入Hive表

Question

我能夠使用JSON Serde org.openx.data.jsonserde.JsonSerDe創建Hive表，但是當我讀取Hive表中的數據時，我無法閱讀。

hive> create table emp (EmpId int , EmpFirstName string , EmpLastName string) ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'; 
OK 
Time taken: 2.148 seconds 

hive> LOAD DATA INPATH '/user/cloudera/EmpData/emp.json' INTO table emp; 
Loading data to table employee.emp 
chgrp: changing ownership of 'hdfs://quickstart.cloudera:8020/user/hive/warehouse/employee.db/emp/emp.json': User does not belong to supergroup 
Table employee.emp stats: [numFiles=1, totalSize=4163] 
OK 
Time taken: 1.141 seconds 

hive> select * from emp; 
OK 
Failed with exception java.io.IOException:org.apache.hadoop.hive.serde2.SerDeException: Row is not a valid JSON Object - JSONException: A JSONObject text must end with '}' at 2 [character 3 line 1] 
Time taken: 0.504 seconds 

Answer 1

錯誤：無法用異常產生java.io.IOException：org.apache.hadoop.hive.serde2.SerDeException：行不是有效的JSON對象 - JSONException：一個JSONObject文字必須以「}結束'在2 [3字符線1]

檢查在/user/cloudera/EmpData/emp.json提供的JSON是有效

可以通過

ALTER TABLE table emp SET SERDEPROPERTIES ("ignore.malformed.json" = "true"); 

消除無效的行

檢查此鏈接 - >https://github.com/rcongiu/Hive-JSON-Serde

編輯： 這是無效的JSON

{ "cols": [ "EmpId", "EmpFirstName", "EmpLastName" ], "data": [ [ 1, "Hannah", "Walton" ], [ 2, "Barrett", "Mendoza" ], [ 3, "Camden", "Kidd" ], [ 4, "Illiana", "Collier" ] ] }

的JSON您提供具有

key:cols and value:[ "EmpId", "EmpFirstName", "EmpLastName" ]

和

key :data and value :[ [ 1, "Hannah", "Walton" ], [ 2, "Barrett", "Mendoza" ], [ 3, "Camden", "Kidd" ], [ 4, "Illiana", "Collier" ]

json的應該是類似

{"EmpId":1,"EmpFirstName":"Hannah","EmpLastName":"Walton"} 
{"EmpId":2,"EmpFirstName":"Barrett","EmpLastName":"Mendoza"} 
{"EmpId":3,"EmpFirstName":"Camden","EmpLastName":"Kidd"}