Cloudera的蜂巢：凡添加JSON-SERDE-1.3.7 jar文件

Question

我使用了Cloudera 5.8.0

首先我運行此命令：

hive> ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar; 
Added [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar] to class path 
Added resources: [/usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar] 

而且比我添加JSON-SERDE -1.3.7 jar文件

hive> ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with- dependencies.jar; 
Added [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar] to class path 
Added resources: [/usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar] 

但是當我嘗試創建該表在配置單元：

CREATE EXTERNAL TABLE tweets ( 
    id BIGINT, 
    created_at STRING, 
    source STRING, 
    favorited BOOLEAN, 
    retweet_count INT, 
    retweeted_status STRUCT< 
    text:STRING, 
    `user`:STRUCT<screen_name:STRING,name:STRING>>, 
    entities STRUCT< 
    urls:ARRAY<STRUCT<expanded_url:STRING>>, 
    user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>, 
    hashtags:ARRAY<STRUCT<text:STRING>>>, 
    text STRING, 
    `user` STRUCT< 
    screen_name:STRING, 
    name:STRING, 
    friends_count:INT, 
    followers_count:INT, 
    statuses_count:INT, 
    verified:BOOLEAN, 
    utc_offset:INT, 
    time_zone:STRING>, 
    in_reply_to_screen_name STRING 
) 
ROW FORMAT SERDE 'com.cloudera.hive.SERDE.JSONSerDe' 
LOCATION '/user/cloudera/flume/tweets' 

我得到這個錯誤：

Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: com.cloudera.hive.SERDE.JSONSerDe

Answer 1

這解決了，你已經找到了JSON-SERDE-1.3.7-JAR-與-dependencies.jar問題

ADD JAR /usr/lib/hive/lib/hive-serdes-1.0-SNAPSHOT.jar; 
ADD JAR /usr/lib/hive/lib/json-serde-1.3.7-jar-with-dependencies.jar; 

CREATE EXTERNAL TABLE tweets (
    id BIGINT, 
    created_at STRING, 
    source STRING, 
    favorited BOOLEAN, 
    retweet_count INT, 
    retweeted_status STRUCT< 
     text:STRING, 
     user:STRUCT<screen_name:STRING,name:STRING>>, 
    entities STRUCT< 
     urls:ARRAY<STRUCT<expanded_url:STRING>>, 
     user_mentions:ARRAY<STRUCT<screen_name:STRING,name:STRING>>, 
     hashtags:ARRAY<STRUCT<text:STRING>>>, 
    text STRING, 
    user STRUCT< 
     screen_name:STRING, 
     name:STRING, 
     friends_count:INT, 
     followers_count:INT, 
     statuses_count:INT, 
     verified:BOOLEAN, 
     utc_offset:INT, 
     time_zone:STRING>, 
    in_reply_to_screen_name STRING 
) 
    ROW FORMAT SERDE 'com.cloudera.hive.serde.JSONSerDe' 
    LOCATION '/user/cloudera/flume/tweets';