2013-06-13 73 views
2

我在Hive 0.11中使用Hadoop 0.20.2。我已成功插入配置單元表中的一些csv文件的配置單元/ hdfs。完美選擇並加入工作。當嘗試分析一些數據,我需要利用內置的蜂房一樣的功能:Hive內置函數不起作用

  • SUBSTR
  • TO_DATE
  • 蘭特

例如:

select sid, request_id, to_date(times), to_unix_timestamp(times) from contents where sid = '5000000032066010373'; 

sid請求ID是字符串這裏,使用這些功能時,timestamp列 Unfortanetely我只得到錯誤(總是相同的錯誤堆棧):

java.lang.RuntimeException: Error in configuring object 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) 
     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) 
     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) 
     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) 
     at org.apache.hadoop.mapred.Child.main(Child.java:170) 
Caused by: java.lang.reflect.InvocationTargetException 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:601) 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) 
     ... 5 more 
Caused by: java.lang.RuntimeException: Error in configuring object 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) 
     at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) 
     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) 
     at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) 
     ... 10 more 
Caused by: java.lang.reflect.InvocationTargetException 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:601) 
     at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) 
     ... 13 more 
Caused by: java.lang.RuntimeException: Map operator initialization failed 
     at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) 
     ... 18 more 
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory 
     at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.<clinit>(GenericUDTFJSONTuple.java:56) 
     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
     at java.lang.reflect.Constructor.newInstance(Constructor.java:525) 
     at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) 
     at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:526) 
     at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:520) 
     at org.apache.hadoop.hive.ql.exec.FunctionRegistry.<clinit>(FunctionRegistry.java:423) 
     at org.apache.hadoop.hive.ql.exec.DefaultUDFMethodResolver.getEvalMethod(DefaultUDFMethodResolver.java:59) 
     at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:154) 
     at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:111) 
     at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:141) 
     at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:970) 
     at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:996) 
     at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) 
     at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) 
     at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:78) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) 
     at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) 
     at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) 
     at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543) 
     at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) 
     at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100) 
     ... 18 more 
Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.JsonFactory 

我到底錯在這裏做什麼?

SHOW FUNCTIONS; 

讓我發現,這些功能在那裏...

+0

你可以檢查,如果你有傑克遜核心ASL。? – zsxwing

+0

是的,版本1.8.8有... – Wolli

+0

試圖降級到配置單元0.10.0,同樣的錯誤信息 – Wolli

回答

0

嘗試在HIVE_HOME/lib下的那些更新傑克遜的* .jar在HADOOP_HOME/lib下,重啓Hadoop和嘗試提交作業。

0

我同時使用蜂巢+ Apache的Hadoop的遇到了同樣的問題0.20.2

的問題是Hadoop的0.20.2沒有lib文件夾中傑克遜罐子,所以無論你在手動添加每次都需要蜂房控制檯:

add jars $HIVE_HOME/lib/jackson-core-asl-1.8.8.jar $HIVE_HOME/lib/jackson-jaxrs-1.8.8.jar $HIVE_HOME/lib/jackson-mapper-asl-1.8.8.jar $HIVE_HOME/lib/jackson-xc-1.8.8.jar; 

或者您將所有這些jar複製到集羣中的每臺機器上?

0

嘗試使用HIVE_HOME/lib中的更新HADOOP_HOME/lib中的jackson * .jar,重新啓動Hadoop並嘗試提交作業。在$ HIVE_HOME/lib目錄\ * \ *。*罐子 -

由於它的工作