2
我正嘗試從hdfs文件中加入hbase表。我對hbase和豬比較陌生。與Pig中的hbase表聯接錯誤：沒有FileSystem for方案：hbase
set hbase.zookeeper.quorum 'mprhdp2-dev.local'
set hbase.zookeeper.property.clientPort 5100
A = LOAD '/raw/JoinTest2.txt' USING PigStorage('\t') AS (Id:chararray, text:chararray);
B = LOAD 'hbase://MediaInformationMock' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('MediaInformation:UmbrellaCode', '-loadKey true -caster=HBaseBinaryConverter') AS (Id:chararray, UmbrellaCode:chararray);
J = JOIN B BY $0, A BY $0;
DUMP J;
2012-10-02 08:23:59,923 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1003: Unable to find an operator for alias J
Details at logfile: /home/user1/pig_1349183747159.log
下面是它在日誌文件中說：
Pig Stack Trace
---------------
ERROR 2017: Internal error creating job configuration.
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to open iterator for alias J
at org.apache.pig.PigServer.openIterator(PigServer.java:857)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:682)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:490)
at org.apache.pig.Main.main(Main.java:111)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Caused by: org.apache.pig.PigException: ERROR 1002: Unable to store alias J
at org.apache.pig.PigServer.storeEx(PigServer.java:956)
at org.apache.pig.PigServer.store(PigServer.java:919)
at org.apache.pig.PigServer.openIterator(PigServer.java:832)
... 12 more
Caused by: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR 2017: Internal error creating job configuration.
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:739)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:259)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:180)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1270)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1255)
at org.apache.pig.PigServer.storeEx(PigServer.java:952)
... 14 more
Caused by: java.io.IOException: No FileSystem for scheme: hbase
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1416)
at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:69)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:1453)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1435)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:232)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:191)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getTotalInputFileSize(JobControlCompiler.java:788)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.estimateNumberOfReducers(JobControlCompiler.java:762)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:600)
... 19 more
任何想法？
豬版本0.10.0 HBase的0.92.1 Hadoop的0.20.2-dev的
DUMP A:
(1,A)
(2,B)
(3,C)
(4,D)
(5,E)
DUMP B:
(1,S)
(10,C)
(11,D)
(12,D)
(13,M)
(14,N)
(15,W)
(16,I)
(17,I)
(18,R)
(19,T)
(2,S)
(20,D)
(21,I)
(22,S)
(23,S)
(24,S)
(25,S)
(3,S)
(4,S)
(5,S)
(6,S)
(7,R)
(8,S)
(9,T)
對於在尋找[錯誤1066：無法打開迭代器別名]時發現此帖子的人（http://stackoverflow.com/questions/34495085/error-1066-unable-to-open-iterator-for- alias-in-pig-generic-solution）這裏是一個[通用解決方案]（http://stackoverflow.com/a/34495086/983722）。 –