2016-07-08 68 views
1

我正在嘗試閱讀Apache Pig Latin中具有非分隔ascii組成的每一行的文本文件。也就是說,該行中的每一列都在該行的特定位置開始和結束。閱讀非分隔asciif文件Apache Pig拉丁文

樣品定義:

+--------+----------------+--------------+ 
| Column | Start Position | End Position | 
+--------+----------------+--------------+ 
| A  | 1    | 6   | 
+--------+----------------+--------------+ 
| B  | 8    | 11   | 
+--------+----------------+--------------+ 
| C  | 13    | 15   | 
+--------+----------------+--------------+ 

樣本數據:

+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+ 
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+ 
| s | a | m | p | l | e | | d | a | t | a | | | h | i | 
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+ 
| d | u | d | e | | | | hi | | | | | b | r | o | 
+---+---+---+---+---+---+---+----+---+----+----+----+----+----+----+ 

預期輸出:

sample, data, hi 
dude, hi, bro 

如何在豬閱讀本? PigStorage似乎沒有足夠的靈活性來允許位置分隔,只有字符串分隔(逗號,製表符等)。

回答

0

看起來像Apache提供這種特定使用案例裝載機:

LOAD 'data.txt' USING org.apache.pig.piggybank.storage.FixedWidthLoader('1-6, 8-11, 13-15', 'SKIP_HEADER') AS (a, b, c); 

https://pig.apache.org/docs/r0.16.0/api/