0
我想將文件從hdfs文件系統複製到本地file系統。我在Java DSL中嘗試了一個簡單的路由。駱駝hdfs2文件協議文件傳輸
from("hdfs2://192.168.1.55:9000/user?owner=honey")
.to("file:/home/honey/Desktop/DATA");
但它不工作。這些是日誌。
2014-11-13 22:16:22,169 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] util.Shell (Shell.java:326) - setsid exited with exit code 0
2014-11-13 22:16:22,186 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] hdfs2.HdfsConsumer (HdfsConsumer.java:87) - Connected to hdfs file-system 192.168.1.55:9000/hdfs://192.168.1.55:9000/user
2014-11-13 22:16:22,189 INFO [main] hdfs2.HdfsConsumer (HdfsConsumer.java:84) - Connected to hdfs file-system 192.168.1.55:9000/hdfs://192.168.1.55:9000/user
2014-11-13 22:16:22,189 INFO [main] impl.DefaultCamelContext (DefaultCamelContext.java:2310) - Route: route1 started and consuming from: Endpoint[hdfs2://192.168.1.55:9000/user?owner=honey]
2014-11-13 22:16:22,190 DEBUG [main] management.DefaultManagementLifecycleStrategy$TimerListenerManagerStartupListener (DefaultManagementLifecycleStrategy.java:893) - Load performance statistics disabled
2014-11-13 22:16:22,190 INFO [main] impl.DefaultCamelContext (DefaultCamelContext.java:1654) - Total 1 routes, of which 1 is started.
2014-11-13 22:16:22,192 INFO [main] impl.DefaultCamelContext (DefaultCamelContext.java:1655) - Apache Camel 2.13.0 (CamelContext: camel-1) started in 1.521 seconds
2014-11-13 22:16:22,202 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] ipc.Client$Connection (Client.java:371) - The ping interval is 60000 ms.
2014-11-13 22:16:22,204 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] ipc.Client$Connection (Client.java:636) - Connecting to /192.168.1.55:9000
2014-11-13 22:16:22,218 DEBUG [IPC Client (1506043150) connection to /192.168.1.55:9000 from honey] ipc.Client$Connection (Client.java:886) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey: starting, having connections 1
2014-11-13 22:16:22,220 DEBUG [IPC Parameter Sending Thread #0] ipc.Client$Connection$3 (Client.java:948) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey sending #0
2014-11-13 22:16:22,227 DEBUG [IPC Client (1506043150) connection to /192.168.1.55:9000 from honey] ipc.Client$Connection (Client.java:1005) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey got value #0
2014-11-13 22:16:22,227 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] ipc.ProtobufRpcEngine$Invoker (ProtobufRpcEngine.java:221) - Call: getFileInfo took 36ms
2014-11-13 22:16:22,253 DEBUG [IPC Parameter Sending Thread #0] ipc.Client$Connection$3 (Client.java:948) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey sending #1
2014-11-13 22:16:22,255 DEBUG [IPC Client (1506043150) connection to /192.168.1.55:9000 from honey] ipc.Client$Connection (Client.java:1005) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey got value #1
2014-11-13 22:16:22,255 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] ipc.ProtobufRpcEngine$Invoker (ProtobufRpcEngine.java:221) - Call: getFileInfo took 2ms
2014-11-13 22:16:22,256 DEBUG [IPC Parameter Sending Thread #0] ipc.Client$Connection$3 (Client.java:948) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey sending #2
2014-11-13 22:16:22,257 DEBUG [IPC Client (1506043150) connection to /192.168.1.55:9000 from honey] ipc.Client$Connection (Client.java:1005) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey got value #2
2014-11-13 22:16:22,257 DEBUG [Camel (camel-1) thread #0 - hdfs2://192.168.1.55:9000/user] ipc.ProtobufRpcEngine$Invoker (ProtobufRpcEngine.java:221) - Call: getFileInfo took 1ms
2014-11-13 22:16:22,259 DEBUG [IPC Parameter Sending Thread #0] ipc.Client$Connection$3 (Client.java:948) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey sending #3
2014-11-13 22:16:22,261 DEBUG [IPC Client (1506043150) connection to /192.168.1.55:9000 from honey] ipc.Client$Connection (Client.java:1005) - IPC Client (1506043150) connection to /192.168.1.55:9000 from honey got value #3
so on.....
在這些下面的日誌中不斷重複。但同樣的事情，如果我試圖複製文件從本地file systemhdfs然後它的工作完美無瑕。任何人都可以提供任何工作鏈接或示例嗎？這將是非常有益的。
編輯： 現在它爲我工作。 readSuffix was read by default，它不會再讀取已經被讀取的文件。現在我的問題是爲什麼有什麼recursive option in consumer side.我想以遞歸方式使用文件。就像我們可以使用遞歸選項在File2中掃描一樣。
@ claus-ibsen你想告訴這裏的東西？ ：P 你的幫助在這裏非常明顯。 –