2
我試圖檢索使用水槽SocialAgent Facebook的數據Facebook的數據。我已經成功地使用TwitterAgent檢索twitter數據。水槽不取使用SocialAgent
但在Facebook上的情況下，我沒有什麼HDFS中。
我的終端停留在這個階段。當我結束這個過程中，我得到這個
我的HDFS的Facebook文件夾empty.I現在用以下flume.conf
SocialAgent.sources = FacebookHttpSource Twitter
SocialAgent.channels = FBmemoryChannel MemChannel
SocialAgent.sinks = fbHDFS HDFS
# For each one of the sources, the type is defined
SocialAgent.sources.FacebookHttpSource.type = org.apache.flume.source.http.HTTPSource
SocialAgent.sources.FacebookHttpSource.port = 51400
SocialAgent.sources.FacebookHttpSource.interceptors = Ts
SocialAgent.sources.FacebookHttpSource.interceptors.Ts.type = org.apache.flume.interceptor.TimestampInterceptor$Builder
# The channel can be defined as follows.
SocialAgent.sources.FacebookHttpSource.channels = FBmemoryChannel
# Each sink's type must be defined
#Specify the channel the sink should use
SocialAgent.sinks.fbHDFS.channel = FBmemoryChannel
SocialAgent.sinks.fbHDFS.type = hdfs
SocialAgent.sinks.fbHDFS.hdfs.path = hdfs://localhost:9000/user/flume/facebook/%Y/%m/%d/%H/
SocialAgent.sinks.fbHDFS.hdfs.fileType = DataStream
SocialAgent.sinks.fbHDFS.hdfs.writeFormat = Text
SocialAgent.sinks.fbHDFS.hdfs.batchSize = 1000
SocialAgent.sinks.fbHDFS.hdfs.rollSize = 0
SocialAgent.sinks.fbHDFS.hdfs.rollCount = 10000
# Each channel's type is defined.
SocialAgent.channels.FBmemoryChannel.type = memory
# Other config values specific to each type of channel(sink or source)
# can be defined as well
# In this case, it specifies the capacity of the memory channel
SocialAgent.channels.FBmemoryChannel.capacity = 10000
SocialAgent.channels.FBmemoryChannel.transactionCapacity = 100
是否有任何其他水槽代理獲取Facebook的數據。
對不起，很長一段時間後，打擾你在這個問題上，我希望你在這個任務中得到成功，如果這樣可以請你分享如何做所有的過程，即從Facebook提取數據，我的意思是所有的步驟吧。我確實知道關於從twitter中提取數據的過程。請幫助完成這項任務。如果無法提及所有步驟/流程，請發送電子郵件至[email protected]。提前致謝。 – Dinesh 2016-05-17 09:33:23