Flume每個文件分配十行

Question

我在做什麼： 我想flume每64 MB數據只寫一個文件。

什麼是水槽這樣做的： 創建每10個事件

我配置一個新的文件：

a1.sources = r1 
a1.sinks = k1 
a1.channels = c1 

# Describe/configure the source 
a1.sources.r1.type = netcat 
a1.sources.r1.bind = localhost 
a1.sources.r1.port = 44444 

# Describe the sink 
a1.sinks.k1.type = hdfs 
a1.sinks.k1.channel = c1 
a1.sinks.k1.hdfs.path = hdfs://lawn-128-61-47-216.lawn.gatech.edu:9000/flume/events/ 
a1.sinks.k1.hdfs.filePrefix = events- 
a1.sinks.k1.hdfs.round = true 
a1.sinks.k1.hdfs.roundValue = 10 
a1.sinks.k1.hdfs.roundUnit = minute 
a1.sinks.k1.hdfs.fileType = DataStream 
a1.sinks.k1.hdfs.writeFormat = Text 
a1.sinks.k1.hdfs.hdfs.rollInterval = 0 
a1.sinks.k1.hdfs.hdfs.rollSize = 67108864 
a1.sinks.k1.hdfs.hdfs.rollCount = 0 
a1.sinks.k1.hdfs.idleTimeout = 0 
a1.sinks.k1.hdfs.batchSize = 1000 
a1.sinks.k1.hdfs.serializer = text 
a1.sinks.k1.sink.serializer.appendNewline = true 




# Use a channel which buffers events in memory 
a1.channels.c1.type = memory 
a1.channels.c1.capacity = 1000 
a1.channels.c1.transactionCapacity = 1000 

# Bind the source and sink to the channel 
a1.sources.r1.channels = c1 
a1.sinks.k1.channel = c1 

Answer 1

有重複的 「HDFS」 令牌

 

    a1.sinks.k1.hdfs.hdfs.rollInterval = 0 
    a1.sinks.k1.hdfs.hdfs.rollSize = 67108864 
    a1.sinks.k1.hdfs.hdfs.rollCount = 0 

如何改變那些到

 

    a1.sinks.k1.hdfs.rollInterval = 0 
    a1.sinks.k1.hdfs.rollSize = 67108864 
    a1.sinks.k1.hdfs.rollCount = 0