卡夫卡kafka.common.MessageSizeTooLargeException在消費者

Question

對於一些較大的消息我遇到了以下錯誤：

kafka.common.MessageSizeTooLargeException: Message size is 1185198 bytes which exceeds the maximum configured message size of 1000012. 

所以按照this thread在經紀人和消費者增加了郵件大小：

fetch.message.max.bytes=10485760 
replica.fetch.max.bytes=10485760 
message.max.bytes=10485760 

添加到config/server.properties

但是然後消息通過但c onsumer出錯誤：

[2015-08-26 21:08:08,722] ERROR Error processing message, stopping consumer: (kafka.tools.ConsoleConsumer$) 
kafka.common.MessageSizeTooLargeException: Found a message larger than the maximum fetch size of this consumer on topic xyz partition 0 at fetch offset 29. Increase the fetch size, or decrease the maximum message size the broker will allow. 
    at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:90) 
    at kafka.consumer.ConsumerIterator.makeNext(ConsumerIterator.scala:33) 
    at kafka.utils.IteratorTemplate.maybeComputeNext(IteratorTemplate.scala:66) 
    at kafka.utils.IteratorTemplate.hasNext(IteratorTemplate.scala:58) 
    at scala.collection.Iterator$class.foreach(Iterator.scala:660) 
    at kafka.utils.IteratorTemplate.foreach(IteratorTemplate.scala:32) 
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:73) 
    at kafka.consumer.KafkaStream.foreach(KafkaStream.scala:25) 
    at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:166) 
    at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala) 
Consumed 3 messages 

似乎消費者並不拿起fetch.message.max.bytes=10485760

kafka_2.9.1-0.8.2.1

任何指針？

Answer 1

您不應該將fetch.message.max.bytes設置爲config/server.properties，而是設置爲您的ConsumerConfig（詳情請參閱here）。如果您使用控制檯使用者，則可能會傳遞一個--consumer.config consumer.properties標誌，其中consumer.properties文件將包含此配置值。