kafka消费时候,报Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. T

小蕊 发表于: 2016-11-18   最后更新时间: 2016-11-18  
  •   8 订阅,1907 游览

设置的是手动提交offset,这一行报错

consumer.commitSync 

报了以下错误

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

导致我的offset回滚了,引起大量的重复消费,怎么解决?







发表于: 7月前   最后更新时间: 7月前   游览量:1907
上一条: kafka 消费端问题
下一条: flume里面的channel选用了kafkaChannel报错
评论…

  • poll()调用太慢,commitSync同步offset的时候已经关闭了,因为kafka是按批来拉取的消息,一批数据处理的时间过长,导致很久没有poll,导致的。
    如何解决呢?
  • 评论…
    • in this conversation
      提问