kafka消费时候,报Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. T

小蕊 发表于: 2016-11-18   最后更新时间: 2016-11-18  
  •   25 订阅,5579 游览

设置的是手动提交offset,这一行报错

consumer.commitSync 

报了以下错误

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.

导致我的offset回滚了,引起大量的重复消费,怎么解决?






征集:如果你也是一个热爱分享文章的人,愿意参与贡献文章来补充完善本网的内容,欢迎入群:14085484

发表于: 1年前   最后更新时间: 1年前   游览量:5579
上一条: kafka 消费端问题
下一条: flume里面的channel选用了kafkaChannel报错

评论…


  • 如何解决呢?
    poll()调用太慢,commitSync同步offset的时候已经关闭了,因为kafka是按批来拉取的消息,一批数据处理的时间过长,导致很久没有poll,导致的。
  • 评论…
    • in this conversation
      提问