Kafka Broker 日志警告无法消除 : Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch

1089 发表于: 2020-04-30   最后更新时间: 2021-09-18 10:36:42   4,640 游览

Broker日志提示大量警告

已查询该文章但未找到解决办法:https://www.orchome.com/1066

2020-04-30 16:50:37,010 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:16, offset:339490219592}, Current: {epoch:498, offset335305734105} for Partition: __consumer_offsets-47
2020-04-30 16:50:37,011 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:16, offset:339490219599}, Current: {epoch:498, offset335305734105} for Partition: __consumer_offsets-47
2020-04-30 16:50:37,011 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:14, offset:105463605}, Current: {epoch:336, offset104508994} for Partition: appDataToK3_test-12
2020-04-30 16:50:37,011 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:17, offset:92001836731}, Current: {epoch:479, offset86819324213} for Partition: __consumer_offsets-4
2020-04-30 16:50:37,012 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:16, offset:339490219606}, Current: {epoch:498, offset335305734105} for Partition: __consumer_offsets-47

描述

  1. kafka日常使用没有特殊操作

  2. 曾怀疑是follower 跟不上 leader 导致提示异常,查看topic ISR ,发现上面日志中 __consumer_offsets-47 有报警告,但是下面查看topic 分区47的 __consumer_offsets IRS是正常

     Topic: __consumer_offsets    Partition: 38    Leader: 153    Replicas: 153,154,150    Isr: 154,153
     Topic: __consumer_offsets    Partition: 39    Leader: 154    Replicas: 154,150,151    Isr: 154,151,150
     Topic: __consumer_offsets    Partition: 40    Leader: 150    Replicas: 150,152,153    Isr: 152,150,153
     Topic: __consumer_offsets    Partition: 41    Leader: 151    Replicas: 151,153,154    Isr: 151,154,153
     Topic: __consumer_offsets    Partition: 42    Leader: 152    Replicas: 152,154,150    Isr: 152,154,150
     Topic: __consumer_offsets    Partition: 43    Leader: 153    Replicas: 153,150,151    Isr: 151,153
     Topic: __consumer_offsets    Partition: 44    Leader: 154    Replicas: 154,151,152    Isr: 154,152,151
     Topic: __consumer_offsets    Partition: 45    Leader: 150    Replicas: 150,153,154    Isr: 154,150,153
     Topic: __consumer_offsets    Partition: 46    Leader: 151    Replicas: 151,154,150    Isr: 151,154,150
     Topic: __consumer_offsets    Partition: 47    Leader: 152    Replicas: 152,150,151    Isr: 152,150,151
     Topic: __consumer_offsets    Partition: 48    Leader: 153    Replicas: 153,151,152    Isr: 151,152,153
     Topic: __consumer_offsets    Partition: 49    Leader: 154    Replicas: 154,152,153    Isr: 154,152,153
    
  3. 下图是监控 JMX kafka.network:type=SocketServer,name=NetworkProcessorAvgIdlePercent 的值
    QQ截图20200430170630.png

问题:

  1. 什么问题导致该警告频繁打印,怎么能消除
  2. 如何解决 follower 跟不上leader问题,可以调整什么参数吗?
发表于 2020-04-30
添加评论

num.replica.fetchers 这个可以调大试试

無执 -> 無执 4年前

一般配置默认就可以的;出现较多、较长时间ISR不同步问题,检查一下Kafka集群性能吧。

这个一般是kafka一些版本的bug:

  • 1.1.0
  • 2.0.0

重新分配一下告警的分区,警告就会没有了,可参考。
https://www.orchome.com/454#item-7

1089 -> 半兽人 4年前

好的,感谢回答,还有个问题,就是kafka正常使用的情况下,如何解决 follower 跟不上leader问题,可以调整什么参数吗?

半兽人 -> 1089 4年前

节点满足这2个“同步”条件,以区分“活着”还是“故障”。leader跟踪“同步”节点。如果一个follower死掉,卡住,或落后,leader将从同步副本列表中移除它。落后是通过replica.lag.max.messages配置控制,卡住是通过replica.lag.time.max.ms配置控制的。

你的答案

查看kafka相关的其他问题或提一个您自己的问题