topic的生产和消费目前状态正常,日志kafkaServer.out无异常,但是使用kafka-consumer-groups.sh
观察后发现。
- 大量topic offset标记不更新,
- 部分topic的lag出现负值
kafka 2.7.1,三节点集群
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group facade-order-api-consumer-group --describe | grep facade.order.apply.shipping.methods.reply
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 8 138 37 -101 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 7 127 159 32 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 4 137 170 33 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 2 122 33 -89 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 0 131 165 34 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 6 139 172 33 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 3 143 180 37 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 5 136 37 -99 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 9 144 181 37 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 1 131 162 31 consumer-2-632e60c7-3c7c-46d3-a109-cb02da3510db /10.77.29.190 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 16 136 166 30 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 11 136 33 -103 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 15 140 176 36 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 10 134 168 34 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 19 133 165 32 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 14 135 33 -102 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 18 130 163 33 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 13 154 191 37 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 12 153 190 37 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
facade-order-api-consumer-group facade.order.apply.shipping.methods.reply 17 133 32 -101 consumer-2-b7a37f0c-65dd-43be-ba10-ac23ad4288a0 /10.77.29.65 consumer-2
因未影响上下游生产和消费,未尝试任何操作进行干预
查看topic那些负数的分区是否是正常的。
bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic foo --describe
如果都正常,lag负数的分区offset也一直在增长,可查看lag为负数的原因:kafka consumer lag 为负数
你的答案