咨询下,kafka broker宕机时间比较久,恢复后为了快速同步清理了该节点的data数据,__consumer_offsets
数据也被删掉了,应用侧倒是消费都正常,日志也没什么报错,但感觉是有隐患的。
所有broker data下都没有__consumer_offsets
数据了。
zk里/brokers/topics/__consumer_offsets
节点是有的下边就没有子节点了。
auto.create.topics.enable
是关闭的,所以没有再次创建__consumer_offsets
。
消费offset topic描述:
$ bin/kafka-topics.sh --bootstrap-server localhost:9092 --topic __consumer_offsets --describe
Topic:__consumer_offsets PartitionCount:50 ReplicationFactor:3 Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer MarkedForDeletion:true
Topic:__consumer_offsets Partition: 0 Leader: none Replicas: 7,2,3 Isr:
Topic:__consumer_offsets Partition: 1 Leader: none Replicas: 8,3,4 Isr:
重新检查了下11个节点data下都没有__consumer_offsets了,这个集群没怎么做过优化没改过特别的配置:
log.dirs=***
num.network.threads=24
num.io.threads=48
socket.send.buffer.bytes=10485760
socket.receive.buffer.bytes=10485760
socket.request.max.bytes=104857600
num.partitions=1
log.retention.hours=240
log.segment.bytes=1073741824
log.retention.check.interval.ms=60000
log.cleaner.enable=false
message.max.bytes =104857600
num.replica.fetchers=8
auto.create.topics.enable=false
zookeeper.session.timeout.ms=120000
auto.leader.rebalance.enable=false
replica.fetch.max.bytes=104857600
replica.fetch.min.bytes=1
controlled.shutdown.enable=true
controlled.shutdown.max.retries=3
controlled.shutdown.retry.backoff.ms=5000
zookeeper.connect=***
zookeeper.connection.timeout.ms=60000
start_log_dir=***
num.recovery.threads.per.data.dir=16
replica.lag.time.max.ms=60000
delete.topic.enable=true
zk数据:
/brokers/topics/__consumer_offsets
{"version":1,"partitions":{"45":[8,7,9],"34":[8,6,7],"12":[8,4,5],"8":[4,10,11],"19":[4,11,1],"23":[8,5,6],"4":[11,6,7],"40":[3,1,2],"15":[11,7,8],"11":[7,3,4],"9":[5,11,1],"44":[7,6,8],"33":[7,5,6],"22":[7,4,5],"26":[11,8,9],"37":[11,9,10],"13":[9,5,6],"46":[9,8,10],"24":[9,6,7],"35":[9,7,8],"16":[1,8,9],"5":[1,7,8],"10":[6,1,2],"48":[11,10,1],"21":[6,2,3],"43":[6,4,5],"32":[6,3,4],"49":[1,11,2],"6":[2,8,9],"36":[10,8,9],"1":[8,3,4],"39":[2,11,1],"17":[2,9,10],"25":[10,7,8],"14":[10,6,7],"47":[10,9,11],"31":[5,2,3],"42":[5,3,4],"0":[7,2,3],"20":[5,1,2],"27":[1,9,10],"2":[9,4,5],"38":[1,10,11],"18":[3,10,11],"30":[4,1,2],"7":[3,9,10],"29":[3,11,1],"41":[4,2,3],"3":[10,5,6],"28":[2,10,11]}}
这种情况应该怎么处理下呢,现在的思路是停生产消费应用,开启自动创建,重启kafka,重置消费组偏移量到最新日期避免消费之前的数据,大神帮看看这样有啥问题没?
kafka是0.11.0版本的 kafka_2.11-0.11.0.0.jar,一月份出的故障kafka启动后没有什么异常,应用能正常消费,我检查下日志情况。
现在什么情况了?
有个思路,可以考虑消费者offset重写到kafka或者zookeeper里。
参考:迁移offset从zookeeper到kafka
你的答案