kafka kraft 集群报Received request for offset 5287 for partition @metadata-0, but we only have log segments in the range 5288 to 5352.

Destiny 发表于: 2022-11-24   最后更新时间: 2022-11-24 09:54:05   1,484 游览

kraft集群,三节点里有两个节点都是报这错,然后重启两个节点就这样报错然后退出了

[2022-11-24 14:00:00,289] INFO [broker-3-to-controller-send-thread]: Recorded new controller, from now on will use broker 172.16.106.162:19093 (id: 3 rack: null) (kafka.server.BrokerToControllerRequestThread)
[2022-11-24 14:00:00,290] INFO [Controller 3] unable to start processing processBrokerHeartbeat because of NotControllerException. (org.apache.kafka.controller.QuorumController)
[2022-11-24 14:00:00,290] INFO [broker-3-to-controller-send-thread]: Recorded new controller, from now on will use broker 172.16.106.162:19093 (id: 3 rack: null) (kafka.server.BrokerToControllerRequestThread)
[2022-11-24 14:00:00,303] ERROR [RaftManager nodeId=3] Caught unexpected error in fetch completion of request FetchPartition(partition=0, currentLeaderEpoch=210983, fetchOffset=5288, lastFetchedEpoch=307, logStartOffset=-1, partitionMaxBytes=0) (org.apache.kafka.raft.Ka
fkaRaftClient)
org.apache.kafka.common.KafkaException: java.io.EOFException: Failed to read `log header` from file channel `sun.nio.ch.FileChannelImpl@74fcb0f`. Expected to read 17 bytes, but reached end of file after reading 1 bytes. Started read from position 0.
        at org.apache.kafka.common.record.RecordBatchIterator.makeNext(RecordBatchIterator.java:40)
        at org.apache.kafka.common.record.RecordBatchIterator.makeNext(RecordBatchIterator.java:24)
        at org.apache.kafka.common.utils.AbstractIterator.maybeComputeNext(AbstractIterator.java:79)
        at org.apache.kafka.common.utils.AbstractIterator.hasNext(AbstractIterator.java:45)
        at org.apache.kafka.common.record.FileRecords.searchForOffsetWithSize(FileRecords.java:316)
        at kafka.log.LogSegment.translateOffset(LogSegment.scala:274)
        at kafka.log.LogSegment.read(LogSegment.scala:297)
        at kafka.log.Log.$anonfun$read$2(Log.scala:1662)
        at kafka.log.Log.read(Log.scala:2482)
        at kafka.raft.KafkaMetadataLog.read(KafkaMetadataLog.scala:64)
        at org.apache.kafka.raft.KafkaRaftClient.tryCompleteFetchRequest(KafkaRaftClient.java:1035)
        at org.apache.kafka.raft.KafkaRaftClient.handleFetchRequest(KafkaRaftClient.java:979)
        at org.apache.kafka.raft.KafkaRaftClient.handleRequest(KafkaRaftClient.java:1606)
        at org.apache.kafka.raft.KafkaRaftClient.handleInboundMessage(KafkaRaftClient.java:1652)
        at org.apache.kafka.raft.KafkaRaftClient.poll(KafkaRaftClient.java:2190)
        at kafka.raft.KafkaRaftManager$RaftIoThread.doWork(RaftManager.scala:53)
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
Caused by: java.io.EOFException: Failed to read `log header` from file channel `sun.nio.ch.FileChannelImpl@74fcb0f`. Expected to read 17 bytes, but reached end of file after reading 1 bytes. Started read from position 0.
        at org.apache.kafka.common.utils.Utils.readFullyOrFail(Utils.java:1078)
        at org.apache.kafka.common.record.FileLogInputStream.nextBatch(FileLogInputStream.java:69)


org.apache.kafka.common.errors.OffsetOutOfRangeException: Received request for offset 5287 for partition @metadata-0, but we only have log segments in the range 5288 to 5352.
[2022-11-24 14:55:23,080] ERROR [RaftManager nodeId=2] Caught unexpected error in fetch completion of request FetchPartition(partition=0, currentLeaderEpoch=211636, fetchOffset=5287, lastFetchedEpoch=0, logStartOffset=-1, partitionMaxBytes=0) (org.apache.kafka.raft.KafkaRaftClient)
org.apache.kafka.common.errors.OffsetOutOfRangeException: Received request for offset 5287 for partition @metadata-0, but we only have log segments in the range 5288 to 5352.
[2022-11-24 14:55:23,084] INFO [Controller 2] unable to start processing registerBroker because of NotControllerException. (org.apache.kafka.controller.QuorumController)
[2022-11-24 14:55:23,100] INFO [RaftManager nodeId=2] Completed transition to Unattached(epoch=211637, voters=[1, 2, 3], electionTimeoutMs=1046) (org.apache.kafka.raft.QuorumState)
[2022-11-24 14:55:23,100] INFO [RaftManager nodeId=2] Vote request VoteRequestData(clusterId=null, topics=[TopicData(topicName='@metadata', partitions=[PartitionData(partitionIndex=0, candidateEpoch=211637, candidateId=3, lastOffsetEpoch=0, lastOffset=5287)])]) is rejected (org.apache.kafka.raft.KafkaRaftClient)
[2022-11-24 14:55:23,271] INFO [ThrottledChannelReaper-ControllerMutation]: Stopped (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2022-11-24 14:55:23,271] INFO [ThrottledChannelReaper-ControllerMutation]: Shutdown completed (kafka.server.ClientQuotaManager$ThrottledChannelReaper)
[2022-11-24 14:55:23,272] INFO [SocketServer listenerType=BROKER, nodeId=2] Shutting down socket server (kafka.network.SocketServer)
[2022-11-24 14:55:23,282] INFO [SocketServer listenerType=BROKER, nodeId=2] Shutdown completed (kafka.network.SocketServer)
[2022-11-24 14:55:23,285] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics)
[2022-11-24 14:55:23,285] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics)
[2022-11-24 14:55:23,285] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics)
[2022-11-24 14:55:23,287] INFO Broker and topic stats closed (kafka.server.BrokerTopicStats)
[2022-11-24 14:55:23,287] INFO [BrokerLifecycleManager id=2] closed event queue. (org.apache.kafka.queue.KafkaEventQueue)
[2022-11-24 14:55:23,288] INFO App info kafka.server for 2 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2022-11-24 14:55:23,288] INFO [BrokerServer id=2] shut down completed (kafka.server.BrokerServer)
[2022-11-24 14:55:23,288] ERROR Exiting Kafka. (kafka.Kafka$)

要如何解决?现在还有1个节点在支撑

发表于 2022-11-24
添加评论

数据目录被异常的清理了:

  1. log.dirs=/tmp,日志在/tmp目录下,Linux系统会自动清理里面的东西,导致数据异常。
  2. 手动清理 kafka下面的.log文件

请问,有定位到是什么问题导致的么

你的答案

查看kafka相关的其他问题或提一个您自己的问题