ERROR Failed to clean up log for __consumer_offsets-41 in dir D:\kafka\node2\logs due to IOException (kafka.server.LogDirFailureChannel)

Tobey 发表于: 2019-12-04   最后更新时间: 2019-12-04 16:47:07   3,436 游览

版本kafka2.1 windows

查看kafka tool三个节点的kafka集群现在只剩下一个节点正常,两个在kafka tool没有显示的节点在windows server上还是运行状态,查看两个没有在kafka tool显示的两个节点的日志其中内容:

[2019-12-04 16:00:02,119] ERROR Failed to clean up log for __consumer_offsets-41 in dir D:\kafka\node2\logs due to IOException (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: D:\kafka\node2\logs\__consumer_offsets-41\00000000000000000000.timeindex.cleaned: 
    The process cannot access the file because it is being used by another process.
    at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
    at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
    at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:269)
    at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108)
    at java.nio.file.Files.deleteIfExists(Files.java:1165)
    at kafka.log.Log$.deleteFileIfExists(Log.scala:2272)
    at kafka.log.LogSegment$.deleteIfExists(LogSegment.scala:644)
    at kafka.log.LogCleaner$.createNewCleanedSegment(LogCleaner.scala:421)
    at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:537)
    at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:512)
    at kafka.log.Cleaner$$anonfun$doClean$4.apply(LogCleaner.scala:511)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at kafka.log.Cleaner.doClean(LogCleaner.scala:511)
    at kafka.log.Cleaner.clean(LogCleaner.scala:489)
    at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:350)
    at kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:319)
    at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:300)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82)
发表于 2019-12-04
添加评论

失败的清理log,因为被其它的程序占用。
你用啥工具手动清理的?

它自动的啊 没有手动操作过

半兽人 -> Tobey 5年前

https://stackoverflow.com/questions/45599625/kafka-unable-to-start-kafka-process-can-not-access-file-00000000000000000000
看样子这个版本和问题都跟你一样,但是没解决方案。
主动删除目录,消息都会丢失,虽然是好了,但是后面可能还会出现。

如果重建环境,还有要清理下zk,保障干净,没有数据混乱。

你的答案

查看kafka相关的其他问题或提一个您自己的问题