优雅的关闭kafka

半兽人 发表于: 2015-03-10   最后更新时间: 2017-04-11  
  •   80 订阅,4120 游览

The Kafka cluster will automatically detect any broker shutdown or failure and elect new leaders for the partitions on that machine. This will occur whether a server fails or it is brought down intentionally for maintenance or configuration changes. For the later cases Kafka supports a more graceful mechanism for stoping a server then just killing it. When a server is stopped gracefully it has two optimizations it will take advantage of:

Kafka集群自动检测broker关闭或者失败,并且在该机器上的分区选举新的leaders,当服务器出现故障或故意进行维护或配置更改时,为这种情况,Kafka支持一个更优雅机制关闭服务然后kill它,当服务器正常停止,它有2个最佳:

  1. It will sync all its logs to disk to avoid needing to do any log recovery when it restarts (i.e. validating the checksum for all messages in the tail of the log). Log recovery takes time so this speeds up intentional restarts.

    它把所有日志同步到磁盘里,当重启时,以避免需要做任何的日志恢复。日志恢复需要时间,所以这样可以加快有意启动。

  2. It will migrate any partitions the server is the leader for to other replicas prior to shutting down. This will make the leadership transfer faster and minimize the time each partition is unavailable to a few milliseconds.
    在关闭之间,它将所有leader分区服务器移动到其他的副本,并且把每个分区不可用的几毫秒的时间降至更低。


Syncing the logs will happen automatically happen whenever the server is stopped other than by a hard kill, but the controlled leadership migration requires using a special setting:

当发生服务器停止不是通过直接kill,就会自动同步日志,但是leader迁移需要使用特殊的设置:

    controlled.shutdown.enable=true


Note that controlled shutdown will only succeed if all the partitions hosted on the broker have replicas (i.e. the replication factor is greater than 1 and at least one of these replicas is alive). This is generally what you want since shutting down the last replica would make that topic partition unavailable.

注意,控制关闭broker和副本上的所有分区才行(即,副本大于1并且这些副本至少一个活着)。这通常因为你关闭最后一个副本将使这个主题分区不可用。







发表于: 1年前   最后更新时间: 1月前   游览量:4120
上一条: kafka修改删除topic
下一条: kafka平衡leader
评论…

  • 评论…
    • in this conversation
      提问