kafka检查消费者位置

半兽人 发表于: 2015-03-10   最后更新时间: 2017-04-11  
  •   85 订阅,4328 游览
Sometimes it's useful to see the position of your consumers. We have a tool that will show the position of all consumers in a consumer group as well as how far behind the end of the log they are. To run this tool on a consumer group named my-group consuming a topic named my-topic would look like this:
有时候需要去查看你的消费者的位置。我们有一个显示【消费者组中】所有消费者的位置的工具。显示日志其落后多远。消费者组名为my-group,消费者topic名为my-topic,如下:
 > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zkconnect localhost:2181 --group test
Group           Topic                          Pid Offset          logSize         Lag             Owner
my-group        my-topic                       0   0               0               0               test_jkreps-mn-1394154511599-60744496-0
my-group        my-topic                       1   0               0               0               test_jkreps-mn-1394154521217-1a0be913-0 


NOTE: Since 0.9.0.0, the kafka.tools.ConsumerOffsetChecker tool has been deprecated. You should use the kafka.admin.ConsumerGroupCommand (or the bin/kafka-consumer-groups.sh script) to manage consumer groups, including consumers created with the new consumer API.

注意:在0.9.0.0,kafka.tools.ConsumerOffsetChecker已经不支持了。你应该使用kafka.admin.ConsumerGroupCommand(或bin/kafka-consumer-groups.sh脚本)来管理消费者组,包括用新消费者API创建的消费者。


bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server localhost:9092 --describe --group test-consumer-group

Managing Consumer Groups

管理消费者组

With the ConsumerGroupCommand tool, we can list, describe, or delete consumer groups. Note that deletion is only available when the group metadata is stored in ZooKeeper. When using the new consumer API (where the broker handles coordination of partition handling and rebalance), the group is deleted when the last committed offset for that group expires. For example, to list all consumer groups across all topics:
用ConsumerGroupCommand工具,我们可以使用list,describe,或delete消费者组(注意,删除只有在分组元数据存储在zookeeper的才可用)。当使用新消费者API(broker协调处理分区和重新平衡),当该组的最后一个提交的偏移到期时,该组被删除。 例如,要列出所有主题中的所有用户组:

 > bin/kafka-consumer-groups.sh --bootstrap-server broker1:9092 --list

  test-consumer-group



To view offsets as in the previous example with the ConsumerOffsetChecker, we "describe" the consumer group like this:
要使用ConsumerOffsetChecker查看上一个示例中消费者组的偏移量,我们按如下所示“describe”消费者组:

  > bin/kafka-consumer-groups.sh --bootstrap-server broker1:9092 --describe --group test-consumer-group

  GROUP                          TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             OWNER
  test-consumer-group            test-foo                       0          1               3               2               consumer-1_/127.0.0.1


If you are using the old high-level consumer and storing the group metadata in ZooKeeper (i.e. offsets.storage=zookeeper), pass --zookeeper instead of bootstrap-server:
如果你使用老的高级消费者并存储分组元数据在zookeeper(即。offsets.storage=zookeeper)通过--zookeeper,而不是bootstrap-server:
  > bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --list








发表于: 1年前   最后更新时间: 2月前   游览量:4328
上一条: kafka镜像集群之间的数据
下一条: kafka扩大集群
评论…

  • 这是旧的版本吧
  • 评论…
    • in this conversation
      提问