Sometimes it's useful to see the position of your consumers. We have a tool that will show the position of all consumers in a consumer group as well as how far behind the end of the log they are. To run this tool on a consumer group named my-group consuming a topic named my-topic would look like this:
 > bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zkconnect localhost:2181 --group test
Group           Topic                          Pid Offset          logSize         Lag             Owner
my-group        my-topic                       0   0               0               0               test_jkreps-mn-1394154511599-60744496-0
my-group        my-topic                       1   0               0               0               test_jkreps-mn-1394154521217-1a0be913-0 

NOTE: Since, the kafka.tools.ConsumerOffsetChecker tool has been deprecated. You should use the kafka.admin.ConsumerGroupCommand (or the bin/kafka-consumer-groups.sh script) to manage consumer groups, including consumers created with the new consumer API.


bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server localhost:9092 --describe --group test-consumer-group

Managing Consumer Groups


With the ConsumerGroupCommand tool, we can list, describe, or delete consumer groups. Note that deletion is only available when the group metadata is stored in ZooKeeper. When using the new consumer API (where the broker handles coordination of partition handling and rebalance), the group is deleted when the last committed offset for that group expires. For example, to list all consumer groups across all topics:
用ConsumerGroupCommand工具,我们可以使用list,describe,或delete消费者组(注意,删除只有在分组元数据存储在zookeeper的才可用)。当使用新消费者API(broker协调处理分区和重新平衡),当该组的最后一个提交的偏移到期时,该组被删除。 例如,要列出所有主题中的所有用户组:

 > bin/kafka-consumer-groups.sh --bootstrap-server broker1:9092 --list


To view offsets as in the previous example with the ConsumerOffsetChecker, we "describe" the consumer group like this:

  > bin/kafka-consumer-groups.sh --bootstrap-server broker1:9092 --describe --group test-consumer-group

  GROUP                          TOPIC                          PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             OWNER
  test-consumer-group            test-foo                       0          1               3               2               consumer-1_/

If you are using the old high-level consumer and storing the group metadata in ZooKeeper (i.e. offsets.storage=zookeeper), pass --zookeeper instead of bootstrap-server:
  > bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --list

发表于: 4年前   最后更新时间: 2年前   游览量:27522
上一条: kafka镜像集群之间的数据
下一条: kafka扩大集群

  • 0.9.0版以后的kafka默认把offset放在__consumer_offsets这个topic内 是不是就是--bootstrap-server这个选项?在实际使用时,好像用--zookeeper还是可以消费到数据的呀 它本质还是读的是__consumer_offsets吗?是否有参数指定这个offset到底存在哪里?
    我用mirrormaker同步的时候发现,消费组的CURRENT-OFFSET 一个分区有unknown 状态,怎么能不换消费组的情况让他消费到