Miss

0 声望

这家伙太懒，什么都没留下

个人动态
  • 半兽人 回复 Miss 在 kafka poll records count 经常为0，只从一个partition中拉取数据，broker 中数据充足 中 ：

    准确的说，客户端请求kafka的offset的位置，这是由客户端控制。

    2年前
  • Miss 回复 半兽人 在 kafka poll records count 经常为0，只从一个partition中拉取数据，broker 中数据充足 中 ：

    这个意思是offset 不提交，就不会下次拉取？

    2年前
  • 半兽人 回复 Miss 在 kafka poll records count 经常为0，只从一个partition中拉取数据，broker 中数据充足 中 ：

    你这个是offset还没来得及提交，就又去拉了。
    改成手动提交offset，保证下次拉取前，offset提交成功。

    2年前
  • Miss 回复 半兽人 在 kafka poll records count 经常为0，只从一个partition中拉取数据，broker 中数据充足 中 ：

    这个是测试代码 和 log，发现poll count 为0 的频率 比较固定，不知道是哪里出的问题。谢谢了

    Properties kafkaProperties = KafkaConfigurer.getKafkaProperties(configPath);

KafkaConsumer<String, ByteBuffer> consumer = new KafkaConsumer<>(kafkaProperties);
consumer.subscribe(Lists.newArrayList("test"),new ConsumerRebalanceListener(){

    @Override
    public void onPartitionsRevoked(Collection<TopicPartition> partitions) {

    }

    @Override
    public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
        consumer.seek(new TopicPartition("test",0),0);

    }
});
while (true) {
    ConsumerRecords<String, ByteBuffer> records = consumer.poll(Duration.ofMillis(3000));
    log.info("========>" + records.count());
}

    2022-07-10_21:23:51.122 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:23:54.127 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:23:57.133 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:00.137 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:02.533 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.534 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.534 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.535 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.536 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.536 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.537 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.538 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.538 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.539 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.539 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:02.539 [main] INFO  a.m.q.c.ConsumerTest - ========>70
2022-07-10_21:24:05.540 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:08.541 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:11.544 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:14.549 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:16.965 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.966 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.967 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.967 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.968 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.969 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.969 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.970 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.971 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.971 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.972 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:16.972 [main] INFO  a.m.q.c.ConsumerTest - ========>111
2022-07-10_21:24:19.974 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:22.976 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:25.977 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:28.977 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:31.428 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.429 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.430 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.431 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.431 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.432 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.433 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.433 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.434 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.435 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.435 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:31.436 [main] INFO  a.m.q.c.ConsumerTest - ========>141
2022-07-10_21:24:34.438 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:37.442 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:40.445 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:43.448 [main] INFO  a.m.q.c.ConsumerTest - ========>0
2022-07-10_21:24:45.281 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.282 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.283 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.283 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.284 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.285 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.285 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.286 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.286 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.287 [main] INFO  a.m.q.c.ConsumerTest - ========>500
2022-07-10_21:24:45.287 [main] INFO  a.m.q.c.ConsumerTest - ========>500
    2年前
  • Miss 回复 半兽人 在 kafka poll records count 经常为0，只从一个partition中拉取数据，broker 中数据充足 中 ： 
    partition CURRENT-OFFSET LOG-END-OFFSET LAG
0 89959331 97139009 7179678

    这个就是的7179678 LAG

    2年前
  • 发表了 kafka poll records count 经常为0，只从一个partition中拉取数据，broker 中数据充足  
    2年前