kafka消费者fetch大小不管设置多大都是实际拉下来都只能拉到3万7千条

Eliot 发表于: 2021-12-09 最后更新时间: 2021-12-11 23:44:28 1,095 游览

本机电脑16g，内存使用信息如下，不管max-partition-fetch-bytes配置多大，只能拉到3万多条，一条消息数据大小为 136k

本次拉取消息条数：37269，处理总耗时：5055ms

电脑内存信息：PhysMem: 15G used (2749M wired), 798M unused.

配置信息：

max-poll-records: ${SPRING_KAFKA_CONSUMER_MAX_POLL_RECORDS:500000}
session-time-out: ${SPRING_KAFKA_CONSUMER_SESSION_TIME_OUT:300000}
max-poll-interval: ${SPRING_KAFKA_CONSUMER_MAX_POLL_INTERVAL:200000}
max-partition-fetch-bytes: ${SPRING_KAFKA_CONSUMER_MAX_PARTITION_FETCH_BYTES:52428800}
fetch-max-wait: ${SPRING_KAFKA_CONSUMER_FETCH_MAX_WAIT:10000}
heartbeat-interval: ${SPRING_KAFKA_CONSUMER_HEARTBEAT_INTERVAL:6666}
request-timeout-ms: ${REQUEST_TIMEOUT_MS_CONFIG:400000}