kafka-consumer-perf-test.sh脚本遇到OutOfMemoryError

一如乞人不需要形象 发表于: 2020-12-29   最后更新时间: 2020-12-29 13:47:13   2,919 游览

大佬，我在使用kafka-consumer-perf-test.sh做消费者performance 测试时报java.lang.OutOfMemoryError，想问是什么原因，怎么解决？

 kafka-consumer-perf-test.sh --broker-list localhost:8083 --messages 100000 --topic test

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
    at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
    at org.apache.kafka.common.memory.MemoryPool$1.tryAllocate(MemoryPool.java:30)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:113)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:448)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:398)
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:549)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:262)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:236)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:463)
    at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1275)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1241)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216)
    at kafka.tools.ConsumerPerformance$.consume(ConsumerPerformance.scala:126)
    at kafka.tools.ConsumerPerformance$.main(ConsumerPerformance.scala:56)
    at kafka.tools.ConsumerPerformance.main(ConsumerPerformance.scala)
kafka
发表于 2020-12-29
一如乞人不需要形象
添加评论

内存不足了，到脚本里面调整一下jvm

回复
回答于 3年前
半兽人

额，不是很懂脚本，改这个吗 KAFKA_HEAP_OPTS="-Xmx512M"？

#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx512M"
fi
exec $(dirname $0)/kafka-run-class.sh kafka.tools.ConsumerPerformance "$@"
0 0 回复

我现在没在电脑旁，你这样加看上去也没问题的。
你进到kafka-run-class.sh一看便知。
有效后，反馈一下吧

0 0 回复

可以，类似于这样

KAFKA_OPTS=-Xmx4096M kafka-consumer-perf-test.sh --broker-list localhost:9092 --messages 100 --topic DCS-ORACLE-CDC-HF3WD -timeout 300000

32 0 回复
你的答案

查看kafka相关的其他问题或提一个您自己的问题。
提问

找不到想要的答案？提一个您自己的问题。