生产项目上kafka单体消息比较大,而且消息发送频率也比较高,近期cpu总是飙升,影响服务正常使用,使用jstack导出线程信息,发现如下有用信息:
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.append(RecordAccumulator.java:196)
- waiting to lock <0x00000005c360ac88> (a java.util.ArrayDeque)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:830)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:784)
at org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer.send(DefaultKafkaProducerFactory.java:170)
at org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:245)
at org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:151)
以及
"kafka-producer-network-thread | producer-1" #79 daemon prio=5 os_prio=0 tid=0x00007f1be46ea800 nid=0x23e2 waiting for monitor entry [0x00007f1c7c8bc000]
java.lang.Thread.State: BLOCKED (on object monitor)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.expiredBatches(RecordAccumulator.java:283)
- waiting to lock <0x00000005c360ab48> (a java.util.ArrayDeque)
at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:280)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:238)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
at java.lang.Thread.run(Thread.java:748)
以上问题不知道怎么引起的。
@半兽人,麻烦帮我看看,谢谢
这个问题主要是单纯你上面的信息是看不出来的,kafka组装合并信息是会消耗cpu,但是一般也不会那么高(因为不涉及到磁盘io,走内存)。如果内存允许的话,调大允许发送大小,减少每个时间窗口组装合并消息的频次。
另外,定位消耗cpu的线程方式:
https://www.orchome.com/833
你的答案