前提:
一台broker服务器、一台生产者、无消费者
设置生产者的线程池为20 ,生产10000条数据,生产到8000条左右的时候就提示如下内容。
请问这个到底是哪个地方的连接数不够了呢?broker服务器上对如下都进行了设置 ,仍然没有解决下面的问题
#### I/O线程等待队列中的最大的请求数,超过这个数量,network线程就不会再接收一个新的请求
queued.max.requests= 500
##### The send buffer (SO_SNDBUF) used by the socket server
socket.send.buffer.bytes=104857600
#### The receive buffer (SO_RCVBUF) used by the socket server
socket.receive.buffer.bytes=104857600
#### The maximum size of a request that the socket server will accept (protection against OOM)
socket.request.max.bytes=104857600
max.connections.per.ip=1000
异常
Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:463)
at sun.nio.ch.Net.bind(Net.java:455)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:91)
... 17 more
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:338)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:188)
at com.baozun.demo.util.KafkaSendUtil.sendMessage(KafkaSendUtil.java:41)
at com.baozun.demo.producer.KafkaProducer$1.run(KafkaProducer.java:58)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.KafkaException: java.io.IOException: Unable to establish loopback connection
at org.apache.kafka.common.network.Selector.<init>(Selector.java:124)
at org.apache.kafka.common.network.Selector.<init>(Selector.java:145)
at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:303)
... 6 more
Caused by: java.io.IOException: Unable to establish loopback connection
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:125)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:69)
at java.security.AccessController.doPrivileged(Native Method)
at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:141)
at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:50)
at java.nio.channels.Pipe.open(Pipe.java:150)
at sun.nio.ch.WindowsSelectorImpl.<init>(WindowsSelectorImpl.java:127)
at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:44)
at java.nio.channels.Selector.open(Selector.java:227)
at org.apache.kafka.common.network.Selector.<init>(Selector.java:122)
... 8 more
Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:463)
at sun.nio.ch.Net.bind(Net.java:455)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:91)
... 17 more
你的怎么解决的,是在本地运行的吗
除kafka自身的配置外,如果你是linux系统,那就得设置limits.conf。
其次排查程序问题,是20个producer共用一个连接 还是20个producer有20个连接。
你好,能详细说下怎么排查程序问题,以及windows系统需要改哪些配置
你的答案