at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
2019-10-0914:01:20,767 - [ERROR] k.m.j.KafkaJMX$ - Failed to connect to service:jmx:rmi:///jndi/rmi://ip:port/jmxrmi
java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
java.net.ConnectException: Connection refused (Connectionrefused)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at kafka.manager.jmx.KafkaJMX$.doWithConnection(KafkaJMX.scala:57)
at kafka.manager.actor.cluster.BrokerViewCacheActor.$anonfun$updateBrokerMetrics$3(BrokerViewCacheActor.scala:359)
Caused by: java.net.ConnectException: Connection refused (Connectionrefused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at java.net.Socket.connect(Socket.java:538)
at java.net.Socket.(Socket.java:434)
at java.net.Socket.(Socket.java:211)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
2019-10-0914:01:20,767 - [ERROR] k.m.a.c.BrokerViewCacheActor - Failed to get broker metrics for BrokerIdentity(1,ip,port,false,true,Map(PLAINTEXT -> 9292))java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is:
java.net.ConnectException: Connection refused (Connectionrefused)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430)
删除不会的,kafka是以offset为整个数据验证,控制的,体量非常小。
你可以看下
kafka-manager
监控的流量吞吐,可能是它引起的。我看了kafka-manager的日志,基本都是下面这些报错日志,没有看到别的日志:
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40) 2019-10-09 14:01:20,767 - [ERROR] k.m.j.KafkaJMX$ - Failed to connect to service:jmx:rmi:///jndi/rmi://ip:port/jmxrmi java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is: java.net.ConnectException: Connection refused (Connection refused) at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430) at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) at kafka.manager.jmx.KafkaJMX$.doWithConnection(KafkaJMX.scala:57) at kafka.manager.actor.cluster.BrokerViewCacheActor.$anonfun$updateBrokerMetrics$3(BrokerViewCacheActor.scala:359) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at java.net.Socket.connect(Socket.java:538) at java.net.Socket.(Socket.java:434) at java.net.Socket.(Socket.java:211) at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40) 2019-10-09 14:01:20,767 - [ERROR] k.m.a.c.BrokerViewCacheActor - Failed to get broker metrics for BrokerIdentity(1,ip,port,false,true,Map(PLAINTEXT -> 9292)) java.rmi.ConnectException: Connection refused to host: 127.0.1.1; nested exception is: java.net.ConnectException: Connection refused (Connection refused) at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:129) at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source) at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2430)
我的意思是一些监控占的带宽比较高,频繁的进行数据交互。
可以用iftop等其他网络工具监控下。
那只能实时用iftop来监控了,昨天出现的case也查不到什么线索了。
还有请教一下kafka-manager的这些metrics数据默认是上报在哪里的,想去查看一下log日志。
Metrics Rate Mean 1 min 5 min 15 min Messages in /sec 0.00 0.00 0.00 0.00 Bytes in /sec 0.00 0.00 0.00 0.00 Bytes out /sec 0.00 0.00 0.00 0.00 Bytes rejected /sec 0.00 0.00 0.00 0.00 Failed fetch request /sec 0.00 0.00 0.00 0.00 Failed produce request /sec 0.00 0.00 0.00 0.00
kafka-manager不清楚了,没有深入过了。
好的 多谢大佬!
一个topic 3个分区 3个备份,如果生产很大的数据量到kafka集群,副本数据同步会不会引起流量迅猛增长?例如一次性生产10G的数据量
当然,副本3个,那就是30G。
假设3个分区的leader平均分配到3个broker上,每个leader所在的broker平均将会收到3G的数据量,然后同步给其它两个副本所在broker,需要同步6G的数据量, 则一段时间内一个broker的出口流量总和是6G。 可以这样理解?
是的 数据要同步给其他副本
多谢!
你的答案