kafka消费者组重新平衡后,消费不到消息

秦时明月 发表于: 2019-09-03   最后更新时间: 2019-09-03 16:24:37   2,321 游览

kafka消费者组重新平衡后,消费不到消息

source, manage-status-resource, serverResource, schedulerPanner]
[2019-09-03 15:32:36,999] e34abde478ab4ed6ae531332728e381c INFO  Metadata(285) - Cluster ID: eqVSzz-bS3yNxm-_yI1nyw
[2019-09-03 15:32:36,999] e34abde478ab4ed6ae531332728e381c INFO  Metadata(285) - Cluster ID: eqVSzz-bS3yNxm-_yI1nyw
[2019-09-03 15:32:37,001] e34abde478ab4ed6ae531332728e381c INFO  AbstractCoordinator(654) - [Consumer clientId=consumer-2, groupId=dx_group2] Discovered group coordinator SHQZ-PS-FWLW-KFAKA01:8092 (id: 2147483647 rack: null)
[2019-09-03 15:32:37,001] e34abde478ab4ed6ae531332728e381c INFO  AbstractCoordinator(654) - [Consumer clientId=consumer-2, groupId=dx_group2] Discovered group coordinator SHQZ-PS-FWLW-KFAKA01:8092 (id: 2147483647 rack: null)
[2019-09-03 15:32:37,003] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Metadata(285) - Cluster ID: eqVSzz-bS3yNxm-_yI1nyw
[2019-09-03 15:32:37,003] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Metadata(285) - Cluster ID: eqVSzz-bS3yNxm-_yI1nyw
[2019-09-03 15:32:37,003] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  AbstractCoordinator(654) - [Consumer clientId=consumer-1, groupId=dx_group] Discovered group coordinator SHQZ-PS-FWLW-KFAKA02:8093 (id: 2147483646 rack: null)
[2019-09-03 15:32:37,003] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  AbstractCoordinator(654) - [Consumer clientId=consumer-1, groupId=dx_group] Discovered group coordinator SHQZ-PS-FWLW-KFAKA02:8093 (id: 2147483646 rack: null)
[2019-09-03 15:32:37,004] e34abde478ab4ed6ae531332728e381c INFO  ConsumerCoordinator(458) - [Consumer clientId=consumer-2, groupId=dx_group2] Revoking previously assigned partitions []
[2019-09-03 15:32:37,004] e34abde478ab4ed6ae531332728e381c INFO  ConsumerCoordinator(458) - [Consumer clientId=consumer-2, groupId=dx_group2] Revoking previously assigned partitions []
[2019-09-03 15:32:37,004] e34abde478ab4ed6ae531332728e381c INFO  AbstractCoordinator(486) - [Consumer clientId=consumer-2, groupId=dx_group2] (Re-)joining group
[2019-09-03 15:32:37,004] e34abde478ab4ed6ae531332728e381c INFO  AbstractCoordinator(486) - [Consumer clientId=consumer-2, groupId=dx_group2] (Re-)joining group
[2019-09-03 15:32:37,007] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  ConsumerCoordinator(458) - [Consumer clientId=consumer-1, groupId=dx_group] Revoking previously assigned partitions []
[2019-09-03 15:32:37,007] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  ConsumerCoordinator(458) - [Consumer clientId=consumer-1, groupId=dx_group] Revoking previously assigned partitions []
[2019-09-03 15:32:37,007] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  AbstractCoordinator(486) - [Consumer clientId=consumer-1, groupId=dx_group] (Re-)joining group
[2019-09-03 15:32:37,007] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  AbstractCoordinator(486) - [Consumer clientId=consumer-1, groupId=dx_group] (Re-)joining group
[2019-09-03 15:32:37,024] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  AbstractCoordinator(450) - [Consumer clientId=consumer-1, groupId=dx_group] Successfully joined group with generation 1
[2019-09-03 15:32:37,024] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  AbstractCoordinator(450) - [Consumer clientId=consumer-1, groupId=dx_group] Successfully joined group with generation 1
[2019-09-03 15:32:37,025] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  ConsumerCoordinator(289) - [Consumer clientId=consumer-1, groupId=dx_group] Setting newly assigned partitions [ypmjwarn-2, ypmjwarn-0, ypmjwarn-1]
[2019-09-03 15:32:37,025] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  ConsumerCoordinator(289) - [Consumer clientId=consumer-1, groupId=dx_group] Setting newly assigned partitions [ypmjwarn-2, ypmjwarn-0, ypmjwarn-1]
[2019-09-03 15:32:37,048] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Fetcher(583) - [Consumer clientId=consumer-1, groupId=dx_group] Resetting offset for partition ypmjwarn-2 to offset 46.
[2019-09-03 15:32:37,048] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Fetcher(583) - [Consumer clientId=consumer-1, groupId=dx_group] Resetting offset for partition ypmjwarn-2 to offset 46.
[2019-09-03 15:32:37,052] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Fetcher(583) - [Consumer clientId=consumer-1, groupId=dx_group] Resetting offset for partition ypmjwarn-1 to offset 40.
[2019-09-03 15:32:37,052] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Fetcher(583) - [Consumer clientId=consumer-1, groupId=dx_group] Resetting offset for partition ypmjwarn-1 to offset 40.
[2019-09-03 15:32:37,053] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Fetcher(583) - [Consumer clientId=consumer-1, groupId=dx_group] Resetting offset for partition ypmjwarn-0 to offset 46.
[2019-09-03 15:32:37,053] 59566d90f40b446fa0d6ea3eaa3ad3e3 INFO  Fetcher(583) - [Consumer clientId=consumer-1, groupId=dx_group] Resetting offset for partition ypmjwarn-0 to offset 46.
[2019-09-03 15:32:37,402] SmlServer.init DEBUG Router(85) - urlMapper:/kafka/data,methodName:data,parameter:[class java.lang.String]
[2019-09-03 15:32:37,402] SmlServer.init DEBUG Router(85) - urlMapper:/kafka/reg,methodName:reg,parameter:[class java.lang.String, class java.lang.String]
[2019-09-03 15:32:37,403] SmlServer.init DEBUG Router(85) - urlMapper:/kafka/start,methodName:start,parameter:[class java.lang.String]
[2019-09-03 15:32:37,403] SmlServer.init DEBUG Router(85) - urlMapper:/kafka/stop,methodName:stop,parameter:[class java.lang.String]
[2019-09-03 15:32:37,403] SmlServer.init DEBUG Router(85) - urlMapper:/resource,methodName:source,parameter:[]
[2019-09-03 15:32:37,404] SmlServer.init DEBUG Router(85) - urlMapper:/resource/export,methodName:exportFile,parameter:[class java.lang.String, class java.lang.String, interface org.hw.sml.server.NanoHTTPD$IHTTPSession]
[2019-09-03 15:32:37,404] SmlServer.init DEBUG Router(85) - urlMapper:/sml/cmd,methodName:evel,parameter:[class java.lang.String]
[2019-09-03 15:32:37,404] SmlServer.init DEBUG Router(85) - urlMapper:/sml/shell/([\w|\-|:]+),methodName:eval,parameter:[class java.lang.String, class java.lang.String, class java.lang.String]
[2019-09-03 15:32:37,404] SmlServer.init DEBUG Router(85) - urlMapper:/status/log,methodName:log,parameter:[class java.lang.String, class java.lang.String, int, int, class java.lang.String, boolean]
[2019-09-03 15:32:37,405] SmlServer.init DEBUG Router(85) - urlMapper:/status,methodName:status,parameter:[]
[2019-09-03 15:32:37,405] SmlServer.init DEBUG Router(85) - urlMapper:/server/sources,methodName:source,parameter:[]
[2019-09-03 15:32:37,405] SmlServer.init DEBUG Router(85) - urlMapper:/server/regist,methodName:regist,parameter:[interface java.util.Map, interface org.hw.sml.server.NanoHTTPD$IHTTPSession]
[2019-09-03 15:32:37,405] SmlServer.init DEBUG Router(85) - urlMapper:/server/proxy/(.*?),methodName:proxy,parameter:[interface org.hw.sml.server.NanoHTTPD$IHTTPSession]
[2019-09-03 15:32:37,405] SmlServer.init DEBUG Router(85) - urlMapper:/server/clear,methodName:remove,parameter:[class java.lang.String]
[2019-09-03 15:32:37,405] SmlServer.init DEBUG Router(85) - urlMapper:/server/clearAll,methodName:clear,parameter:[]
[2019-09-03 15:32:37,417] SmlServer.init INFO  SmlServer(44) - nanohttp server bind-->[8181],daemon:[false],timeout:[120000]
[2019-09-03 15:33:01,149] e34abde478ab4ed6ae531332728e381c INFO  AbstractCoordinator(450) - [Consumer clientId=consumer-2, groupId=dx_group2] Successfully joined group with generation 13
[2019-09-03 15:33:01,149] e34abde478ab4ed6ae531332728e381c INFO  AbstractCoordinator(450) - [Consumer clientId=consumer-2, groupId=dx_group2] Successfully joined group with generation 13
[2019-09-03 15:33:01,150] e34abde478ab4ed6ae531332728e381c INFO  ConsumerCoordinator(289) - [Consumer clientId=consumer-2, groupId=dx_group2] Setting newly assigned partitions [ypmjwarn-2]
[2019-09-03 15:33:01,150] e34abde478ab4ed6ae531332728e381c INFO  ConsumerCoordinator(289) - [Consumer clientId=consumer-2, groupId=dx_group2] Setting newly assigned partitions [ypmjwarn-2]
发表于 2019-09-03
添加评论

日志正常,
你重新启动消费者,可以消费到吗?

秦时明月 -> 半兽人 5年前

消费不到,这个数据放的比较少

秦时明月 -> 半兽人 5年前

这个日志是正常的么

秦时明月 -> 半兽人 5年前

我本地测试遇到过这个问题,过了一晚上自己好了,这是不是哪里可以设置的

半兽人 -> 秦时明月 5年前

你问题描述的信息太少了,不知道该怎么回答你。

你的答案

查看kafka相关的其他问题或提一个您自己的问题