1、kafka版本kafka_2.11-0.10.1.1
2、kafka共三个节点,其中broker1和broker2均报错,broker3未报错
3、broker1报错如下:
[2017-01-06 14:08:32,579] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,4] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
[2017-01-06 14:08:32,579] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,1] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
4、broker2报错如下:
[2017-01-06 14:08:33,579] ERROR [ReplicaFetcherThread-0-2], Error for partition [xdfjc1,2] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
5、创建的topic为3副本5个分区
6、经尝试验证:副本*分区数<12
时不会报错
7、希望大牛指点一二
我也遇到这种问题,但是不影响使用。报错也是只报一次
https://www.orchome.com/237 看下是否是因为broker id关系错乱了。
你跟官网提供的错误基本一致。
https://issues.apache.org/jira/browse/KAFKA-3114
另外,
副本分区>12分区
报错,我觉得是一个正确的行为。你大可不必纠结。例如,3个节点。3个分区 3个副本,这样3个broker节点上分别备份了全部的副本,如果4个副本,那么1个broker上就要有1个重复的副本,这样是完全不合理的。
谢谢你的回答。不过当
4个分区*3个副本
时也是报错的4分区*3副本还错,这就不正常了,之前broker id是否配置冲突过?
请问要如何查看是否冲突?
这是报错情况:
[2017-01-09 15:48:51,152] ERROR [ReplicaFetcherThread-0-2], Error for partition [xdfjc1,4] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread) [2017-01-09 15:48:51,152] ERROR [ReplicaFetcherThread-0-2], Error for partition [xdfjc1,1] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread) [2017-01-09 15:48:51,154] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,3] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread) [2017-01-09 15:48:51,154] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,0] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
这是topic具体信息:
[root@ws-node01 config]# kafka-topics.sh --describe --zookeeper ws-node01:2181,ws-node02:2181,ws-node03:2181/kafka --topic xdfjc1 Topic:xdfjc1 PartitionCount:5 ReplicationFactor:3 Configs: Topic: xdfjc1 Partition: 0 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0 Topic: xdfjc1 Partition: 1 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1 Topic: xdfjc1 Partition: 2 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2 Topic: xdfjc1 Partition: 3 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2 Topic: xdfjc1 Partition: 4 Leader: 2 Replicas: 2,1,0 Isr: 2,1,0
我尝试着创建副本大于broker数,则根本无法创建成功。
bin/kafka-topics.sh --create --zookeeper 192.168.190.11:2282/kafka_logs --replication-factor 4 --partitions 4 --topic test2 Error while executing topic command : replication factor: 4 larger than available brokers: 3 [2017-01-09 16:41:43,660] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 4 larger than available brokers: 3 (kafka.admin.TopicCommand$)
我的版本:kafka_2.11-0.10.1.0
简单的排查了解一下
1、你搭建的集群是否能正常的生产和消费消息(最好是外部应用调集群)。
2、hosts配置(配置了计算机名)
3、集群在搭建的过程中,config/server.properties中broker.id=0。这个broker.id是否在启动之后发现没修改,改了之后又启动了并未清空数据。
如果以上3步都是正常的,我建议你删除掉zookeeper/kafka中的缓存,重新启动kafka。
根据你说的,1、2、3都已经试过了,但还是会报错。不过很奇怪的是,报错是偶尔的,我一共创建了五次topic,都是三分本*5分区,出现报错共2两次,有三次未发生报错。。。
把所有数据清空了,重新来吧。
你的答案