创建topic时报错: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.

我是谁 发表于: 2017-01-06   最后更新时间: 2017-01-06  
  •   8 订阅,1051 游览

说明:

1、kafka版本kafka_2.11-0.10.1.1
2、kafka共三个节点,其中broker1和broker2均报错,broker3未报错
3、broker1报错如下:
[2017-01-06 14:08:32,579] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,4] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
[2017-01-06 14:08:32,579] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,1] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
4、broker2报错如下:
[2017-01-06 14:08:33,579] ERROR [ReplicaFetcherThread-0-2], Error for partition [xdfjc1,2] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
5、创建的topic为3副本5个分区
6、经尝试验证:副本*分区数<12时不会报错
7、希望大牛指点一二







发表于: 4月前   最后更新时间: 4月前   游览量:1051
上一条: kafka broker数据丢失问题
下一条: 使用camus将kafka中的数据入hive时,总有几个task失败,导致一些表入不了hive
评论…

  • http://orchome.com/237 看下是否是因为broker id关系错乱了。
    你跟官网提供的错误基本一致。
    https://issues.apache.org/jira/browse/KAFKA-3114

    另外,副本*分区>12分区报错,我觉得是一个正确的行为。你大可不必纠结。
    例如,3个节点。3个分区 * 3个副本,这样3个broker节点上分别备份了全部的副本,如果4个副本,那么1个broker上就要有1个重复的副本,这样是完全不合理的。
    • 请问要如何查看是否冲突?
      这是报错情况:

      [2017-01-09 15:48:51,152] ERROR [ReplicaFetcherThread-0-2], Error for partition [xdfjc1,4] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
      [2017-01-09 15:48:51,152] ERROR [ReplicaFetcherThread-0-2], Error for partition [xdfjc1,1] to broker 2:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
      [2017-01-09 15:48:51,154] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,3] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)
      [2017-01-09 15:48:51,154] ERROR [ReplicaFetcherThread-0-1], Error for partition [xdfjc1,0] to broker 1:org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. (kafka.server.ReplicaFetcherThread)

      这是topic具体信息:

      [root@ws-node01 config]# kafka-topics.sh --describe --zookeeper ws-node01:2181,ws-node02:2181,ws-node03:2181/kafka --topic xdfjc1

      Topic:xdfjc1 PartitionCount:5 ReplicationFactor:3 Configs:
      Topic: xdfjc1 Partition: 0 Leader: 1 Replicas: 1,2,0 Isr: 1,2,0
      Topic: xdfjc1 Partition: 1 Leader: 2 Replicas: 2,0,1 Isr: 2,0,1
      Topic: xdfjc1 Partition: 2 Leader: 0 Replicas: 0,1,2 Isr: 0,1,2
      Topic: xdfjc1 Partition: 3 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2
      Topic: xdfjc1 Partition: 4 Leader: 2 Replicas: 2,1,0 Isr: 2,1,0
        • # 我尝试着创建副本大于broker数,则根本无法创建成功。
           bin/kafka-topics.sh --create --zookeeper 192.168.190.11:2282/kafka_logs --replication-factor 4 --partitions 4 --topic test2

          Error while executing topic command : replication factor: 4 larger than available brokers: 3
          [2017-01-09 16:41:43,660] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: replication factor: 4 larger than available brokers: 3
           (kafka.admin.TopicCommand$)

          我的版本:kafka_2.11-0.10.1.0
            • 简单的排查了解一下
              1、你搭建的集群是否能正常的生产和消费消息(最好是外部应用调集群)。
              2、hosts配置(配置了计算机名)
              3、集群在搭建的过程中,config/server.properties中broker.id=0。这个broker.id是否在启动之后发现没修改,改了之后又启动了并未清空数据。

              如果以上3步都是正常的,我建议你删除掉zookeeper/kafka中的缓存,重新启动kafka。
                • 根据你说的,1、2、3都已经试过了,但还是会报错。不过很奇怪的是,报错是偶尔的,我一共创建了五次topic,都是三分本*5分区,出现报错共2两次,有三次未发生报错。。。
                    我也遇到这种问题,但是不影响使用。报错也是只报一次
                  • 评论…
                    • in this conversation
                      提问