使用camus将kafka中的数据入hive时,总有几个task失败,导致一些表入不了hive

漂泊的美好 发表于: 2017-01-07   最后更新时间: 2017-01-07  
  •   8 订阅,511 游览

screenshot
上图显示一个camus任务123个task成功,12个失败,18个被Kill


task失败错误信息图片
screenshot
WARN [main] org.apache.kafka.clients.NetworkClient:Error while fetching metadata with correlation id 12213: [hive-dbname-cdrtable=LEADER_NOT_AVAILABLE
WARN [main] org.apache.kafka.clients.NetworkClient:Error while fetching metadata with correlation id 12215: [hive-dbname-cdrtable=LEADER_NOT_AVAILABLE]
WARN [main] org.apache.kafka.clients.NetworkClient:Error while fetching metadata with correlation id 12217: [hive-dbname-cdrtable=LEADER_NOT_AVAILABLE]
WARN [main] org.apache.kafka.clients.NetworkClient:Error while fetching metadata with correlation id 12219: [hive-dbname-cdrtable=LEADER_NOT_AVAILABLE]
WARN [main] org.apache.kafka.clients.NetworkClient:Error while fetching metadata with correlation id 12221: [hive-dbname-cdrtable=LEADER_NOT_AVAILABLE]
WARN [main] org.apache.kafka.clients.NetworkClient:Error while fetching metadata with correlation id 12222: [hive-dbname-cdrtable=LEADER_NOT_AVAILABLE]


求大神帮帮忙







发表于: 4月前   最后更新时间: 4月前   游览量:511
上一条: 创建topic时报错: org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.
下一条: kafka适配kerberos后启动失败
评论…

  • 有可能是zookeeper版本低于3.4.8,升级下zookeeper。
    • 目前我只遇到过一次,分享给你了,kafka新功能遇到的(如streams),有个很严重的bug现象,我在本地做测试,低版本的zookeeper第一次是ok的,但是当我重启应用就开始刷这种错,我还没有时间看源码。
      • 评论…
        • in this conversation
          提问