求救：java 连接 kafka kerberos 问题

按照博主大大的笔记，安装了kerbros认证，但是用java连接kafka时，报错：

Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner  authentication information from the user

详细日志：

Exception in thread "main" org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:717)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:597)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:579)
    at kafka.test.KbConsumer.main(KbConsumer.java:30)
Caused by: org.apache.kafka.common.KafkaException: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner  authentication information from the user
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:94)
    at org.apache.kafka.common.network.ChannelBuilders.create(ChannelBuilders.java:93)
    at org.apache.kafka.common.network.ChannelBuilders.clientChannelBuilder(ChannelBuilders.java:51)
    at org.apache.kafka.clients.ClientUtils.createChannelBuilder(ClientUtils.java:84)
    at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:657)
    ... 3 more
Caused by: javax.security.auth.login.LoginException: Could not login: the client is being asked for a password, but the Kafka client code does not currently support obtaining a password from the user. not available to garner  authentication information from the user
    at com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:940)
    at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:760)
    at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
    at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
    at org.apache.kafka.common.security.authenticator.AbstractLogin.login(AbstractLogin.java:58)
    at org.apache.kafka.common.security.kerberos.KerberosLogin.login(KerberosLogin.java:109)
    at org.apache.kafka.common.security.authenticator.LoginManager.<init>(LoginManager.java:55)
    at org.apache.kafka.common.security.authenticator.LoginManager.acquireLoginManager(LoginManager.java:89)
    at org.apache.kafka.common.network.SaslChannelBuilder.configure(SaslChannelBuilder.java:86)
    ... 7 more

代码以及配置：
Consumer.java

 public class KbConsumer {
    public static void main(String[] args) {
        System.setProperty("java.security.krb5.conf",System.getProperty("user.dir") + "\\krb5.conf");
        System.setProperty("java.security.auth.login.config", System.getProperty("user.dir") + "\\kafka_client_jaas.conf");
        Properties props = new Properties();
        props.put(BOOTSTRAP_SERVERS_CONFIG, "10.1.2.46:1234");
        props.put(ENABLE_AUTO_COMMIT_CONFIG, "true");
        props.put(GROUP_ID_CONFIG, "test_consumer_group");
        props.put(AUTO_COMMIT_INTERVAL_MS_CONFIG, 1000);
        props.put(AUTO_OFFSET_RESET_CONFIG, "earliest");
        props.put("sasl.kerberos.service.name", "kafka");
        props.put(KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        props.put(VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        props.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, "SASL_PLAINTEXT");

        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
        consumer.subscribe(Collections.singleton("wwxx"));
        while (true) {
            ConsumerRecords<String, String> records = consumer.poll(100);
            for (ConsumerRecord<String, String> record : records)
                System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());

        }
    }
}

kafka_client_jaas.conf

KafkaClient {
        com.sun.security.auth.module.Krb5LoginModule required
        useKeyTab=true
        storeKey=true
        useTicketCache=true
        keyTab="/etc/security/keytabs/kafka.keytab"
        principal="clients@EX.COM";
};

krb5.conf

[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = EX.COM
 dns_lookup_realm = false
 dns_lookup_kdc = true
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 EX.COM = {
  kdc = 10.1.2.46
  admin_server = 10.1.2.46
 }

[domain_realm]
kafka = EX.COM
host = EX.COM
zookeeper = EX.COM
127.0.0.1 = EX.COM
10.1.2.46 = EX.COM
bd005 = EX.COM

kafka.keytab

4 03/13/18 14:27:39 zookeeper/10.1.2.46@EX.COM (aes128-cts-hmac-sha1-96) 
   4 03/13/18 14:27:39 zookeeper/10.1.2.46@EX.COM (des-hmac-sha1) 
   4 03/13/18 14:27:39 zookeeper/10.1.2.46@EX.COM (arcfour-hmac) 
   4 03/13/18 14:27:39 zookeeper/10.1.2.46@EX.COM (des-cbc-md5) 
   4 03/13/18 14:27:39 zookeeper/10.1.2.46@EX.COM (des3-cbc-sha1) 
   2 03/13/18 14:28:25 kafka/10.1.2.46@EX.COM (aes128-cts-hmac-sha1-96) 
   2 03/13/18 14:28:25 kafka/10.1.2.46@EX.COM (des3-cbc-sha1) 
   2 03/13/18 14:28:25 kafka/10.1.2.46@EX.COM (arcfour-hmac) 
   2 03/13/18 14:28:25 kafka/10.1.2.46@EX.COM (des-hmac-sha1) 
   2 03/13/18 14:28:25 kafka/10.1.2.46@EX.COM (des-cbc-md5) 
   4 03/13/18 14:27:39 zookeeper/10.1.2.46@EX.COM (aes256-cts-hmac-sha1-96) 
   2 03/13/18 10:26:01 kafka/127.0.0.1@EX.COM (aes256-cts-hmac-sha1-96) 
   2 03/13/18 10:26:01 kafka/127.0.0.1@EX.COM (aes128-cts-hmac-sha1-96) 
   2 03/13/18 10:26:01 kafka/127.0.0.1@EX.COM (des3-cbc-sha1) 
   2 03/13/18 10:26:01 kafka/127.0.0.1@EX.COM (arcfour-hmac) 
   2 03/13/18 10:26:01 kafka/127.0.0.1@EX.COM (des-hmac-sha1) 
   2 03/13/18 10:26:01 kafka/127.0.0.1@EX.COM (des-cbc-md5) 
   2 03/13/18 10:26:34 zookeeper/127.0.0.1@EX.COM (aes256-cts-hmac-sha1-96) 
   2 03/13/18 10:26:34 zookeeper/127.0.0.1@EX.COM (aes128-cts-hmac-sha1-96) 
   2 03/13/18 10:26:34 zookeeper/127.0.0.1@EX.COM (des3-cbc-sha1) 
   2 03/13/18 10:26:34 zookeeper/127.0.0.1@EX.COM (arcfour-hmac) 
   2 03/13/18 10:26:34 zookeeper/127.0.0.1@EX.COM (des-hmac-sha1) 
   2 03/13/18 10:26:34 zookeeper/127.0.0.1@EX.COM (des-cbc-md5) 
   2 03/13/18 10:28:19 kafka/bd005@EX.COM (aes256-cts-hmac-sha1-96) 
   2 03/13/18 10:28:19 kafka/bd005@EX.COM (aes128-cts-hmac-sha1-96) 
   2 03/13/18 10:28:19 kafka/bd005@EX.COM (des3-cbc-sha1) 
   2 03/13/18 10:28:19 kafka/bd005@EX.COM (arcfour-hmac) 
   2 03/13/18 10:28:19 kafka/bd005@EX.COM (des-hmac-sha1) 
   2 03/13/18 10:28:19 kafka/bd005@EX.COM (des-cbc-md5) 
   2 03/13/18 10:31:45 clients@EX.COM (aes256-cts-hmac-sha1-96) 
   2 03/13/18 10:31:45 clients@EX.COM (aes128-cts-hmac-sha1-96) 
   2 03/13/18 10:31:45 clients@EX.COM (des3-cbc-sha1) 
   2 03/13/18 10:31:45 clients@EX.COM (arcfour-hmac) 
   2 03/13/18 10:31:45 clients@EX.COM (des-hmac-sha1) 
   2 03/13/18 10:31:45 clients@EX.COM (des-cbc-md5) 
   2 03/13/18 14:28:25 kafka/10.1.2.46@EX.COM (aes256-cts-hmac-sha1-96)

principal

   K/M@EX.COM
admin/admin@EX.COM
clients@EX.COM
host/10.1.2.46@EX.COM
host/bd005@EX.COM
kadmin/admin@EX.COM
kadmin/bd005@EX.COM
kadmin/changepw@EX.COM
kafka/10.1.2.46@EX.COM
kafka/127.0.0.1@EX.COM
kafka/bd005@EX.COM
krbtgt/EX.COM@EX.COM
test/10.1.2.46@EX.COM
test/bd005@EX.COM
zookeeper/10.1.2.46@EX.COM
zookeeper/127.0.0.1@EX.COM

kafka可以正常启动，也可以自主生产消费。consumer的代码可能有误，新人刚接触kafka，还望高人指点。
此外还有一个问题，对topic添加指定权限后，在其他服务器上也无法对topic进行读写。
使用的命令：

bin/kafka-acls.sh --authorizer-properties zookeeper.connect={zookeeper-host} --add --allow-principal User:* --allow-host *   --operation all --topic wwxx

我也是丢到服务器上会有这个问题，服务器的路径不起作用。在本地跑，用本地路径就没事儿。你解决了服务器上的问题了吗？？

回答于 5年前

有→_→点累

这个错误是说kafka客户端不支持获取认证信息，把认证加到jvm环境中。

回答于 6年前

無名

落樱留独殇 -> 無名 6年前

-Djava.security.krb5.conf=D:/iSpace/kafka/krb5.conf
-Djava.security.auth.login.config=D:/iSpace/kafka/kafka_client_jaas.conf

你是指这两个吗？加了，还是不行

無名 -> 落樱留独殇 6年前

报什么错，跟之前的一样吗？

嗯，现在解决了，问题出在jaas.conf文件的 keyTab="D:/iSpace/kafka/kafka.keytab"，这个我之前指定的路径是服务器上的，改成本地就可以了。不过还有一个问题，就是我新创建了一个topic，没有分配权限，但是可以生产和消费……

你用的是嵌入到jvm的方式吗？

--allow-host * ？

我把服务上的认证文件拷到本地，添加到IDEA的jvm里了。

最后那个分配权限的命令是根据之前做SASL_PLAIN认证写的，意思是想允许来自任何IP的任何用户都可以对topic进行读写。但是，我新创建一个了topic，不分配任何权限也可以进行读写，不知道是不是SASL没生效的问题……

你最后的命令，都是允许吧。

是的，刚才又跑了遍代码，又出现那个问题了，上午的时候把
-Djava.security.krb5.conf=D:/iSpace/kafka/krb5.conf
-Djava.security.auth.login.config=D:/iSpace/kafka/kafka_client_jaas.conf
添加到 jvm 里，把 jaas.conf 的 keyTab="D:/iSpace/kafka/kafka.keytab"也改成本地的了，确实跑通了啊，见鬼了……

Σ( ° △ °|||)︴

貌似没法对topic进行权限细分……不知道哪里的问题，唉…

https://www.orchome.com/185

我在server.properties中添加了

authorizer.class.name = kafka.security.auth.SimpleAclAuthorizer
super.users=User:kafka/bd005@EX.COM

然后给 pruducer 和 consumer 分配权限，这是分配后的信息

Current ACLs for resource `Topic:wwxx1`: 
  User:clients has Allow permission for operations: Read from hosts: 10.1.2.46
 User:clients@EX.COM has Allow permission for operations: Write from hosts: 10.1.2.46
 User:clients@EX.COM has Allow permission for operations: Read from hosts: 10.1.2.46
 User:clients has Allow permission for operations: Write from hosts: 10.1.2.46 

Current ACLs for resource `Group:news-consumer-group`: 
  User:clients has Allow permission for operations: Read from hosts: 10.1.2.46
 User:clients@EX.COM has Allow permission for operations: Read from hosts: 10.1.2.46

通过命令生产和消费，分别出现了:

生产：

WARN Bootstrap broker 10.1.2.46:9988 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

消费：

WARN Error while fetching metadata with correlation id 3 : {wwxx1=TOPIC_AUTHORIZATION_FAILED} (org.apache.kafka.clients.NetworkClient)

能看出来是哪个地方出现了问题吗？

认证失败？这块你不是已经测通过了吗

kerberos用户认证通了，但是ACL有问题，给topic分配权限后，使用命令测试就出现上面的错，客户端也是，可以连上kafka但是发送/消费不了。https://www.orchome.com/500 我是按照这个教程做的，也看了 https://www.orchome.com/378 这个问题，但是没理解……

你的答案

查看kafka相关的其他问题或提一个您自己的问题。

求救：java 连接 kafka kerberos 问题

你的答案

昵称