Kafka connect 错误

电杆GG 发表于: 2018-09-25   最后更新时间: 2018-09-25 12:43:41   4,492 游览

使用命令:

./connect-standalone.sh ../config/connect-standalone.properties ../config/connect-file-source.properties ../config/connect-file-sink.properties 启动之后,

一直刷:

[2018-09-25 11:30:20,523] INFO [Consumer clientId=consumer-1, groupId=connect-local-file-sink] (Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:509)
[2018-09-25 11:30:20,553] INFO [Consumer clientId=consumer-1, groupId=connect-local-file-sink] Successfully joined group with generation 1 (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:473)
[2018-09-25 11:30:20,554] INFO [Consumer clientId=consumer-1, groupId=connect-local-file-sink] Setting newly assigned partitions [testconnec-0] (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator:280)
[2018-09-25 11:30:20,572] INFO [Consumer clientId=consumer-1, groupId=connect-local-file-sink] Resetting offset for partition testconnec-0 to offset 0. (org.apache.kafka.clients.consumer.internals.Fetcher:583)
[2018-09-25 11:30:30,261] INFO WorkerSourceTask{id=local-file-source-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:397)
[2018-09-25 11:30:30,262] INFO WorkerSourceTask{id=local-file-source-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:414)
[2018-09-25 11:30:40,264] INFO WorkerSourceTask{id=local-file-source-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:397)
[2018-09-25 11:30:40,267] INFO WorkerSourceTask{id=local-file-source-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:414)
[2018-09-25 11:30:50,268] INFO WorkerSourceTask{id=local-file-source-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:397)
[2018-09-25 11:30:50,268] INFO WorkerSourceTask{id=local-file-source-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:414)
[2018-09-25 11:31:00,271] INFO WorkerSourceTask{id=local-file-source-0} Committing offsets (org.apache.kafka.connect.runtime.WorkerSourceTask:397)
[2018-09-25 11:31:00,273] INFO WorkerSourceTask{id=local-file-source-0} flushing 0 outstanding messages for offset commit (org.apache.kafka.connect.runtime.WorkerSourceTask:414)

在添加改变时也没有改变

root@localhost data]# echo "sfv,bvhgccv" >> test.txt 
[root@localhost data]# echo "sllllllcv" >> test.txt。

无法写入test.sink.txt 中

[root@localhost data]# cat test.sink.txt

消费消息中也不能显示:

[root@localhost bin]# ./kafka-console-consumer.sh --bootstrap-server 192.168.16.143:19092 --topic testconnec --from-beginning 
^CProcessed a total of 0 messages
发表于 2018-09-25
添加评论

请问这个问题怎么解决了

配置信息如下:
connect-standalone.properties  >>
bootstrap.servers=192.168.16.129:19092,192.168.16.142:19092,192.168.16.143:19092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000

connect-file-source.properties >>
name=local-file-source
connector.class=FileStreamSource
tasks.max=1
file=/usr/local/kafka/data/test.txt
topic=testconnec


connect-file-sink.properties  >>
name=local-file-sink
connector.class=FileStreamSink
tasks.max=1
file=/usr/local/kafka/data/test.sink.txt
topics=testconnec

半兽人 -> 电杆GG 6年前

你的配置没有问题。
1、检查你的文件权限
2、保证你的连接器测试的机器能正常的消费和生产(为测试网络正常)。

你的答案

查看kafka相关的其他问题或提一个您自己的问题