情况是这样,在我的环境下,我需要用到nginx代理一部分端口到内网。具体的情况如下图所示
每台kafka会启动3个端口,一个为19090(为内网不加密端口,纯内网使用),一个为19091(未加密,供外网访问),一个为19092(加密,供外网访问)。
采用内外网分流,外网访问为SSL加密模式和PLAINTEXT两种,内网为PLAINTEXT未加密模式,即19091端口【外网】、19090【内网】为未加密端口,19092端口【外网】为加密端口。
外网访问未加密端口 114.251.155.29:9092,114.251.155.29:9093,114.251.155.29:9094 分别会被代理到内网的192.168.1.10:19091,192.168.1.11:19091,192.168.1.12:19091端口。
外网访问加密端口114.251.155.29:9095,114.251.155.29:9096,114.251.155.29:9097分别会被代理到内网的192.168.1.10:19092,192.168.1.11:19092,192.168.1.12:19092端口。
内网访问192.168.1.10:19090,192.168.1.11:19090,192.168.1.12:19090
kafka配置如下
broker.id=1
delete.topic.enable=true
listeners=INTERNAL://192.168.1.10:19090,OUTERPLAINTEXT://192.168.1.10:19091,OUTERSSL://192.168.1.10:19092
advertised.listeners=INTERNAL://192.168.1.10:19090,OUTERPLAINTEXT://公网IP:9092,OUTERSSL://公网IP:9095
listener.security.protocol.map=INTERNAL:PLAINTEXT,OUTERPLAINTEXT:PLAINTEXT,OUTERSSL:SSL
inter.broker.listener.name=INTERNAL
ssl.keystore.location=/usr/local/ca/certificates/kafka.keystore
ssl.keystore.password=Pass2020
ssl.key.password=Pass2020
ssl.truststore.location=/usr/local/ca/certificates/kafka.truststore
ssl.truststore.password=Pass2020
ssl.endpoint.identification.algorithm=
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.keystore.type=JKS
ssl.truststore.type=JKS
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/mnt/disk01/kafka/logs
num.partitions=3
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=3
transaction.state.log.replication.factor=3
transaction.state.log.min.isr=2
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0
nginx 主要配置如下
upstream ICsn01PLAINTEXT {
server 192.168.1.10:19091 weight=1;
}
upstream ICsn02PLAINTEXT {
server 192.168.1.11:19091 weight=1;
}
upstream ICsn03PLAINTEXT {
server 192.168.1.12:19091 weight=1;
}
upstream ICsn01SSL {
server 192.168.1.10:19092 weight=1;
}
upstream ICsn02SSL {
server 192.168.1.11:19092 weight=1;
}
upstream ICsn03SSL {
server 192.168.1.12:19092 weight=1;
}
server {
listen 9092;
proxy_pass ICsn01PLAINTEXT;
}
server {
listen 9093;
proxy_pass ICsn02PLAINTEXT;
}
server {
listen 9094;
proxy_pass ICsn03PLAINTEXT;
}
server {
listen 9095;
proxy_pass ICsn01SSL;
}
server {
listen 9096;
proxy_pass ICsn02SSL;
}
server {
listen 9097;
proxy_pass ICsn02SSL;
}
通过ssl加密端口连接发送数据时报错为如下【这里是指定往2号分区发送数据】
Error while producing message to topic :test_topic-2@-1
org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition.
指定往1号分区发送数据又不报错,请问我这是哪里出问题了,着急啊!!!!!
你的答案