2020年4月15号，Kafka 2.5.0发布。
以下是Kafka 2.5.0版本中解决JIRA问题的摘要，有关该版本的完整文档，入门指南以及关于该项目的信息，请参考Kafka官方文档，或kafka中文教程。
有关升级的注意事项：在升级你的集群之前，请仔细阅读此版本的升级文档。升级有关不兼容性和破坏性的变更，性能变化以及可能影响Kakfa生产的任何其他变化。
Kafka 2.5.0包含许多重要的新功能，以下是一些重要变更摘要：
- 支持 TLS 1.3（目前默认 1.2 版本）
- Kafka Streams引入Co-groups
- Kafka Consumer增量平衡机制（Incremental rebalance）
- 新指标可提供更好的运营监控
- 升级Zookeeper为3.5.7
- 不再支持Scala 2.11
新功能
[KAFKA-6049] - Kafka Streams: 在DSL中增加Cogroup
[KAFKA-6144] - 允许从同步备用服务器提供交互式查询
[KAFKA-7251] - 支持 TLS 1.3
[KAFKA-8843] - Zookeeper迁移工具对TLS的支持
[KAFKA-9352] - topic分区对任务的不均衡分配
[KAFKA-9445] - 允许从单个分区中获取密钥，而不是遍历实例上的所有存储
[KAFKA-9487] - 跟进：KAFKA-9445（允许从单个分区获取密钥）； 解决代码审查意见
改进
[KAFKA-6607] - Kafka Streams lag not zero when input topic transactional
[KAFKA-6614] - kafka-streams to configure internal topics message.timestamp.type=CreateTime
[KAFKA-6819] - Refactor build-in StreamsMetrics internal implementations
[KAFKA-7052] - ExtractField SMT throws NPE - needs clearer error message
[KAFKA-7317] - Use collections subscription for main consumer to reduce metadata
[KAFKA-7538] - Improve locking model used to update ISRs and HW
[KAFKA-7639] - Read one request at a time from socket to reduce broker memory usage
[KAFKA-7658] - Add KStream#toTable to the Streams DSL
[KAFKA-7689] - Add Commit/List Offsets Operations to AdminClient
[KAFKA-8179] - Incremental Rebalance Protocol for Kafka Consumer
[KAFKA-8455] - Add VoidSerde to Serdes
[KAFKA-8482] - alterReplicaLogDirs should be better documented
[KAFKA-8503] - Implement default.api.timeout.ms for AdminClient
[KAFKA-8507] - Support --bootstrap-server in all command line tools
[KAFKA-8710] - InitProducerId changes for KIP-360
[KAFKA-8805] - Bump producer epoch following recoverable errors
[KAFKA-8821] - Avoid pattern subscription to allow for stricter ACL settings
[KAFKA-8834] - Distinguish URPs caused by reassignment plus other metrics
[KAFKA-8855] - Collect and Expose Client's Name and Version in the Brokers
[KAFKA-8904] - Reduce metadata lookups when producing to a large number of topics
[KAFKA-8953] - Consider renaming `UsePreviousTimeOnInvalidTimestamp` timestamp extractor
[KAFKA-8960] - Move Task determineCommitId in gradle.build to Project Level
[KAFKA-8964] - Refactor Stream-Thread-level Metrics
[KAFKA-8968] - Refactor Task-level Metrics
[KAFKA-8980] - Refactor State-Store-level Metrics
[KAFKA-8991] - Enable scalac optimizer
[KAFKA-9011] - Add KStream#flatTransform and KStream#flatTransformValues to Scala API
[KAFKA-9016] - Warn when log dir stopped serving replicas
[KAFKA-9039] - Optimize replica fetching CPU utilization with large number of partitions
[KAFKA-9040] - Implement --all option for describing configs
[KAFKA-9086] - Refactor Processor Node Streams Metrics
[KAFKA-9091] - KIP-538: Add a metric tracking the number of open connections with a given SSL cipher type
[KAFKA-9098] - Name Repartition Filter, Source, and Sink Processors
[KAFKA-9102] - Increase default zk session timeout and max lag
[KAFKA-9106] - metrics exposed via JMX shoud be configurable
[KAFKA-9110] - Improve efficiency of disk reads when TLS is enabled
[KAFKA-9115] - Add Roll-up Throughput Sensor for Processing on Source Nodes
[KAFKA-9152] - Improve Sensor Retrieval
[KAFKA-9183] - SaslSslAdminClientIntegrationTest takes too long to execute
[KAFKA-9202] - serde in ConsoleConsumer with access to headers
[KAFKA-9226] - Section on deletion of segment files is out of date
[KAFKA-9277] - move all group state transition rules into their states
[KAFKA-9316] - ConsoleProducer help info not expose default properties
[KAFKA-9324] - Drop support for Scala 2.11 (KIP-531)
[KAFKA-9337] - Simplifying standalone mm2-connect config
[KAFKA-9360] - emitting checkpoint and heartbeat set to false will not disable the activity in their SourceTask
[KAFKA-9375] - Add thread names to kafka connect threads
[KAFKA-9384] - Loop improvements
[KAFKA-9405] - Use Map API computeIfAbsent Where Applicable
[KAFKA-9408] - Use StandardCharsets UTF-8 instead of UTF-8 Name
[KAFKA-9420] - Bump APIs to enable flexible versions
[KAFKA-9422] - Track the set of topics a connector is using
[KAFKA-9426] - OffsetsForLeaderEpochClient Use Switch Statement
[KAFKA-9437] - KIP-559: Make the Kafka Protocol Friendlier with L7 Proxies
[KAFKA-9460] - Enable TLSv1.2 by default and disable all others protocol versions
[KAFKA-9474] - Kafka RPC protocol should support type 'double'
[KAFKA-9477] - Doc: Add RoundRobinAssignor as an option to consumer configs
[KAFKA-9483] - Add Scala KStream#toTable to the Streams DSL
[KAFKA-9499] - Improve deletion process by batching more aggressively
[KAFKA-9515] - Upgrade ZooKeeper to 3.5.7
[KAFKA-9567] - Docs and system tests for ZooKeeper 3.5.7 and KIP-515
[KAFKA-9575] - "Notable changes in 2.5.0" doesn't mention ZooKeeper 3.5.7
[KAFKA-9586] - Fix errored json filename in ops documentation
[KAFKA-9758] - Add documentations for KIP-523 and KIP-527
[KAFKA-9810] - Document Connect Root REST API on /
Bug
[KAFKA-3061] - Get rid of Guava dependency
[KAFKA-4203] - Java producer default max message size does not align with broker default
[KAFKA-5868] - Kafka Consumer Rebalancing takes too long
[KAFKA-6266] - Kafka 1.0.0 : Repeated occurrence of WARN Resetting first dirty offset of __consumer_offsets-xx to log start offset 203569 since the checkpointed offset 120955 is invalid. (kafka.log.LogCleanerManager$)
[KAFKA-7925] - Constant 100% cpu usage by all kafka brokers
[KAFKA-8025] - Flaky Test RocksDBGenericOptionsToDbOptionsColumnFamilyOptionsAdapterTest#shouldForwardAllDbOptionsCalls
[KAFKA-8030] - Flaky Test TopicCommandWithAdminClientTest#testDescribeUnderMinIsrPartitionsMixed
[KAFKA-8081] - Flaky Test TopicCommandWithAdminClientTest#testDescribeUnderMinIsrPartitions
[KAFKA-8162] - IBM JDK Class not found error when handling SASL authentication exception
[KAFKA-8211] - Flaky Test: ResetConsumerGroupOffsetTest.testResetOffsetsExportImportPlan
[KAFKA-8245] - Flaky Test DeleteConsumerGroupsTest#testDeleteCmdAllGroups
[KAFKA-8340] - ServiceLoader fails when used from isolated plugin path directory
[KAFKA-8700] - Flaky Test QueryableStateIntegrationTest#queryOnRebalance
[KAFKA-8705] - NullPointerException was thrown by topology optimization when two MergeNodes have common KeyChaingingNode
[KAFKA-8764] - LogCleanerManager endless loop while compacting/cleaning segments
[KAFKA-8786] - Deprecated Gradle features making it incompatible with Gradle 6.0.
[KAFKA-8819] - Plugin path for converters not working as intended
[KAFKA-8842] - Reading/Writing confused in Connect QuickStart Guide
[KAFKA-8928] - Logged producer config does not always match actual config values
[KAFKA-8940] - Flaky Test SmokeTestDriverIntegrationTest.shouldWorkWithRebalance
[KAFKA-8945] - Incorrect null check in the constructor for ConnectorHealth and AbstractState
[KAFKA-9002] - Flaky Test org.apache.kafka.streams.integration.RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenCreated
[KAFKA-9014] - AssertionError thrown by SourceRecordWriteCounter when SourceTask.poll returns an empty list
[KAFKA-9024] - org.apache.kafka.connect.transforms.ValueToKey throws NPE
[KAFKA-9025] - ZkSecurityMigrator not working with zookeeper chroot
[KAFKA-9051] - Source task source offset reads can block graceful shutdown
[KAFKA-9056] - Selector outbound byte metric does not count partial sends
[KAFKA-9068] - Fix incorrect JavaDocs for `Stores.xxxSessionStore(...)`
[KAFKA-9074] - Connect's Values class does not parse time or timestamp values from string literals
[KAFKA-9077] - System Test Failure: StreamsSimpleBenchmarkTest
[KAFKA-9083] - Various parsing issues in Values class
[KAFKA-9131] - failed producer metadata updates result in the unrelated error message
[KAFKA-9143] - DistributedHerder misleadingly log error on connector task reconfiguration
[KAFKA-9159] - Consumer.endOffsets Throw TimeoutException: Failed to get offsets by times in 30000ms after a leader change
[KAFKA-9169] - Standby Tasks point ask for incorrect offsets on resuming post suspension
[KAFKA-9184] - Redundant task creation and periodic rebalances after zombie worker rejoins the group
[KAFKA-9192] - NullPointerException if field in schema not present in value
[KAFKA-9204] - ReplaceField transformation fails when encountering tombstone event
[KAFKA-9218] - MirrorMaker 2 can fail to create topics
[KAFKA-9219] - NullPointerException when polling metrics from Kafka Connect
[KAFKA-9233] - Kafka consumer throws undocumented IllegalStateException
[KAFKA-9241] - SASL Clients are not forced to re-authenticate if they don't leverage SaslAuthenticateRequest
[KAFKA-9255] - MessageSet v1 protocol wrong specification
[KAFKA-9258] - Connect ConnectorStatusMetricsGroup sometimes NPE
[KAFKA-9267] - ZkSecurityMigrator should not create /controller node
[KAFKA-9287] - Transaction completion may block unnecessarily after abortable error
[KAFKA-9297] - CreateTopic API do not work with older version of the request/response
[KAFKA-9305] - Add version 2.4 to streams system tests
[KAFKA-9329] - KafkaController::replicasAreValid should return error
[KAFKA-9330] - Calling AdminClient.close in the AdminClient completion callback causes deadlock
[KAFKA-9338] - Incremental fetch sessions do not maintain or use leader epoch for fencing purposes
[KAFKA-9355] - RocksDB statistics are removed from JMX when EOS enabled and empty local state dir
[KAFKA-9364] - Fix misleading consumer logs on throttling
[KAFKA-9379] - Flaky Test TopicCommandWithAdminClientTest.testCreateAlterTopicWithRackAware
[KAFKA-9386] - Flaky test AclAuthorizerTest.testHighConcurrencyDeletionOfResourceAcls
[KAFKA-9390] - Non-key joining of KTable not compatible with confluent avro serdes
[KAFKA-9457] - Flaky test org.apache.kafka.common.network.SelectorTest.testGracefulClose
[KAFKA-9462] - Correct exception message in DistributedHerder
[KAFKA-9480] - Value for Task-level Metric process-rate is Constant Zero
[KAFKA-9491] - Fast election during reassignment can lead to replica fetcher failures
[KAFKA-9492] - ProducerResponse with record-level errors throw NPE with older client version
[KAFKA-9500] - Foreign-Key Join creates an invalid topology
[KAFKA-9503] - TopologyTestDriver processes intermediate results in the wrong order
[KAFKA-9505] - InternalTopicManager may falls into infinite loop with partially created topics
[KAFKA-9507] - AdminClient should check for missing committed offsets
[KAFKA-9512] - Flaky Test LagFetchIntegrationTest.shouldFetchLagsDuringRestoration
[KAFKA-9517] - KTable Joins Without Materialized Argument Yield Results That Further Joins NPE On
[KAFKA-9535] - Metadata not updated when consumer encounters FENCED_LEADER_EPOCH
[KAFKA-9556] - KIP-558 cannot be fully disabled and when enabled topic reset not working on connector deletion
[KAFKA-9558] - getListOffsetsCalls doesn't update node in case of leader change
[KAFKA-9563] - Fix Kafka connect consumer and producer override documentation
[KAFKA-9577] - Client encountering SASL_HANDSHAKE protocol version errors on 2.5 / trunk
[KAFKA-9599] - create unique sensor to record group rebalance
[KAFKA-9601] - Workers log raw connector configs, including values
[KAFKA-9634] - ConfigProvider does not document thread safety
[KAFKA-9658] - Removing default user quota doesn't take effect until broker restart
[KAFKA-9661] - Config synonyms are no longer included in kafka-configs --describe output
[KAFKA-9662] - Throttling system test fails when messages are produced before consumer starts up
[KAFKA-9668] - Iterating over KafkaStreams.getAllMetadata() results in ConcurrentModificationException
[KAFKA-9700] - Negative estimatedCompressionRatio leads to misjudgment about if there is no room
[KAFKA-9701] - Consumer could catch InconsistentGroupProtocolException during rebalance
[KAFKA-9706] - Flatten transformation fails when encountering tombstone event
[KAFKA-9707] - InsertField.Key transformation should apply to tombstone records
[KAFKA-9712] - Reflections library 0.9.12 introduced in 2.5 causes regression scanning for plugins on plugin_path
[KAFKA-9741] - ConsumerCoordinator must update ConsumerGroupMetadata before calling onPartitionsRevoked()
[KAFKA-9749] - TransactionMarkerRequestCompletionHandler should treat storage exceptions as retriable
[KAFKA-9750] - Flaky test kafka.server.ReplicaManagerTest.testFencedErrorCausedByBecomeLeader
[KAFKA-9752] - Consumer rebalance can be stuck after new member timeout with old JoinGroup version
[KAFKA-9770] - Caching State Store does not Close Underlying State Store When Exception is Thrown During Flushing
[KAFKA-9801] - Static member could get empty assignment unexpectedly
[KAFKA-9807] - Race condition updating high watermark allows reads above LSO
[KAFKA-9815] - Consumer may never re-join if inconsistent metadata is received once
Task
[KAFKA-7737] - Consolidate InitProducerId API
[KAFKA-8963] - Benchmark and optimize incremental fetch session handler
[KAFKA-9343] - Add ps command for Kafka and zookeeper process on z/OS.
[KAFKA-9606] - Document Metrics Changes from KIP-444
Test
[KAFKA-9181] - Flaky test kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithoutDescribeAclViaSubscribe
[KAFKA-9188] - Flaky Test SslAdminClientIntegrationTest.testSynchronousAuthorizerAclUpdatesBlockRequestThreads
[KAFKA-9319] - Run some system tests using TLSv1.3
[KAFKA-9334] - Add more unit tests for Materialized class
[KAFKA-9509] - Fix flaky test MirrorConnectorsIntegrationTest.testReplication
[KAFKA-9523] - Reduce flakiness of BranchedMultiLevelRepartitionConnectedTopologyTest
Sub-task
[KAFKA-8421] - Allow consumer.poll() to return data in the middle of rebalance
[KAFKA-8617] - Replace EndTxn request/response with automated protocol
[KAFKA-8847] - Deprecate and remove usage of supporting classes in kafka.security.auth
[KAFKA-8988] - Replace CreatePartitions request/response with automated protocol
[KAFKA-9026] - Replace DescribeAcls request/response with automated protocol
[KAFKA-9027] - Replace CreateAcls request/response with automated protocol
[KAFKA-9028] - Replace DeleteAcls request/response with automated protocol
[KAFKA-9383] - Expose Consumer Group Metadata for Transactional Producer
[KAFKA-9417] - Integration test for new EOS model with vanilla Producer and Consumer
[KAFKA-9418] - Add new sendOffsets API to include consumer group metadata
[KAFKA-9428] - Expose standby information in KafkaStreams via queryMetadataForKey API
[KAFKA-9429] - Allow ability to control whether stale reads out of state stores are desirable
[KAFKA-9431] - Expose API in KafkaStreams to fetch all local offset lags
[KAFKA-9519] - Deprecate ZooKeeper access for kafka-configs.sh
[KAFKA-9582] - Ensure EOS task producer not through fatal error in unclean shutdown
[KAFKA-9759] - Add documentation change for KIP-562
