发布通知 - Kafka - 2.0.0版本

以下是Kafka 2.0.0版本中解决的JIRA问题的摘要。有关该版本的完整文档,入门指南以及有关该项目的信息,请参阅Kafka官方网站。

有关升级的注意事项:在升级群集之前,请仔细阅读此版本的升级文档。升级有关不兼容性和破坏性的变更,性能变化以及可能影响Kafka生产部署的任何其他更改的任何关键信息。

新功能

[KAFKA-6576] - 可配置配额管理(KIP-257)
[KAFKA-6841] - 添加对前缀ACL的支持
[KAFKA-6886] - Kafka Connect配置的Externalize Secrets
[KAFKA-6935] - KIP-295添加Streams配置以进行可选优化

改善

[KAFKA-1894] - 避免消费者长时间或无限制地阻塞
[KAFKA-2061] - 提供 a--version标志来打印kafka版本
[KAFKA-3379] - 更新依靠旧生产者使用新生产者的工具。
[KAFKA-3473] - KIP-237: 更多控制器健康指标
[KAFKA-3591] - 如果提供的查询不匹配任何值,JmxTool应退出
[KAFKA-3806] - 调整log.retention.hours和offsets.retention.minutes的默认值
[KAFKA-3834] - 消费者不应该阻止coordinator的发现
[KAFKA-4292] - KIP-86: 可配置的SASL回调处理程序
[KAFKA-4914] - 分区重新分配工具应该在ZooKeeper中持久化状态之前检查类型
[KAFKA-4936] - 允许动态路由输出记录
[KAFKA-5058] - 将传感器添加到KafkaStreams以跟踪由于具有空键而丢弃的记录
[KAFKA-5327] - 控制台消费者应该只poll最多的消息
[KAFKA-5370] - 将旧消费者的使用替换为新消费者
[KAFKA-5523] - ReplayLogProducer不使用新的Kafka使用者
[KAFKA-5529] - ConsoleProducer不推荐使用的BaseProducer
[KAFKA-5674] - max.connections.per.ip最小值为零以允许IP地址阻塞
[KAFKA-5907] - 支持Java 9中的aggregatedJavadoc
[KAFKA-5965] - 从Streams Resetter Tool中删除不推荐使用的AdminClient
[KAFKA-5987] - 文档生成中使用的Kafka度量标准模板应保持标记的顺序
[KAFKA-6024] - 考虑在调用acquireAndEnsureOpen()之前移动在KafkaConsumer中的验证
[KAFKA-6028] - 改善配额节流通信。
[KAFKA-6057] - 用户在偏移重置工具中忘记了`--execute`
[KAFKA-6058] - KIP-222: 将“描述使用者组”和“列出使用者组”添加到KafkaAdminClient
[KAFKA-6106] - 推迟线程内任务的正常处理,直到完成所有任务的恢复
[KAFKA-6184] - 报告消费者偏移量与日志的起始偏移量之间的滞后量度量
[KAFKA-6376] - 改进跳过记录的Streams指标
[KAFKA-6424] - QueryableStateIntegrationTest#queryOnRebalance应该接受原始文本
[KAFKA-6454] - 允许处理器API中的时间戳操作
[KAFKA-6473] - 将MockProcessorContext添加到public test-utils
[KAFKA-6481] - 提高ControllerChannelManager.addUpdateMetadataRequestForBrokers函数的性能
[KAFKA-6486] - TimeWindows causes unordered calls to windowed aggregation functions
[KAFKA-6514] - Add API version as a tag for the RequestsPerSec metric
[KAFKA-6515] - Add toString() method to kafka connect Field class
[KAFKA-6526] - Update controller to handle changes to unclean.leader.election.enable
[KAFKA-6535] - Set default retention ms for Streams repartition topics to Long.MAX_VALUE
[KAFKA-6538] - Enhance ByteStore exceptions with more context information
[KAFKA-6546] - Add ENDPOINT_NOT_FOUND_ON_LEADER error code for missing listener
[KAFKA-6560] - Use single-point queries than range queries for windowed aggregation operators
[KAFKA-6562] - KIP-255: OAuth Authentication via SASL/OAUTHBEARER
[KAFKA-6608] - Add TimeoutException to KafkaConsumer#position()
[KAFKA-6611] - Re-write simple benchmark in system tests with JMXTool
[KAFKA-6615] - Add scripts for DumpLogSegments
[KAFKA-6628] - RocksDBSegmentedBytesStoreTest does not cover time window serdes
[KAFKA-6630] - Speed up the processing of TopicDeletionStopReplicaResponseReceived events on the controller
[KAFKA-6640] - Improve efficiency of KafkaAdminClient.describeTopics()
[KAFKA-6657] - Add StreamsConfig prefix for different consumers
[KAFKA-6659] - Improve error message if state store is not found
[KAFKA-6670] - Implement a Scala wrapper library for Kafka Streams
[KAFKA-6673] - Segment and Stamped implement Comparable, but don't override equals.
[KAFKA-6677] - Remove EOS producer config max.in.flight.request.per.connection=1
[KAFKA-6685] - Connect deserialization log message should distinguish key from value
[KAFKA-6688] - The Trogdor coordinator should track task statuses
[KAFKA-6726] - KIP-277 - Fine Grained ACL for CreateTopics API
[KAFKA-6727] - org.apache.kafka.clients.admin.Config has broken equals and hashCode method.
[KAFKA-6730] - Simplify state store recovery
[KAFKA-6738] - Kafka Connect handling of bad data
[KAFKA-6750] - Add listener name to AuthenticationContext
[KAFKA-6760] - responses not logged properly in controller
[KAFKA-6772] - Broker should load credentials from ZK before requests are allowed
[KAFKA-6776] - Connect Rest Extension Plugin
[KAFKA-6795] - Add unit test for ReplicaAlterLogDirsThread
[KAFKA-6802] - Improve logging when topics aren't known and assignments skipped
[KAFKA-6813] - Remove deprecated APIs from KIP-120 and KIP-182 in Streams
[KAFKA-6849] - Add transformValues() method to KTable
[KAFKA-6850] - KIP-244: Add Record Header support to Kafka Streams Processor API
[KAFKA-6896] - add producer metrics exporting in KafkaStreams.java
[KAFKA-6905] - Document that Processor objects can be reused
[KAFKA-6913] - Add primitive numeric converters to Connect
[KAFKA-6930] - Update KafkaZkClient debug log
[KAFKA-6938] - Add documentation for accessing Headers on Kafka Streams Processor API
[KAFKA-6955] - Use Java AdminClient in DeleteRecordsCommand
[KAFKA-6957] - Add getter to AbstractStream class to make internalTopologyBuilder accessible outside of package
[KAFKA-6979] - Add max.block.ms to consumer for default timeout behavior
[KAFKA-6997] - Kafka run class doesn't exclude test-sources jar
[KAFKA-7000] - KafkaConsumer.position should wait for assignment metadata
[KAFKA-7009] - Mute logger for reflections.org at the warn level in system tests
[KAFKA-7050] - Decrease consumer request timeout to 30s
[KAFKA-7066] - Make Streams Runtime Error User Friendly in Case of Serialisation exception
[KAFKA-7091] - AdminClient should handle FindCoordinatorResponse errors
[KAFKA-7111] - Review the NetworkClient log level used

Bug

[KAFKA-3177] - Kafka consumer can hang when position() is called on a non-existing partition.
[KAFKA-3417] - Invalid characters in config properties not being validated?
[KAFKA-3665] - Default ssl.endpoint.identification.algorithm should be https
[KAFKA-3899] - Consumer.poll() stuck in loop if wrong credentials are supplied
[KAFKA-4041] - kafka unable to reconnect to zookeeper behind an ELB
[KAFKA-4831] - Extract WindowedSerde to public APIs
[KAFKA-4879] - KafkaConsumer.position may hang forever when deleting a topic
[KAFKA-4883] - invalid client sasl.jaas.config triggers NullPointerException
[KAFKA-5253] - TopologyTestDriver must handle streams created with patterns
[KAFKA-5540] - Deprecate and remove internal converter configs
[KAFKA-5588] - Remove deprecated new-consumer option for tools
[KAFKA-5624] - Unsafe use of expired sensors
[KAFKA-5697] - StreamThread.shutdown() need to interrupt the stream threads to break the loop
[KAFKA-5807] - Check Connector.config() and Transformation.config() returns a valid ConfigDef
[KAFKA-5919] - Adding checks on "version" field for tools using it
[KAFKA-5974] - Removed unused parameter ProcessorContext
[KAFKA-6052] - Windows: Consumers not polling when isolation.level=read_committed
[KAFKA-6054] - ERROR "SubscriptionInfo - unable to decode subscription data: version=2" when upgrading from 0.10.0.0 to 0.10.2.1
[KAFKA-6238] - Issues with protocol version when applying a rolling upgrade to 1.0.0
[KAFKA-6253] - Improve sink connector topic regex validation
[KAFKA-6264] - Log cleaner thread may die on legacy segment containing messages whose offsets are too large
[KAFKA-6288] - Broken symlink interrupts scanning the plugin path
[KAFKA-6292] - KafkaConsumer ran into Unknown error fetching data for topic-partition caused by integer overflow in FileLogInputStream
[KAFKA-6299] - Fix AdminClient error handling when metadata changes
[KAFKA-6351] - libs directory has duplicate javassist jars
[KAFKA-6361] - Fast leader fail over can lead to log divergence between leader and follower
[KAFKA-6378] - NullPointerException on KStream-GlobalKTable leftJoin when KeyValueMapper returns null
[KAFKA-6386] - Deprecate KafkaStreams constructor taking StreamsConfig parameter
[KAFKA-6390] - Update ZooKeeper to 3.4.12, Gradle and other minor updates
[KAFKA-6394] - Prevent misconfiguration of advertised listeners
[KAFKA-6427] - Inconsistent exception type from KafkaConsumer.position
[KAFKA-6445] - Remove deprecated metrics in 2.0
[KAFKA-6446] - KafkaProducer with transactionId endless waits when bootstrap server is down
[KAFKA-6503] - Connect: Plugin scan is very slow
[KAFKA-6513] - New Connect header support doesn't define `converter.type` property correctly
[KAFKA-6530] - Use actual first offset of messages when rolling log segment for magic v2
[KAFKA-6534] - Consumer.poll may not trigger rebalance in time when there is a task migration
[KAFKA-6552] - 鈥渆ntity_type鈥� not exactly in description of kafka-configs.sh
[KAFKA-6566] - SourceTask#stop() not called after exception raised in poll()
[KAFKA-6577] - Connect standalone SASL file source and sink test fails without explanation
[KAFKA-6581] - ConsumerGroupCommand hangs if even one of the partition is unavailable
[KAFKA-6637] - if set topic config segment.ms=0 Kafka broker won't be able to start
[KAFKA-6650] - The controller should be able to handle a partially deleted topic
[KAFKA-6656] - Use non-zero status code when kafka-configs.sh fails
[KAFKA-6658] - Fix RoundTripWorkload and make k/v generation configurable
[KAFKA-6672] - ConfigCommand failing to alter configs
[KAFKA-6697] - JBOD configured broker should not die if log directory is invalid
[KAFKA-6704] - Checking hasNext from SegementIterator could throw InvalidStateStoreException
[KAFKA-6709] - broker failed to handle request due to OOM
[KAFKA-6710] - Streams integration tests hang during shutdown
[KAFKA-6711] - GlobalStateManagerImpl should not write offsets of in-memory stores in checkpoint file
[KAFKA-6724] - ConsumerPerformance resets offsets on every startup
[KAFKA-6728] - Kafka Connect Header Null Pointer Exception
[KAFKA-6729] - KTable should use user source topics if possible and not create changelog topic
[KAFKA-6731] - waitOnState waits for the wrong state instead of the target one
[KAFKA-6737] - Is Kafka imapcted by critical vulnerqbilty CVE-2018-7489
[KAFKA-6739] - Down-conversion fails for records with headers
[KAFKA-6741] - Transient test failure: SslTransportLayerTest.testNetworkThreadTimeRecorded
[KAFKA-6742] - TopologyTestDriver error when dealing with stores from GlobalKTable
[KAFKA-6743] - ConsumerPerformance fails to consume all messages on topics with large number of partitions
[KAFKA-6747] - kafka-streams Invalid transition attempted from state READY to state ABORTING_TRANSACTION
[KAFKA-6748] - Scheduler cannot be cancelled from Punctuator
[KAFKA-6749] - TopologyTestDriver fails when topoloy under test uses EXACTLY_ONCE
[KAFKA-6752] - Unclean leader election metric no longer working
[KAFKA-6765] - Intermittent test failure in CustomQuotaCallbackTest
[KAFKA-6775] - AbstractProcessor created in SimpleBenchmark should call super#init
[KAFKA-6778] - DescribeConfigs does not return error for non-existent topic
[KAFKA-6782] - GlobalKTable GlobalStateStore never finishes restoring when consuming aborted messages
[KAFKA-6796] - Surprising UNKNOWN_TOPIC error for produce/fetch requests to non-replicas
[KAFKA-6809] - connections-created metric does not behave as expected
[KAFKA-6825] - DEFAULT_PRODUCTION_EXCEPTION_HANDLER_CLASS_CONFIG is private
[KAFKA-6826] - Avoid range scans when forwarding values in window store aggregations
[KAFKA-6829] - Consumer should retry when encountering unknown topic or partition error
[KAFKA-6834] - log cleaner should handle the case when the size of a message set is larger than the max message size
[KAFKA-6844] - Race condition between StreamThread and GlobalStreamThread stopping
[KAFKA-6853] - ResponseMetadata calculates latency incorrectly (and therefore ZooKeeperRequestLatencyMs is incorrect)
[KAFKA-6855] - Kafka fails to start with Java 10 due to faulty Java version detection
[KAFKA-6857] - LeaderEpochFileCache.endOffsetFor() should check for UNDEFINED_EPOCH explicitly
[KAFKA-6860] - NPE when reinitializeStateStores with eos enabled
[KAFKA-6870] - Concurrency conflicts in SampledStat
[KAFKA-6871] - KStreams Scala API: incorrect Javadocs and misleading parameter name
[KAFKA-6877] - Remove completedFetch upon a failed parse if it contains no records.
[KAFKA-6878] - NPE when querying global state store not in READY state
[KAFKA-6879] - Controller deadlock following session expiration
[KAFKA-6894] - Improve error message when connecting processor with a global store
[KAFKA-6897] - Mirrormaker waits to shut down forever on produce failure with abort.on.send.failure=true
[KAFKA-6906] - Kafka Streams does not commit transactions if data is produced via wall-clock punctuation
[KAFKA-6919] - Trogdor documentation points to wrong location for trogdor.sh
[KAFKA-6927] - Broker uses significant amount of memory during down-conversion
[KAFKA-6929] - ZkData - Consumers offsets Zookeeper path is not correct
[KAFKA-6936] - Scala API Wrapper for Streams uses default serializer for table aggregate
[KAFKA-6937] - In-sync replica delayed during fetch if replica throttle is exceeded
[KAFKA-6949] - alterReplicaLogDirs() should grab partition lock when accessing log of the future replica
[KAFKA-6967] - TopologyTestDriver does not allow pre-populating state stores that have change logging
[KAFKA-6973] - setting invalid timestamp causes Kafka broker restart to fail
[KAFKA-6975] - AdminClient.deleteRecords() may cause replicas unable to fetch from beginning
[KAFKA-6981] - Missing Connector Config (errors.deadletterqueue.topic.name) kills Connect Clusters
[KAFKA-6991] - Connect Rest Extension Plugin issue with Class Loader
[KAFKA-6993] - Fix defective documentations for KStream/KTable methods
[KAFKA-7012] - Performance issue upgrading to kafka 1.0.1 or 1.1
[KAFKA-7021] - Source KTable checkpoint is not correct
[KAFKA-7023] - Kafka Streams RocksDB bulk loading config may not be honored with customized RocksDBConfigSetter
[KAFKA-7028] - super.users doesn't work with custom principals
[KAFKA-7029] - ReplicaVerificationTool should not use the deprecated SimpleConsumer
[KAFKA-7031] - Kafka Connect API module depends on Jersey
[KAFKA-7032] - The TimeUnit is neglected by KakfaConsumer#close(long, TimeUnit)
[KAFKA-7039] - DelegatingClassLoader creates plugin instance even if its not Versioned
[KAFKA-7043] - Connect isolation whitelist does not include new primitive converters (KIP-305)
[KAFKA-7047] - Connect isolation whitelist does not include SimpleHeaderConverter
[KAFKA-7048] - NPE when creating connector
[KAFKA-7055] - Kafka Streams Processor API allows you to add sinks and processors without parent
[KAFKA-7056] - Connect's new numeric converters should be in a different package
[KAFKA-7058] - ConnectSchema#equals() broken for array-typed default values
[KAFKA-7064] - "Unexpected resource type GROUP" when describing broker configs using latest admin client
[KAFKA-7067] - ConnectRestApiTest fails assertion
[KAFKA-7068] - ConfigTransformer doesn't handle null values
[KAFKA-7076] - Broker startup could be inefficient when using old message format
[KAFKA-7082] - Concurrent createTopics calls may throw NodeExistsException
[KAFKA-7104] - ReplicaFetcher thread may die because of inconsistent log start offset in fetch response
[KAFKA-7112] - StreamThread does not check for state again after pollRequests()
[KAFKA-7136] - PushHttpMetricsReporter may deadlock when processing metrics changes
[KAFKA-7168] - Broker shutdown during SSL handshake may be handled as handshake failure
[KAFKA-7182] - SASL/OAUTHBEARER client response is missing %x01 separators
[KAFKA-7185] - getMatchingAcls throws StringIndexOutOfBoundsException for empty resource name
[KAFKA-7193] - ZooKeeper client times out with localhost due to random choice of ipv4/ipv6
[KAFKA-7194] - Error deserializing assignment after rebalance

Task

[KAFKA-2983] - Remove old Scala consumer and all related code, tests, and tools
[KAFKA-4423] - Drop support for Java 7
[KAFKA-5660] - Don't throw TopologyBuilderException during runtime
[KAFKA-5677] - Remove deprecated punctuate method
[KAFKA-6769] - Upgrade jetty library version
[KAFKA-6800] - Update documentation for SASL/PLAIN and SCRAM to use callbacks
[KAFKA-6805] - Allow dynamic broker configs to be configured in ZooKeeper before starting broker
[KAFKA-6810] - Enable dynamic reconfiguration of SSL truststores
[KAFKA-6854] - Log cleaner fails with transaction markers that are deleted during clean
[KAFKA-6911] - Incorrect check for keystore/truststore dynamic update
[KAFKA-6912] - Add authorization tests for custom principal types
[KAFKA-6916] - AdminClient does not refresh metadata on broker failure
[KAFKA-6917] - Request handler deadlocks attempting to acquire group metadata lock
[KAFKA-6921] - Remove old Scala producer and all related code, tests, and tools
[KAFKA-7001] - Rename errors.allowed.max property in Connect to errors.tolerance
[KAFKA-7002] - Allow replication factor to be set via a configuration property for the Connect DLQ topic
[KAFKA-7003] - Add headers with error context in messages written to the Connect DeadLetterQueue topic

Test

[KAFKA-2951] - Additional authorization test cases
[KAFKA-5739] - Rewrite KStreamPeekTest at processor level avoiding driver usage
[KAFKA-5944] - Add unit tests for handling of authentication failures in clients
[KAFKA-6716] - discardChannel should be released in MockSelector#completeSend

Sub-task

[KAFKA-2391] - Blocking call such as position(), partitionsFor(), committed() and listTopics() should have a timeout
[KAFKA-2837] - FAILING TEST: kafka.api.ProducerBounceTest > testBrokerFailure
[KAFKA-3368] - Add the Message/Record set protocol to the protocol docs
[KAFKA-3387] - Update GetOffsetShell tool to not rely on old producer.
[KAFKA-4641] - Improve test coverage of StreamsThread
[KAFKA-4651] - Improve test coverage of Stores
[KAFKA-6447] - Add Delegation Token Operations to KafkaAdminClient
[KAFKA-7005] - Remove duplicate Java Resource class.
[KAFKA-7006] - Remove duplicate Scala ResourceNameType class
[KAFKA-7007] - Use JSON for /kafka-acl-extended-changes path
[KAFKA-7010] - Rename ResourceNameType.ANY to MATCH
[KAFKA-7011] - Investigate if its possible to drop the ResourceNameType field from Java Resource class.
[KAFKA-7030] - Add configuration to disable message down-conversion






发表于: 29天前   最后更新时间: 2天前   游览量:101
上一条:
下一条: Apache Kafka 0.8.2.0 发布

评论…


  • 评论…
    • in this conversation