You have a topic t1 with six partitions. You use Kafka Connect to send data from topic t1 in your Kafka cluster to Amazon S3. Kafka Connect is configured for two tasks.
When using Kafka Connect with sink connectors (like S3 Sink),each task is assigned one or more topic partitions. Thetotal number of partitions (6)isevenly distributedacross the available tasks (2). Thus, each task will handle3 partitions.
FromKafka Connect Documentation:
“Kafka Connect divides the topic partitions among available tasks. For example, a topic with six partitions and two tasks results in each task handling three partitions.”
[Reference:Kafka Connect Concepts > Tasks and Partitions, ============]
Contribute your Thoughts:
Chosen Answer:
This is a voting comment (?). You can switch to a simple comment. It is better to Upvote an existing comment if you don't have anything to add.
Submit