Let’s break it down step by step using official Kafka architecture principles:
Total partitions: 10
Each partition stores: 25 GB per day
Total data for the topic per day (no replication): 10 partitions × 25 GB = 250 GB
Replication factor: 3 → three copies of each partition across the brokers
Each partition is replicated across three brokers. Since the replication factor is three, each broker will have one full replica of one-third of the partitions. In a balanced cluster with three brokers:
Each broker stores a replica of every partition (since replication factor = number of brokers).
So, each broker will store the full 250 GB of data for that topic.
Hence:
Each broker will consume: 250 GB
This is confirmed in the official Apache Kafka documentation:
“A replica is a fully-qualified copy of a partition. When the replication factor equals the number of brokers, each broker contains a full copy of the topic data.”
“Each broker stores the total size of data for the topic, regardless of which partitions it leads.”
Page Reference:
Confluent Kafka: The Definitive Guide, 1st Edition, Chapter 6 (Kafka Cluster), p. 209–210.
Apache Kafka Documentation, “Replication” and “Storage” sections.
‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾
Submit