Kafka Connect is running on a two node cluster in distributed mode. The connector is a source connector that pulls data from Postgres tables (users/payment/orders), writes to topics with two partitions, and with replication factor two. The development team notices that the data is lagging behind.
What should be done to reduce the data lag*?
The Connector definition is listed below:
{
"name": "confluent-postgresql-source",
"connector class": "PostgresSource",
"topic.prefix": "postgresql_",
& nbsp;& nbsp;& nbsp;…
"db.name": "postgres",
"table.whitelist": "users.payment.orders”,
"timestamp.column.name": "created_at",
"output.data format": "JSON",
"db.timezone": "UTC",
"tasks.max": "1"
}
Submit