Databricks Databricks-Certified-Associate-Developer-for-Apache-Spark-3.5 Question # 14 Topic 2 Discussion

Comprehensive and Detailed Explanation From Exact Extract:

ThegroupBy()operation causes a shuffle because it requires all values for a specific key to be brought together, which may involve moving data across partitions.

In contrast:

filter()andselect()are narrow transformations and do not cause shuffles.

coalesce()tries to reduce the number of partitions and avoids shuffling by moving data to fewer partitions without a full shuffle (unlikerepartition()).