Databricks ML Data Scientist Databricks-Machine-Learning-Associate Question # 22 Topic 3 Discussion

To compute the root mean-squared-error (RMSE) of a linear regression model using Spark ML, theRegressionEvaluatorclass is used. TheRegressionEvaluatoris specifically designed for regression tasks and can calculate various metrics, including RMSE, based on the columns containing predictions and actual values.

The correct code block to compute RMSE from thepreds_dfDataFrame is:

regression_evaluator = RegressionEvaluator( predictionCol="prediction", labelCol="actual", metricName="rmse") rmse = regression_evaluator.evaluate(preds_df)

This code creates an instance ofRegressionEvaluator, specifying the prediction and label columns, as well as the metric to be computed ("rmse"). It then evaluates the predictions inpreds_dfand assigns the resulting RMSE value to thermsevariable.

Options A and B incorrectly useBinaryClassificationEvaluator, which is not suitable for regression tasks. Option D also incorrectly usesBinaryClassificationEvaluator.

References:

PySpark ML Documentation