When training a neural network for house price prediction, how does the optimization algorithm (like Gradient Descent) know whether its current weight settings are doing a good job or a terrible job, and in which direction it should adjust those weights?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Think of a cost function like the scorecard in a game of golf—except you want your score to be as close to zero as possible. When your model starts training, its weights are completely random, so its predictions are awful. The cost function (or loss function) calculates exactly how far off those predictions are from the real answers. If your model predicts a house is worth $100k but it actually sold for $500k, the loss function screams: "Hey, you're off by $400k!" The optimizer takes that feedback and adjusts the weights to bring the error down next time. Without a cost function, your model would be stumbling around in the dark with no idea if it's getting warmer or colder. It's the ultimate guide for training.
Full explanation below image
Full Explanation
In machine learning and deep learning, a loss function (often used interchangeably with cost function, though technically a loss function computes error for a single training example while a cost function averages it over the entire dataset) is a mathematical formula that measures the discrepancy between the model's predictions and the actual ground truth labels.
The primary purpose of the cost function is to provide a single quantitative value representing the model's current performance. This value serves as the objective function that the optimization algorithm (such as Gradient Descent or Adam) seeks to minimize. During training, the optimizer calculates the gradients of the cost function with respect to the model's weights. These gradients indicate the direction and magnitude of changes needed to reduce the overall error. Popular examples of cost functions include Mean Squared Error (MSE) for regression tasks and Binary or Categorical Cross-Entropy for classification tasks.
Let's examine the incorrect options: - Activation functions (Option A) are used to introduce non-linearity into the network, allowing it to learn complex non-linear decision boundaries, but they do not measure prediction error. - The learning rate (Option C) is a hyperparameter that controls the step size the optimizer takes during gradient descent; it does not compute the prediction error itself. - Regularization functions (Option D) are added to the cost function to penalize model complexity (like L1/L2 regularization) and prevent overfitting, but they are not the primary metric measuring the baseline prediction error against actual target data.
Therefore, the cost function (Option B) is the component that quantifies the error between predictions and target values.