During the data preprocessing stage, a machine learning engineer discovers that several independent variables in the dataset are highly correlated with one another. What is a standard engineering technique used to address this issue?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal: having highly correlated features in your dataset is like having two people in a meeting who always say the exact same thing. It's redundant, and it makes it really hard for your model to figure out who is actually contributing what. In data science, we call this multicollinearity. If you leave it alone, it can mess up your model's stability and performance. To fix this, you can use a dimensionality reduction technique like PCA (Principal Component Analysis). PCA takes all those correlated features and translates them into a brand-new set of features called principal components that are completely uncorrelated with each other. It keeps the most important information while cutting out the noise. Option A is the smart move.
Full explanation below image
Full Explanation
In machine learning, high correlation between input features is known as multicollinearity. Multicollinearity can cause numerical instability (especially in linear models) because it makes it difficult to estimate the individual relationship between each independent variable and the target variable.
A standard technique to address this is Principal Component Analysis (PCA), which is an unsupervised dimensionality reduction method. PCA projects the original features onto a lower-dimensional space. It does this by finding new directions (eigenvectors) of maximum variance in the data, which are orthogonal (at 90 degrees) to one another. Because these new features (principal components) are orthogonal, they are mathematically uncorrelated. This eliminates multicollinearity while retaining most of the variance (information) present in the original dataset.
Let's check the distractors: - Option B is incorrect because adding more samples to the dataset will not resolve the high correlation between the features if the underlying relationship between those variables is a physical or logical rule (e.g., height in inches and height in centimeters). - Option C is incorrect because dropping the target label converts a supervised learning task into an unsupervised task, which does not solve the target task. - Option D is incorrect because L1 regularization is applied to model weights during the training optimization process (to penalize and zero-out weights of less important features), not to the final predictions after inference has finished.
For the exam, associate PCA with dimensionality reduction and the creation of uncorrelated features (components) from correlated ones.