Let's say there are
a) three factor covariates
b) Y variable, whether it be continuous or binary
If I only had to choose one covariate and PCA the other two (just for sake of ideation, not real modelling) to predict the Y variable, it would make sense that the one covariate shows least amount of error with the Y variable, correct (assuming there wouldn't be overfit)?
So then what would be the best approach to find this one covariate? Correlation? Regression/logit after one-hot coding? Something else?