gravatar for Mensur Dlakic

1 hour ago by

USA

Generally speaking, using these abbreviations and jargon will not be sufficient for most posters to understand what you are doing or what you are asking. Keep in mind what your audience is next time and how general your question is. This one is very specialized and requires more information if you want constructive responses.

I will do my best with what you have told us. First, doing recursive feature elimination (RFE) simultaneously with cross-validation (CV) (RFECV) is not the same as doing them one at a time. Depending on what you scoring function is during RFECV (what is it you are optimizing for), you may end up with lower F1 score. When you do a separate feature elimination, it may end up with higher F1 score but a lower score that was actually used for optimization. In other words, RFECV will not find feature combination that will optimize absolutely all scores. Any given combo of features may produce the best log-loss score but not the best F1 or vice versa. If you wish to optimize specifically for F1, you should use it as your scoring function.



Source link