Given that you use AUCPR you might need to explicitly set the parameter maximize=True, otherwise in your final run you could minimize the AUCPR, which could explain poor results.Either increase n_estimators or include it in the parameters grid. From your code it is unclear how many rounds you use.

In particular, check early_stopping_rounds and eval_metric.Īre there other parameters that could influence or improve my model? Make sure that you use the consistent parameters in tuning process and in the final evaluation, or just include these parameters in the parameters grid if possible.You can assess the stability of your current results by using different random seeds and repeating the entire exercise. 5-10, the results with cv = 3 might be very unstable. Furthermore, if you think that subsample and cilsample_bytree (a typo?) are relevant parameters include them in the parameters grid. Instead of using the hard-coded parameters in the XGBClassifier call, use the optimal parameters found by tuning process, i.e.I suggest the following in order to investigate and fix the problem In this case it is incorrect to talk about the model performance 'after running the optimization'. If the distribution of training and test data differ significantly it could lead to the situation when the parameters providing the best results on the training data perform poorly on test data.Īs already mentioned by others, the parameters you are testing after the tuning were not included in the parameter grid. If understand correctly, the choice of the best parameters is based on the cv results on training data, while in your final run the performance is assessed based on test dataset. Such behavior rather indicates that something is wrong with your approach or your data. However, assuming you have enough data and your parameter grid is wide enough it is rather unlikely that the tuning of hyper-parameters would not be able to find better results. Yes, theoretically, by pure luck, it is possible that your initial guess, before optimization of hyper-parameters, provides better results than the best of parameter combination found in the parameters grid.

Is it possible that after running the optimization my score won't get better (and even worse?) ? Is it possible that after running the optimization my score won't get better (and even worse?) ? Where is my mistake? Are there other parameters that could influence or improve my model? I always get lower score than what I got before the optimization which makes me suspect that I'm missing something in the way doing it/basic principle in this process. The problem is that when I check the model score clf_xgb.score(X_test,y_test) (and now based on the result changing the params.)Īfter this step I choose the best hyperparameters and run the model clf_xgb=xgb.XGBClassifier(seed=42, Grid_search = GridSearchCV(estimator = clf_xgb, param_grid = param_grid, In order to optimize my model I define param grid and than fit with the train data and then according to the results run again with nre parameters, e.g- #ROUND 1

I am trying recently to optimize models but for some reason, whenever I try to run the optimization the model score in the end is worse than before, so I believe I do something wrong.

Progressively worse typo generator