XGBoost 调参实战：从入门到竞赛金牌

XGBoost 调参实战从入门到竞赛金牌1. XGBoost 核心参数XGBoost 参数分类 ├── 树参数 │ ├── max_depth树深度3-10 │ ├── min_child_weight叶子最小权重 │ ├── gamma分裂最小增益 │ └── subsample行采样比例 ├── 学习参数 │ ├── learning_rate学习率0.01-0.3 │ ├── n_estimators树数量 │ └── colsample_bytree列采样比例 ├── 正则化参数 │ ├── reg_alphaL1 正则化 │ └── reg_lambdaL2 正则化 └── 类别参数 ├── scale_pos_weight正负样本比 └── objective目标函数2. 分步调参法importxgboostasxgbfromsklearn.model_selectionimportGridSearchCV# 步骤 1确定树数量和学习率params_1{learning_rate:0.1,n_estimators:100,max_depth:5,min_child_weight:1,subsample:0.8,colsample_bytree:0.8,}# 步骤 2调 max_depth 和 min_child_weightparam_grid_2{max_depth:[3,5,7,9],min_child_weight:[1,3,5,7],}grid_2GridSearchCV(xgb.XGBClassifier(**params_1,use_label_encoderFalse),param_grid_2,cv5,scoringaccuracy)grid_2.fit(X_train,y_train)print(f最佳:{grid_2.best_params_})# 步骤 3调 subsample 和 colsample_bytreeparam_grid_3{subsample:[0.6,0.7,0.8,0.9],colsample_bytree:[0.6,0.7,0.8,0.9],}# 步骤 4调正则化param_grid_4{reg_alpha:[0,0.01,0.1,1],reg_lambda:[0,0.01,0.1,1],}# 步骤 5降低学习率增加树数量params_final{learning_rate:0.01,n_estimators:1000,early_stopping_rounds:50,}3. Optuna 自动调参importoptunadefobjective(trial):params{n_estimators:trial.suggest_int(n_estimators,100,1000),max_depth:trial.suggest_int(max_depth,3,10),learning_rate:trial.suggest_float(learning_rate,0.01,0.3,logTrue),min_child_weight:trial.suggest_int(min_child_weight,1,10),subsample:trial.suggest_float(subsample,0.6,1.0),colsample_bytree:trial.suggest_float(colsample_bytree,0.6,1.0),reg_alpha:trial.suggest_float(reg_alpha,1e-8,10.0,logTrue),reg_lambda:trial.suggest_float(reg_lambda,1e-8,10.0,logTrue),gamma:trial.suggest_float(gamma,0,5),}modelxgb.XGBClassifier(**params,use_label_encoderFalse,eval_metriclogloss)scorescross_val_score(model,X_train,y_train,cv5,scoringaccuracy)returnscores.mean()studyoptuna.create_study(directionmaximize)study.optimize(objective,n_trials200)4. 竞赛技巧竞赛常用技巧 ├── 特征工程 模型调参 ├── 多模型融合Blending/Stacking ├── 5-10 折交叉验证 ├── 数据增强SMOTE/噪声注入 ├── 伪标签Pseudo Labeling └── 后处理阈值优化总结阶段参数范围1learning_rate0.12max_depth, min_child_weight3-10, 1-73subsample, colsample0.6-1.04reg_alpha, reg_lambda0-105learning_rate0.01, n_estimators1000