Ames housing stacked ensemble

Result: the stack blends XGBoost, LightGBM, CatBoost, Ridge, and Lasso on a log target. The final RMSE is add the stack score in log space.

Problem

Ames is a small tabular dataset with a long tail of prices and a lot of categorical detail. It is also a trap for leakage. The goal was not a higher leaderboard score at any cost. The goal was a stack that could beat a single model under a fold safe pipeline.

Data

The dataset is the Ames housing table. The target is sale price. The train and validation design is add fold count, repeats, and seed policy. Outliers are handled with add the exact rule and where it is fit.

Approach and why

Log transform the target to reduce the influence of expensive homes.
Use an explicit outlier policy, fit inside each fold if the rule uses the data.
Ordinal encode ordered categorical features and keep the order documented.
Fit every preprocessing step inside a pipeline so it cannot see the validation fold.
Train XGBoost, LightGBM, CatBoost, Ridge, and Lasso.
Generate out of fold predictions and fit the stack on those predictions.
Compare the stack with a mean baseline, Ridge, and the best single model.

The stack is only useful if it beats the best single model. The baseline table is the claim.

Results

Model	RMSE, log target	Notes
Mean baseline	add score	Predicts the training mean
Ridge	add score	Linear baseline
Best single model	add model and score	add which model
Stack	add score	Out of fold predictions

The back transformed error in price units is add score and transform details. The leaderboard rank is add rank or remove the claim.

Evaluation

Leakage: outlier removal and any target based encoding must be fit inside the fold. The pipeline is checked for this.
Baseline: mean, Ridge, and the best single model.
Validation: repeated K fold with fixed seeds. Out of fold predictions are used for the stack.
Metric: RMSE on the log target, plus the back transformed error. The transform is stated.
Reproducibility: pinned dependencies, saved fold assignments, fixed seeds, and model artifacts.

A reviewer should challenge any stack score produced on the same data used to fit the weights. That is the most likely optimistic result here.

Limitations

The dataset is small and old. The outlier policy can change the score. The stack adds complexity for a gain that must be shown, not assumed. The result is pending, so no ranking claim.

Next steps

Run nested cross validation, report mean and standard deviation across repeats, add SHAP explanations, and package the pipeline behind a typed inference endpoint if the deployment path is confirmed.