Beer Quality Model Selection

Bilal Vandenberge and I built this project for a Kaggle beer-quality competition. Given a beer style and twelve brewing measurements, the task is to predict its exact quality grade. Kaggle scores accuracy, so predicting grade 6 instead of 5 counts as an error just like predicting 9 instead of 5.

The main difficulty is the label distribution. The training set contains grades from 3 to 9, but 4,236 of its 4,469 examples are grades 5, 6 or 7. That is 94.8% of the data. Grade 9 appears only three times. A model can score reasonably well by staying near the middle of the scale while doing very little for the rare grades.

We started with K-nearest neighbours, then moved to gradient boosting once we had a clear baseline. Accuracy remained the main selection metric because that is what Kaggle used, but we also tracked macro-F1 to see how the model behaved beyond the dominant grades.

Modeling approach

K-nearest neighbours gave us a straightforward distance-based baseline. Since its predictions depend on distances between examples, the numerical measurements were standardized first. With k=15, the report records about 0.58 accuracy and 0.24 macro-F1. Testing several values of k also showed that the best setting for accuracy was not the best one for balanced performance across grades.

After KNN, we moved to scikit-learn's HistGradientBoostingClassifier. It gave us a way to model nonlinear relationships between the brewing measurements and test feature selection, engineered ratios and class weights. We tuned its learning rate, leaf structure, regularization and histogram resolution over 80 Bayesian-search trials using stratified cross-validation.

We added two ratios to the original measurements:

IBU per ABV = bitterness / alcohol
CO2 per ABV = free CO2 / alcohol

Duplicating examples until every class had the same size would have given a lot of influence to very few observations, especially the three grade-9 rows. We used capped square-root class weights instead. If $\tilde{n}$ is the median class count and $n_c$ is the number of training examples in class $c$, then:

$$ w_c = \min\left(\sqrt{\frac{\tilde{n}}{n_c}}, 3\right) $$

The notebook also tests a decision adjustment over five out-of-fold runs. Since that adjustment is tuned for macro-F1, I kept it as a diagnostic. The submitted predictions come directly from the model because the competition metric is accuracy.

Results

Evaluation	Acc.	Macro-F1
Always predict grade 6	0.444	-
KNN baseline, k=15	~0.58	~0.24
Tuned HistGradientBoosting, holdout split	0.613	0.342

In the clean notebook run, the best mean cross-validation accuracy was 0.618 across five folds. The run used the pinned Python 3.11 environment, and the fitted parameters and cell outputs are saved in the notebook.

Accuracy is the number used to rank the Kaggle submissions. Macro-F1 is included because it helps show whether an improvement comes only from grades 5, 6 and 7. With just three grade-9 examples, performance on every rare grade cannot be estimated reliably from a single split.

The technical report goes further into the class distribution, feature-importance analysis, model choices and experiments. The public PDF credits both contributors but omits the student numbers.

Repository

File	Role
notebook.ipynb	Preprocessing, Bayesian tuning, class weighting, five-seed OOF diagnostic and direct submission generation
Rapport.pdf	Four-page methodology and results report
data/train.csv	4,469 labeled competition rows
data/test.csv	511 unlabeled rows used for submission
submission.csv	511 direct predictions for the accuracy-scored test set

Run the notebook

conda env create -f environment.yml
conda activate beer-ml
OMP_NUM_THREADS=1 jupyter lab notebook.ipynb

The notebook uses pandas and NumPy to prepare the data, scikit-learn for preprocessing and modeling, and scikit-optimize for the Bayesian search. OMP_NUM_THREADS=1 prevents each fit from starting its own group of threads while the cross-validation search distributes folds across the available cores.

The clean rerun took about 12 minutes.

Authors

Developed by Mateo Coda-Forno and Bilal Vandenberge. I maintain this repository.