Steam Recommender System

Collaborative filtering recommender system for Steam games using implicit-feedback ALS.

Key result: NDCG@10 = 79.79% on held-out test data.

Overview

This project builds an end-to-end recommendation pipeline on top of 5M+ Steam playtime records from UC San Diego's McAuley Lab dataset. The pipeline covers raw data ingestion, cleaning and reshaping nested JSON, hyperparameter tuning via grid search, and final evaluation using a ranking metric.

The model uses Alternating Least Squares (ALS) with implicit feedback, treating playtime as a confidence-weighted preference signal rather than an explicit star rating.

A full write-up of the project is available at the blog post.

What This Demonstrates

• End-to-end DS workflow: data ingestion → cleaning/reshaping → modeling → evaluation
• Working with real-world messy nested data (Steam user-item histories)
• Recommender-system fundamentals:
  • Implicit feedback (playtime) vs. explicit ratings
  • Sparse user-item matrices (CSR format)
  • ALS factorization and practical hyperparameter tuning
• Pragmatic model validation using ranking metrics (NDCG@K)

Pipeline

Part 1 — Data Collection

Downloads the UCSD Steam dataset (V1 User-Items): 70.6 MB compressed → 527.5 MB extracted. The raw file is a JSON-by-line format using Python dict literals (not valid JSON), parsed with ast.literal_eval and pickled to preserve nested structure.

Part 2 — Data Preparation

Resolves four issues in the raw data:

1. Naming confusion — steam_id is the true unique identifier; user_id is just a username handle.
2. Nested structure — explodes one row per user into one row per user–item pair.
3. Cold-start — drops users with fewer than 10 games, retaining 57,333 users and 5,038,365 interactions.
4. Heavy-tailed playtime — applies log(1 + playtime) to compress the tail so power users don't dominate.

Splits are done per user (not randomly) so every user appears in train, validation, and test.

Sample	Users	Items	Matrix Elements
0.1%	57	1,590	90,630
1%	573	4,110	2,355,030
10%	5,733	7,911	45,353,763
100%	57,333	10,976	629,287,008

Part 3 — Implicit ALS Model

Uses the implicit library. Grid search over 125 combinations:

Hyperparameter	Values
Latent factors	16, 24, 32, 48, 64
Regularization λ	0.001, 0.01, 0.1, 10, 100
Confidence weight α	0.1, 0.5, 1.0, 5.0, 10.0

Best validation configuration: factors=64, λ=10, α=0.5 (NDCG@10 = 89.47% on validation).

Regularization was the dominant factor — λ=10 outperformed all other values by a wide margin.

Results

Split	NDCG@10
Validation	89.47%
Test	79.79%

Final model trained on train + validation combined, evaluated on the held-out test set using the 1% user sample.

Setup

pip install -r requirements.txt

Key dependencies: numpy, pandas, scipy, implicit, scikit-learn, tqdm, rich, requests

Usage

Download and extract the raw data:

python download.py

Run the notebooks in order:

notebooks/01-Download-Data.ipynb
notebooks/02-Clean-Data.ipynb
notebooks/03-MF-Model.ipynb

Project Structure

.
├── download.py              # Download, extract, and validate raw data
├── evaluate.py              # Hit ratio and NDCG evaluation utilities
├── requirements.txt
├── notebooks/
│   ├── 01-Download-Data.ipynb
│   ├── 02-Clean-Data.ipynb
│   └── 03-MF-Model.ipynb
├── src/
│   ├── config/              # Directory/path/URL constants
│   └── datasets/            # Data loading and preprocessing modules
├── data/
│   └── raw/                 # Downloaded and extracted data (not tracked)
└── related-work/            # Reference papers

Data Source

McAuley, J. and Kang, W.-C. (UCSD). Australian Steam Users — V1 User Items. Available at: https://cseweb.ucsd.edu/~jmcauley/datasets.html#steam_data