🏆 Datathon 2025 - Production Prediction Challenge

A machine learning project for predicting production values using tabular data and image embeddings. This repository contains the complete pipeline for data preprocessing, feature engineering, model training (CatBoost & Neural Networks), and model explainability.

---

📁 Project Structure

Datathon-2025/
├── data/                     # Dataset directory (gitignored)
│   ├── train.csv            # Training data
│   ├── test.csv             # Test data
│   └── sample_submission.csv # Submission format
│
├── src/                      # Core source modules
│   ├── cleaning.py          # Data cleaning utilities
│   ├── features.py          # Feature engineering
│   ├── model_training.py    # Model training functions
│   ├── evaluation.py        # Model evaluation metrics
│   └── visualization.py     # Plotting and visualization
│
├── notebooks/                # Jupyter notebooks for EDA & experimentation
│   ├── 01_eda_tabular.ipynb # Exploratory analysis (tabular)
│   ├── 01_eda_image.ipynb   # Exploratory analysis (images)
│   ├── 02_modeling_tabular.ipynb
│   └── 02_modeling_image.ipynb
│
├── scripts/                  # Utility scripts
│   ├── preprocess_data.py   # Data preprocessing
│   ├── run_sweep.py         # Hyperparameter sweep (W&B)
│   └── import_check.py      # Dependency verification
│
├── reports/                  # Streamlit dashboard
│   └── app.py               # Interactive model showcase
│
├── BBDD Output/             # Submission files and logs
│
├── main.py                  # Main training pipeline (CatBoost)
├── 4-Catboost.py            # Advanced CatBoost with KNN features
├── Neural_Network.py        # Neural network implementation
├── sweep.yaml               # Weights & Biases sweep configuration
└── requirements.txt         # Python dependencies

---

🚀 Getting Started

Prerequisites

• Python 3.8+
• pip

Installation

1. Clone the repository:

   git clone <repository-url>
   cd Datathon-2025

2. Create a virtual environment (recommended):

   python -m venv venv
   source venv/bin/activate  # Linux/Mac
   venv\Scripts\activate     # Windows

3. Install dependencies:

   pip install -r requirements.txt

4. Add your data: Place your train.csv and test.csv files in the data/ directory.

---

🔧 Usage

Running the Main Pipeline

The main pipeline performs end-to-end training with CatBoost:

python main.py

This will:

1. Load and clean the data
2. Perform feature engineering
3. Apply PCA on image embeddings
4. Train a CatBoost model with k-fold cross-validation
5. Generate predictions and save outputs

Running the Advanced CatBoost Model

For the enhanced CatBoost model with additional KNN features:

python 4-Catboost.py

Running the Neural Network Model

python Neural_Network.py

Hyperparameter Tuning with Weights & Biases

python scripts/run_sweep.py

Configure sweeps in sweep.yaml.

Interactive Dashboard

Launch the Streamlit dashboard for model exploration:

streamlit run reports/app.py

---

📊 Pipeline Overview

1. Data Cleaning (src/cleaning.py)

• Missing value analysis and imputation
• Type conversion and formatting
• Outlier handling

2. Feature Engineering (src/features.py)

• Preprocessing pipeline
• Categorical encoding
• Numerical transformations

3. Embedding Processing

• Parse image embeddings from string format
• Apply PCA dimensionality reduction
• Standard scaling of PCA components

4. Model Training (src/model_training.py)

• CatBoost: Gradient boosting with native categorical support
• Neural Network: Deep learning approach with batch normalization and dropout

5. Evaluation (src/evaluation.py)

• Classification metrics (AUC, precision, recall, F1)
• Regression metrics (RMSE)

6. Visualization (src/visualization.py)

• SHAP summary plots
• Feature importance
• Cross-validation scores

---

🛠️ Key Features

Feature	Description
PCA on Embeddings	Reduces 512-dimensional image embeddings to configurable components
KNN Features	Aggregated target values from nearest neighbors in embedding space
Asymmetric Loss	Custom loss function penalizing overprediction 3x in neural network
Cross-Validation	Robust k-fold validation with out-of-fold predictions
SHAP Explainability	Feature importance and individual prediction explanations
W&B Integration	Experiment tracking and hyperparameter sweeps

---

📈 Output Files

After running the pipeline, outputs are saved to:

File	Description
output/submission.csv	Final predictions for submission
output/oof_predictions.csv	Out-of-fold predictions for stacking
output/feature_importance.csv	Feature importance rankings
BBDD Output/submissions_log.csv	Log of all submissions with scores

---

🧪 Experimentation

Notebooks

Explore the data and experiment with models using the Jupyter notebooks in notebooks/:

• 01_eda_tabular.ipynb - Tabular data exploration
• 01_eda_image.ipynb - Image embedding analysis
• 02_modeling_tabular.ipynb - Model experimentation
• 02_modeling_image.ipynb - Image-based model experiments

---

📦 Dependencies

Key libraries used in this project:

• Data Processing: pandas, numpy
• Machine Learning: scikit-learn, catboost, lightgbm, xgboost
• Deep Learning: tensorflow
• Visualization: matplotlib, seaborn, shap
• Experiment Tracking: wandb
• Web App: streamlit

See requirements.txt for the complete list.

---

📝 Configuration

Model Parameters

Key parameters in main.py:

RANDOM_STATE = 42
N_FOLDS = 5
METRIC = 'auc'  # Use 'rmse' for regression

catboost_params = {
    'learning_rate': 0.03,
    'depth': 6,
    'l2_leaf_reg': 3,
    'min_data_in_leaf': 20,
}

PCA Configuration

N_PCA_COMPONENTS = 10  # Number of principal components
EXPECTED_EMBED_DIMS = 512  # Original embedding dimensions

---

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/new-feature)
3. Commit your changes (git commit -am 'Add new feature')
4. Push to the branch (git push origin feature/new-feature)
5. Open a Pull Request

---

📄 License

This project was created for Datathon 2025.

---

🙏 Acknowledgments

• Datathon Barcelona 2025 organizers
• Weights & Biases for experiment tracking
• SHAP library for model explainability