v0.12.0

What's Changed

Now supports installation with both Python 3.11 and 3.12.

v0.11.1

What's Changed

🛠 Fixed: Version number is now correctly reported when the package is installed via pip (e.g. from GitHub or PyPI).

Previously, the version fallbacked to 0.0.0 due to reliance on pyproject.toml at runtime.

Now uses importlib.metadata for reliable version access across environments.

v0.11.0

Release Notes — v0.11.0

⚠️ NOTE: There is a known issue with version reporting when installing via pip. Please use v0.11.1 for the fix. ⚠️

🎉 Major Feature Release: Enhanced Validation & Testing Capabilities

We're excited to announce the release of mmm-eval v0.11.0, bringing significant improvements to our Marketing Mix Model validation toolkit. This release introduces advanced testing methodologies and enhanced metric support to help you build more robust and reliable MMMs.

🚀 What's New

✨ New Validation Tests

• Placebo/Falsification Test - Validate your model's robustness by testing against synthetic data where no true relationship exists
• In-Sample Accuracy Validation - Comprehensive accuracy testing using the same metrics as holdout validation for better model assessment

🔢 Enhanced Metrics

• SMAPE (Symmetric Mean Absolute Percentage Error) - Added to both accuracy and cross-validation accuracy tests for more balanced error measurement
• Improved Percentage Display - All percentage-based metrics now display as actual percentages (e.g., 15 instead of 0.15) for better readability

🔧 Platform Improvements

• Meridian Integration Enhancements - Better support for national models and non-spend media metrics
• Streamlined Testing - Meridian testing files are now tracked in the repository, eliminating the need for manual file management
• Robust Timeseries Handling - Improved splitting logic that works seamlessly with complex multi-index datasets

🛠️ Technical Improvements

API Enhancements

• Added fit_and_predict() method to adapter classes for more flexible model workflows
• Improved configuration handling with optional PyMCFitSchema usage
• Enhanced progress tracking and seed control for prediction operations

Data Handling

• Replaced to_dict() methods with more efficient to_df() implementations
• Individual metric results now properly contained within test dataframes
• Better parameter derivation from fit configurations

⚠️ Breaking Changes

• Holdout Split Strategy - Changed from relative to fixed test set size for more consistent validation results
• Percentage Display - All percentage metrics now display as actual percentages rather than decimals

📈 Performance & Reliability

• Fixed bugs in Meridian adapter for national model support
• Improved timeseries splitting robustness
• Enhanced error handling and validation

📚 Documentation

Updated documentation reflects all new features and improvements. Check out our comprehensive guides for implementing the new validation tests.

🔭 Looking Ahead

We're continuing to expand our validation capabilities and platform support. Future releases will focus on:

• Additional MMM platform integrations
• Advanced diagnostic visualizations
• Automated reporting features
• Enhanced statistical testing methodologies

Initial Public Release (v0.7.0)

📦 Release Notes — v0.7.0

🎉 Initial Public Release: mmm-eval, an open source framework for MMM validation

We are excited to announce the first public release of mmm-eval, an open-source library for validating and benchmarking Marketing Mix Models (MMMs) with ease.

This toolkit is designed for marketing scientists, analysts, and model developers who need a standardized and reproducible way to test the accuracy and robustness of MMM implementations.

🚀 Highlights

✅ Supports multiple MMM platforms:

• Google Meridian
• PyMC-Marketing

✅ Standardized validation metrics:

MAPE, holdout tests, and more

✅ Reproducible benchmarking:

Compare model predictions against ground truth and/or synthetic datasets.

✅ Extensible framework:

Plug in your own MMM estimator or validation dataset.

✅ Easy integration:

Works seamlessly with pandas DataFrames provided as input.

🔧 Project Status

This is an early-access public release (v0.7.0), intended as a foundation for community adoption and future contributions.
We aim to support additional MMM libraries and extend metric coverage in upcoming releases.

📚 Resources

📖 Documentation

🐍 PyPI:

Coming soon.

🙌 Contributing

We welcome contributions, bug reports, and feature requests!

📅 Looking Ahead

Planned features for future releases include:

• Support for additional MMM tools (e.g., Robyn)
• More advanced validation diagnostics
• Automated reporting and visualization