A/B Testing & Conversion Funnel Analysis

Analyse Olist e-commerce conversion funnels and simulate A/B tests with bootstrap confidence intervals and Bonferroni correction.

Status

Active

Features

• Olist order funnel analysis — full conversion funnel from order creation to delivery
• Monthly trends — funnel volumes broken down by month to spot seasonality
• Payment/category segmentation — funnel completion rates by payment method and product category
• Delivery vs satisfaction — scatter plot of delivery delay against review score
• A/B simulation — synthetic A/A and A/B tests with known ground-truth effects
• Bootstrap confidence intervals — non-parametric resampling (10 000 iterations) for robust inference
• Bonferroni correction — multiple-comparison adjustment demonstrated via 20 simultaneous A/A tests

Architecture

Script	Dependency	Purpose
funnel_analysis.py	PostgreSQL 16+ (Olist dataset)	Queries database, builds 6 funnel/satisfaction charts
ab_test_simulator.py	Standalone (synthetic data)	Runs 4 A/B test scenarios, produces 4 charts

Both scripts output PNG charts to output/.

Requirements

• Python 3.10+
• PostgreSQL 16+ (only for funnel_analysis.py)
• Packages listed in requirements.txt

Setup

git clone https://github.com/Tabasiarash/ab-testing-funnel.git
cd ab-testing-funnel
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

# For funnel analysis only — load Olist database
# See https://github.com/Tabasiarash/ecommerce-sales-intelligence
# Then edit the DB_URL in funnel_analysis.py (line 28) or set env var

Configuration

Hardcoded credential (tech debt): DB_URL is hardcoded in funnel_analysis.py:28 as postgresql://analyst:analyst_pass@localhost:5432/olist. This should be moved to an environment variable. A .env.example file is provided as a reference.

Usage

# Standalone A/B test simulator (no DB needed)
python ab_test_simulator.py

# Funnel analysis (requires running PostgreSQL with Olist)
python funnel_analysis.py

All charts are written to the output/ directory as PNG files.

Deployment

Not applicable — these are single-run analysis scripts intended for local execution or notebook-style exploration.

Development

• Adding a new funnel query: Open funnel_analysis.py, write a new query function (see query_funnel_overall for the pattern), call it from main(), and add a plotting step.
• Adding a new A/B scenario: Open ab_test_simulator.py, add a new scenario function following the existing pattern (generate synthetic data, run test, plot), then invoke it from main().
• Styling: Both scripts use matplotlib with the Agg backend and seaborn-style theme via plt.rcParams.

Known limitations

• Database credentials are hardcoded in funnel_analysis.py:28 (should use environment variables)
• Funnel analysis requires the full Olist dataset loaded into PostgreSQL — no fallback to cached data
• data/ and output/ directories exist but are empty (output is generated at runtime)
• No docker-compose.yml is provided for the Olist database
• A/B simulator uses synthetic data — results are pedagogical, not derived from real experiments

License

MIT