NYC Taxi Mobility Analytics

Free book: Data Engineering: From Zero to Production & Interview Ready

244 pages, by the author of this repo. One order followed through a fictional Indian e-commerce company, with each tool arriving at the moment the story needs it. Every figure in it was measured rather than asserted, and the code behind each one is in the Field Kit.

Read it free, no card →

End-to-end open-source mobility analytics using NYC Taxi data, Python EDA, SQL, and Streamlit.

An end-to-end data analytics project using NYC Taxi & Limousine Commission Yellow Taxi trip records. The project covers data download, cleaning, EDA, SQL analysis, and a Streamlit dashboard.

Portfolio Snapshot

Area	What this project proves
Mobility analytics	Demand, zones, revenue, routes, airport behavior, and tips.
Python	Large Parquet ingestion, cleaning, feature engineering, EDA.
SQL	Repeatable warehouse-style questions using SQLite.
Dashboarding	Interactive Streamlit + Plotly dashboard over 2.7M cleaned trips.
Communication	Workflow docs, data dictionary, interview guide, and dashboard walkthrough.

Preview

Business Goal

Help a city mobility or marketplace analytics team answer:

• When is demand highest?
• Which pickup zones generate the most trips?
• Which boroughs generate the most revenue?
• How do airport routes behave?
• How does tipping differ by area?

Dataset

Source: NYC Taxi & Limousine Commission Trip Record Data.

This project uses January 2024 Yellow Taxi trips plus the official taxi zone lookup table.

Architecture

flowchart LR
    A[NYC TLC Parquet + Zone CSV] --> B[Download]
    B --> C[Raw Data]
    C --> D[Python Cleaning + Feature Engineering]
    D --> E[Processed Parquet]
    D --> F[SQLite DB]
    E --> G[EDA Charts]
    F --> H[SQL Result CSVs]
    E --> I[Streamlit Dashboard]

Loading

flowchart TB
    subgraph Data["Data Layer"]
        A[NYC TLC Parquet]
        B[Taxi Zone Lookup]
        C[Clean Trip Model]
        D[SQLite DB]
    end

    subgraph Analysis["Analysis Layer"]
        E[EDA Summary]
        F[Matplotlib + Seaborn Charts]
        G[SQL Result CSVs]
    end

    subgraph Product["Product Layer"]
        H[Streamlit Dashboard]
        I[Interview Story]
    end

    A --> C
    B --> C
    C --> D
    C --> E
    C --> F
    D --> G
    C --> H
    G --> H
    E --> I

Loading

Run End to End

git clone https://github.com/analyticsdurgesh/NYC-Taxi-Mobility-Analytics.git
cd NYC-Taxi-Mobility-Analytics
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
bash scripts/run_all.sh
streamlit run dashboard/app.py --server.port 8502

Open:

http://localhost:8502

Project Steps

1. src/download_data.py downloads Yellow Taxi Parquet data and the official zone lookup CSV.
2. src/prepare_data.py cleans trip records, removes operational outliers, joins pickup/dropoff zones, writes Parquet, and builds SQLite.
3. src/eda.py uses pandas, seaborn, and matplotlib to generate exploratory charts and an EDA summary.
4. src/sql_analysis.py runs business SQL queries and exports result CSVs.
5. dashboard/app.py creates an interactive Streamlit + Plotly dashboard.

Documentation

• End-to-End Usage Guide
• Workflow
• Data Dictionary
• Dashboard Walkthrough
• Interview Guide
• SQL Queries

SQL Questions Answered

• Hourly demand and revenue.
• Top pickup zones.
• Airport route behavior.
• Tipping by pickup borough.
• Payment type mix.

Dashboard Views

• Demand by hour.
• Demand heatmap by day and hour.
• Top pickup zones and routes.
• Revenue and tip behavior by borough.
• SQL insights tables.

Generated Analytical Assets

Asset	Path
EDA summary	outputs/eda_summary.md
Hourly demand chart	outputs/figures/hourly_taxi_demand.png
Borough trip volume chart	outputs/figures/borough_trip_volume.png
Demand heatmap	outputs/figures/taxi_demand_heatmap.png
SQL results	outputs/sql_results/*.csv

Interview Story

I built this as a city mobility analytics product. The pipeline starts from public NYC TLC trip data, joins official taxi zones, cleans operational outliers, creates trip-level features, answers business questions with SQL, and serves demand and revenue insights through an interactive dashboard. It demonstrates large public-data analytics, feature engineering, SQL, visualization, and stakeholder-ready storytelling.

Folder Map

data/raw/               raw downloaded data, ignored by git
data/processed/         parquet and sqlite analytics DB, ignored by git
outputs/figures/        generated EDA images
outputs/sql_results/    exported SQL result CSVs
src/                    pipeline scripts
dashboard/              Streamlit dashboard