Borrower & Application Profile EDA

[RigelAlgebar]

Issue 1 – Borrower & Application Profile EDA with OOP + Functional Pipeline

Goal
Build an EDA toolkit focused on the borrower/application profile and how it relates to loan_status.

You must use:

• OOP: create a class that encapsulates borrower-related EDA.
• Functional programming: use functions as data (dict/list of callables) to build a small EDA pipeline.

---

Columns in scope

You are responsible for the following columns:

Borrower / application profile

• id
• member_id
• emp_title
• emp_length
• home_ownership
• annual_inc
• annual_inc_joint
• verification_status
• verification_status_joint
• zip_code
• addr_state
• purpose
• title
• desc
• issue_d
• pymnt_plan
• policy_code
• url

Shared target

• loan_status (used for default-rate calculations)

---

Files to edit

• src/eda_borrower.py
• notebooks/eda_borrower_demo.ipynb

---

1. Implement the BorrowerProfileEDA class

In src/eda_borrower.py, create and implement the following:

import pandas as pd
from typing import Dict, Any, List, Callable

BORROWER_COLS = [
    "id", "member_id",
    "emp_title", "emp_length",
    "home_ownership",
    "annual_inc", "annual_inc_joint",
    "verification_status", "verification_status_joint",
    "zip_code", "addr_state",
    "purpose", "title", "desc",
    "issue_d", "pymnt_plan", "policy_code",
    "url",
]

class BorrowerProfileEDA:
    def __init__(self, df: pd.DataFrame, target_col: str = "loan_status"):
        """
        Store the full DataFrame and the name of the target column.
        """
        self.df = df
        self.target_col = target_col

    def structure_summary(self) -> pd.DataFrame:
        """
        Return a DataFrame with one row per column in BORROWER_COLS:
        - column: column name
        - dtype: pandas dtype
        - n_missing: number of missing values
        - missing_pct: percentage of missing values
        - n_unique: number of unique values
        """
        ...

    def income_summary(self) -> pd.DataFrame:
        """
        Return basic stats (count, mean, std, min, max, quartiles)
        for:
        - annual_inc
        - annual_inc_joint

        Use df[["annual_inc", "annual_inc_joint"]].describe().T
        or equivalent.
        """
        ...

    def categorical_freqs(self, max_levels: int = 10) -> Dict[str, pd.Series]:
        """
        For important categorical borrower columns (e.g. home_ownership,
        addr_state, purpose), return a dict:

            {
              "home_ownership": Series of top levels,
              "addr_state": Series of top levels,
              ...
            }

        Each Series should be the result of value_counts().head(max_levels).
        """
        ...

    def default_rate_by_category(self, col: str) -> pd.Series:
        """
        For a given categorical column (e.g. 'home_ownership' or 'purpose'),
        compute the default rate per category.

        Default rate = mean of self.target_col for each category.
        Return a pandas Series indexed by category, with values in [0, 1].
        """
        ...

You are free to decide which categorical columns to include in categorical_freqs, but it should at least cover:

• home_ownership
• addr_state
• purpose

---

2. Add a functional EDA pipeline

Still in src/eda_borrower.py, define a small functional pipeline that uses functions as data:

def borrower_eda_steps(eda: BorrowerProfileEDA) -> Dict[str, Callable[[], Any]]:
    """
    Return a dict mapping step names to zero-argument callables.

    Each callable should run ONE piece of EDA when called.
    Example keys (you can adjust names if you like):
        - "structure"
        - "income"
        - "freqs"
        - "default_by_home_ownership"
        - "default_by_purpose"
    """
    ...


def run_borrower_eda_pipeline(eda: BorrowerProfileEDA) -> Dict[str, Any]:
    """
    Run all steps from borrower_eda_steps(eda).

    - Get the dict: step_name -> function
    - Iterate over items
    - Call each function
    - Collect the outputs in a dict: step_name -> result

    This function should clearly show functional programming:
    we store functions in a dict, then loop and call them.
    """
    ...

Example idea (you don’t have to copy exactly):

def borrower_eda_steps(eda: BorrowerProfileEDA) -> Dict[str, Callable[[], Any]]:
    return {
        "structure": eda.structure_summary,
        "income": eda.income_summary,
        "freqs": lambda: eda.categorical_freqs(max_levels=10),
        "default_by_home_ownership": lambda: eda.default_rate_by_category("home_ownership"),
        "default_by_purpose": lambda: eda.default_rate_by_category("purpose"),
    }

---

3. Create the demo notebook

In notebooks/eda_borrower_demo.ipynb:

1. Load the dataset:

   import pandas as pd
   from src.eda_borrower import BorrowerProfileEDA, run_borrower_eda_pipeline
   
   df = pd.read_csv("data/loan_sample.csv")  # or the correct path in your repo
   

2. Instantiate the EDA class:

   eda = BorrowerProfileEDA(df, target_col="loan_status")
   

3. Run the pipeline and inspect the results:

   report = run_borrower_eda_pipeline(eda)
   

4. Display at least:

   report["structure"]          # table of borrower column structure
   report["income"]             # income stats
   report["freqs"]              # categorical frequencies
   report["default_by_home_ownership"]  # default rate by home_ownership
   report["default_by_purpose"]         # default rate by purpose
   

You can add markdown cells explaining what each result means in plain language (e.g., class imbalance, missingness, etc.).

---

Acceptance Criteria ✅

• BorrowerProfileEDA:

  • Initializes correctly with a DataFrame.
  • structure_summary() returns a DataFrame with the requested columns/metrics.
  • income_summary() returns a DataFrame with stats for annual_inc and annual_inc_joint.
  • categorical_freqs() returns a dict of Series with top categories.
  • default_rate_by_category(col) returns a Series of default rates per category.

• Functional pipeline:

  • borrower_eda_steps(eda) returns a dict of callables.
  • run_borrower_eda_pipeline(eda) iterates over that dict, calls each function, and returns a dict of results.

• Notebook:

  • Runs top-to-bottom without errors.
  • Shows the structure summary, income summary, categorical frequencies, and default-rate-by-category analysis.
  • Contains only EDA (no model training).