Email-Sentiment-Analysis-

Analysis of sentiments among mails of a broker insurance company

How to start:

Clone the repo and follow the following steps:

Environement creation and activation

uv sync
source ./venv/bin/activate

To be able to use the uv kernel in a jupyter notebook, run:

uv run python -m ipykernel install --user --name=email-sentiment-venv

Then you'll find the kernel in jupyter kernel

Data loading

Create a .env file at the root of the project with the following variables:

AWS_ACCESS_KEY_ID="..."
AWS_SECRET_ACCESS_KEY="..."
AWS_SESSION_TOKEN= "..."
AWS_BUCKET_NAME="nlp-data"
AWS_USERNAME="..." #ex: kbourbon

These credentials can be found in your SSPCloud account under My Account → Storage Connection. Note that AWS_SESSION_TOKEN expires periodically and must be refreshed.

Functions understanding

Please look at the toy_example.ipynb to understand the use of each function defined in the repertory

BERTopic churn signal pipeline

The topic modelling code is now organized around reusable classes:

• EmailPreprocessor: extracts xxobj / xxcont, removes email noise, flags and polite formulas, then optionally lemmatizes with spaCy.
• EmailBERTopicPipeline: trains BERTopic with one of the presets fine, medium or broad.
• TopicAnalyzer: compares churn/non-churn topic representation, creates simple human labels, samples emails and plots topic wordclouds.
• TopicFeatureBuilder: creates client-level topic features for churn modelling, optionally limited to selected topics.
• ChurnClassifier: trains a baseline Random Forest on those topic features.

Example:

import pandas as pd

from src.topic_modeling import EmailPreprocessor, EmailBERTopicPipeline, TopicAnalyzer
from src.features import TopicFeatureBuilder
from src.models import ChurnClassifier

interactions = pd.read_csv("data/apprentissage_interactions_2023_11.csv", sep=";")
portfolio = pd.read_csv("data/apprentissage_portefeuille_2023_11.csv", sep=";")

preprocessor = EmailPreprocessor(use_spacy=True)
emails = preprocessor.prepare_dataframe(interactions)

pipeline = EmailBERTopicPipeline(config="medium")
emails_with_topics = pipeline.fit_transform(emails)

central = portfolio[["client_code", "target_resiliation_6mois"]].merge(
    emails_with_topics,
    on="client_code",
    how="inner",
)

analyzer = TopicAnalyzer(topic_model=pipeline.topic_model)
topic_summary = analyzer.topic_class_summary(central)
labels = analyzer.build_human_labels()
central = analyzer.apply_labels(central, labels)

features = TopicFeatureBuilder().build_features(
    emails_with_topics,
    portfolio_df=portfolio,
    selected_topics=topic_summary.head(30).index.tolist(),
)

clf = ChurnClassifier().fit(features)
metrics = clf.evaluate(features)

The three BERTopic presets can be selected with config="fine", config="medium" or config="broad". Use fine to get many granular topics, broad to force larger clusters, and medium as the first exploration baseline.