🏥 Text-to-PySUS Conversational Analyzer

Text-to-PySUS is an intelligent, conversational Data Analyst application that allows users to explore and analyze the Brazilian Mortality Information System (SIM - DATASUS) using natural language.

By leveraging the power of LLMs (like Qwen2.5-Coder) and the PySUS library, this tool completely abstracts away the complexity of data extraction and analysis. You just ask questions in plain English (or Portuguese), and the system dynamically generates, safely executes Python code, and presents rich, interactive, and easy-to-understand visualizations.

🚀 Features

• Natural Language Queries: Ask complex demographic and epidemiological questions, and the AI translates them into data insights.
• Autonomous Data Engineering: The system downloads, caches, and filters parquet files from DATASUS automatically.
• Secure Execution Environment: Dynamically generated Python code is securely evaluated inside an isolated Docker sandbox.
• Smart Visualizations:
  • Automatically selects the best chart type (Bar, Horizontal Bar, Line, Pie) based on the user's intent.
  • Applies "Top-N" grouping to prevent cluttered graphics.
  • Beautiful, modern UI aesthetics using Plotly.
• Intelligent Context & Insights:
  • Translates cryptic CID-10 codes (e.g., X954) into human-readable descriptions (e.g., Agressão por disparo de arma de fogo).
  • Generates an executive summary and highlights key insights automatically before displaying charts.
  • Presents a detailed data table below the charts for better context.

📸 Screenshots

Here is a glimpse of the application in action, demonstrating the intelligent chart generation, CID translation, and automated insights:

Example of a natural language query displaying automated insights and an intelligent summary.

Modern horizontal bar chart showing mortality causes, grouping less common occurrences to maintain readability.

The system automatically translates CID codes into their full medical descriptions to provide clear data tables.

Rich UI displaying Top-N analysis with carefully chosen colors and layouts.

Executive summary and deep data insights presented prior to the graphical data.

🛠️ Architecture

The project consists of a streamlined architecture designed for safety and speed:

1. Frontend (Streamlit): Handles the conversational UI and rendering Plotly figures.
2. AI Agent (LangGraph & Ollama): Analyzes the user intent, picks tools, and constructs the appropriate Python code to extract data.
3. Data Sandbox (Docker): Safely executes the AI-generated code. It mounts the downloaded data volumes and uses PySUS to load SIM records.
4. Analytics Engine: Processes the dataframes, translates CIDs, constructs the Plotly figures, and generates insights.

⚙️ Setup & Installation

Prerequisites

• Python 3.9+
• Docker (must be running for the sandbox environment)
• Ollama (running locally with qwen2.5-coder:7b)

1. Clone the repository

git clone https://github.com/NicolasDev-web/Text-to-PySUSConversationalAnalyzer.git
cd Text-to-PySUSConversationalAnalyzer

2. Install dependencies

python -m venv .venv
# Activate the virtual environment
# On Linux/macOS:
source .venv/bin/activate
# On Windows:
.venv\Scripts\activate

pip install -r requirements.txt

3. Build the Sandbox Environment

The sandbox is a vital component for secure code execution.

# On Windows
build_sandbox.bat

# Or manually run:
docker build -t pysus-sandbox -f Dockerfile.sandbox .

4. Run the Application

streamlit run app.py

🧠 How it Works Under the Hood

When a user asks: "What was the main cause of mortality for black women in Ceará in 2023?"

1. The AI identifies the parameters (State: CE, Year: 2023, Filters: Black Women).
2. It generates a Python snippet that utilizes a pre-injected carregar_sim() function.
3. The snippet is executed in the Docker sandbox, where pysus downloads and caches the necessary DataFrames.
4. The Analytics Engine intercepts the results, processes the DataFrame, applies a "Top-N" function to group remaining values into "Others," and translates the CIDs.
5. An interactive Plotly chart, textual summary, and insights are securely returned via a JSON payload and displayed on Streamlit.