Skip to content

Mission: Find Zava's Hidden Product-Quality Crisis #33

Description

@a-villarruel

Mission

Mission: Find Zava's Hidden Product-Quality Crisis

Business Objective

The objective of this investigation was to identify the strongest evidence-backed product risk cluster by combining structured operational data with unstructured customer feedback.

Following the mission requirements, I investigated sales performance, support activity, customer satisfaction, support conversations, product reviews, and semantic similarity across customer documents using SQL MCP tools and SQL Server vector search.

Rather than relying on a single metric, the investigation correlated quantitative and qualitative evidence to determine which product should be prioritized for immediate business intervention.

Investigation Question

Can an AI agent identify a product-quality issue that is not immediately visible through traditional sales reporting by correlating:

  • Sales performance
  • Customer support activity
  • Customer satisfaction
  • Product reviews
  • Support conversations
  • Semantic similarity across customer documents

The goal was not only to identify the highest-risk product, but also to explain its underlying root cause using multiple independent sources of evidence.

This investigation intentionally emphasized the agentic journey rather than a single prompt, demonstrating how iterative reasoning, SQL exploration, and semantic retrieval can be combined to solve a real business problem.

Harness and model

GitHub Copilot (Agent Mode) powered by GPT-5.5

Turn-by-turn journey

1. Environment Discovery

Prompt

Describe the PromptathonDb schema and available SQL MCP tools.

Agent Actions

  • Explored the database schema and relationships.
  • Identified the available business entities.
  • Documented how sales, products, support, customers, and documents were connected.
  • Identified SQL MCP capabilities, including vector similarity search.

Outcome

Established a complete understanding of the available business data before starting the investigation.


2. Initial Business Exploration

Prompt

Analyze overall business performance across products by combining sales, support tickets, customer satisfaction, and available review data.

Agent Actions

Generated SQL analysis comparing:

  • Sales volume
  • Revenue
  • Support ticket volume
  • Customer satisfaction
  • Product reviews

Outcome

Several products initially appeared to be potential candidates. After comparing all available evidence, the Premium Short Sleeve Men's Top (SKU ZCPTM-SS-M-BW) emerged as the strongest candidate due to its combination of:

  • High sales volume
  • High support workload
  • Low customer satisfaction
  • Multiple negative customer reviews

3. Root Cause Investigation

Prompt

Investigate the root cause behind the issues affecting the identified product.

Agent Actions

  • Queried SupportTickets.
  • Analyzed SupportChats transcripts.
  • Grouped recurring customer complaints into common themes.

Outcome

The dominant complaint pattern was consistently identified as:

Smart connectivity fails after washing.

The issue appeared repeatedly across multiple independent customer conversations.


4. Semantic Validation using SQL Vector Search

Prompt

Use the SQL MCP vector search tool to identify semantically similar customer documents.

Agent Actions

Executed the SQL MCP tool FindSimilarDocsByDocId to retrieve semantically related reviews and support conversations.

Outcome

The similarity search revealed multiple independent documents describing nearly identical failures:

  • Smart fabric disconnects after washing
  • Application loses sensor pairing
  • Connectivity cannot be restored

The repeated semantic patterns confirmed that the issue was not isolated to a single customer or support ticket.


5. Business Assessment

The investigation combined structured and unstructured evidence to conclude that the product suffers from a recurring durability issue affecting the smart-fabric connectivity layer.

Evidence originated from:

  • Sales transactions
  • Customer support activity
  • Customer satisfaction
  • Support conversations
  • Product reviews
  • SQL Server vector similarity search

Independent evidence consistently converged toward the same business conclusion.

The identified risk cluster fully satisfied the mission criteria by combining meaningful sales impact, elevated support burden, low customer satisfaction, recurring complaint themes, and semantically similar negative customer documents.


6. Artifacts Produced

The following artifacts were generated during the investigation and are attached to this submission:

  • Product_Investigation_Notebook.ipynb
  • Executive_Summary.md
  • Architecture.md (Mermaid)
  • Root cause analysis documented within the Executive Summary and investigation notes
  • Investigation documentation captured throughout the submission
  • SQL MCP tool-call evidence

7. Business Recommendation

Based on the available evidence, Zava should prioritize an engineering quality investigation of the Premium Short Sleeve Men's Top smart connectivity components.

Recommended actions include:

  • Perform a root-cause analysis of the smart-fabric connectivity layer, particularly its durability after laundering.
  • Validate the issue across related product variants.
  • Monitor return rates and support tickets following corrective actions.
  • Coordinate Product Engineering, Quality Assurance, and Customer Support to confirm the scope and prioritize remediation.

Given the product's meaningful sales volume and the consistency of customer complaints across support conversations, reviews, and semantic similarity analysis, addressing this issue is likely to improve customer satisfaction while reducing future support demand.


8. Lessons Learned

This investigation demonstrated that GitHub Copilot Agent Mode delivers the greatest value when treated as an analytical partner rather than a SQL generator.

The workflow evolved naturally through:

Schema Discovery

Business Exploration

Hypothesis Generation

Evidence Collection

Semantic Validation

Business Recommendation

The investigation was intentionally iterative rather than linear. Initial exploration identified several candidate products with elevated support activity. Additional analysis of customer satisfaction, support conversations, and semantic similarity was required to eliminate weaker hypotheses and confidently isolate the product-quality issue affecting the Premium Short Sleeve Men's Top.

Each iteration increased confidence in the findings while separating observations, assumptions, and validated evidence.

Supporting Evidence

The following screenshots illustrate key milestones of the investigation, including environment discovery, business analysis, SQL MCP tool execution, and the generated solution artifacts. Together they document the end-to-end agentic workflow followed throughout the investigation.

Image Image Image Image Image

Completion

  • Yes, the agent completed the mission or goal.
  • No, the agent did not complete the mission or goal.

Bonus work

Beyond completing the requested mission, I intentionally approached the Promptathon as a real business investigation rather than a SQL exercise.

Additional work included:

  • Produced a fully reproducible Jupyter Notebook documenting the complete investigation.
  • Generated a Mermaid architecture/data-flow diagram describing the analytical workflow.
  • Produced an Executive Summary aligned with the mission's executive brief requirement, focusing on business impact, evidence, recommendations, assumptions, and limitations.
  • Validated findings using SQL Server vector search through the SQL MCP tool FindSimilarDocsByDocId.
  • Correlated structured operational data with unstructured customer conversations and reviews.
  • Explicitly documented assumptions, evidence, and dataset limitations separately.
  • Preserved SQL MCP tool-call evidence to support the investigation.

Most importantly, I adopted an iterative agentic workflow. Rather than relying on a single prompt, I progressively guided the agent through exploration, hypothesis generation, validation, and refinement. Each iteration increased confidence in the findings and demonstrated how AI can augment—rather than replace—the analytical reasoning process.

The final conclusion emerged only after independent evidence from sales, support, customer satisfaction, chat transcripts, product reviews, and vector similarity search consistently converged on the same root cause, rather than relying on a single analytical signal.

This investigation demonstrates how an agentic workflow can transform raw business data into actionable insights through iterative reasoning, structured analytics, and semantic search.

Key Takeaways

  • AI agents are most effective when used as collaborative analytical partners rather than SQL generators.
  • Combining structured SQL analytics with semantic retrieval provides richer business insights than traditional reporting alone.
  • SQL Server vector search enabled the investigation to validate recurring customer issues across multiple independent sources.
  • The end-to-end workflow demonstrated how AI can support evidence-based decision-making through iterative reasoning, structured analysis, and semantic search.

Attached Deliverables

The following artifacts accompany this submission and provide a complete, reproducible record of the investigation:

Product_Investigation_Notebook.ipynb – Reproducible notebook containing the end-to-end analytical workflow, SQL queries, intermediate findings, and final conclusions.
Executive_Summary.md – Executive-level summary of the investigation, business impact, evidence, recommendations, assumptions, and limitations.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions