Skip to content

docs: add MLflow tracking as concrete example of AI provider agnosticism principle #1331

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
atxtechbro wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs: add MLflow tracking as concrete example of AI provider agnosticism principle #1331

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions knowledge/principles/ai-provider-agnosticism.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,3 +73,19 @@ Historical note: `.claude/command-templates/` are intentionally retired; they re
- **[OSE](ose.md)**: External perspective prevents vendor lock-in

This principle ensures AI assistant capabilities remain available even when individual providers experience issues, supporting continuous development workflow with triple redundancy.

## Implementation Example: MLflow Session Tracking

@amazon-q-developer amazon-q-developer Bot Sep 25, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a brief explanation of what MLflow is and why session tracking is relevant to AI provider agnosticism. This would help readers who may not be familiar with MLflow understand the context and relevance of this example.


The MLflow tracking system demonstrates provider agnosticism by extracting actual commands from transcripts rather than maintaining provider-specific patterns:

**Anti-pattern (N×M complexity):**
- Different regex patterns for each provider
- Provider detection logic
- Maintenance burden grows with each new AI assistant

**Correct pattern (provider-agnostic):**
Comment on lines +84 to +86

@amazon-q-developer amazon-q-developer Bot Sep 25, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The anti-pattern description could be more specific to better illustrate the N×M complexity problem. Consider adding concrete examples of what these provider-specific patterns might look like to make the contrast with the provider-agnostic approach clearer.

Suggested change
- Maintenance burden grows with each new AI assistant
**Correct pattern (provider-agnostic):**
**Anti-pattern (N×M complexity):**
- Different regex patterns for each provider (e.g., Claude's `$ command` vs OpenAI's `>>> bash: command`)
- Provider detection logic (`if provider == "claude"` vs `if provider == "openai"`)
- Maintenance burden grows with each new AI assistant (3 providers × 5 patterns = 15 combinations)

- Single parser looks for actual `git`, `gh`, and bash commands
- No provider detection needed
- Works automatically with any AI assistant

Comment on lines +88 to +90

@amazon-q-developer amazon-q-developer Bot Sep 25, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The correct pattern description could benefit from more specific technical details about how the provider-agnostic approach works. Consider mentioning the specific command types that are extracted or the parsing strategy used.

Suggested change
- No provider detection needed
- Works automatically with any AI assistant
**Correct pattern (provider-agnostic):**
- Single parser looks for actual `git`, `gh`, and bash commands using universal patterns
- Extracts commands regardless of formatting (`$ git add`, `>>> bash: git add`, etc.)
- No provider detection needed - focuses on command semantics, not syntax
- Works automatically with any AI assistant that outputs executable commands

See: `tracking/parse_session.py:72-95` - Extracts real commands regardless of AI formatting

@amazon-q-developer amazon-q-developer Bot Sep 25, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The line reference tracking/parse_session.py:72-95 appears to be inaccurate. Looking at the actual file, the provider-agnostic parsing logic spans a much larger range (approximately lines 60-110) and the specific command extraction patterns are around lines 72-95, but the core provider-agnostic approach extends beyond this range. Consider updating the reference to be more accurate or specify the exact functionality being referenced.