feat: improve error resilience, add provider presets, boxed report outtput, and enhanced README - #6
Conversation
…tput, and enhanced README - Fix crash when OpenAI-compatible providers return non-standard error payloads (e.g. OpenRouter 504s missing the choices array) - Add automatic model fallback: when a retryable error (429/504/timeout) occurs, the client transparently retries with a known-good fallback model for the same provider - Add provider presets for 9 backends: anthropic, openai, openrouter, gemini, ollama, groq, together, nvidia, mistral — each with auto- configured base URLs, default models, and fallback models - Make LLM request timeout configurable via --timeout CLI flag (default: 70000ms primary, ~45500ms compressed fallback) - Wrap each finding in a colored box border (red for HIGH, yellow for MED, gray for LOW) for improved terminal readability - Expand README with Quick Start guide, Supported Providers table with setup examples, model fallback docs, and Troubleshooting section - Add tests for undefined choices, OpenRouter error payloads, automatic model fallback, and new provider preset resolution
epic comment - will review soon. |
There was a problem hiding this comment.
Pull request overview
This PR strengthens the tool’s robustness and usability by improving handling of OpenAI-compatible provider quirks, adding provider presets + automatic model fallback, making timeouts configurable, and enhancing the human-readable report/README experience.
Changes:
- Add provider presets for multiple OpenAI-compatible backends and extend provider parsing/validation accordingly.
- Improve OpenAI-compatible client resilience (non-standard error payloads, retryable errors) and add fallback-model retry behavior with tests.
- Add configurable LLM request timeouts and expand README documentation (quick start, provider setup, troubleshooting).
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/report.ts | Adds “boxed” rendering for findings with severity-colored borders/badges. |
| src/llm/types.ts | Expands the Provider union to include additional backends. |
| src/llm/openai.ts | Adds retryable-error detection, fallback model retry, and better handling for missing choices. |
| src/llm/openai.test.ts | Adds coverage for undefined choices and fallback behavior. |
| src/llm/index.ts | Introduces provider presets and improves provider normalization/error messages. |
| src/llm/config.test.ts | Adds tests for new provider names and preset resolution. |
| src/index.ts | Adds --timeout CLI flag and wires it into analysis options. |
| src/analyze.ts | Makes timeouts configurable and derives a compressed-pass timeout from the primary. |
| README.md | Adds Quick Start, Supported Providers, fallback docs, and Troubleshooting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const boxed = contentLines.map((line) => { | ||
| return color("│") + " " + line; | ||
| }); |
| env.OPENAI_MODEL ?? | ||
| preset?.defaultModel ?? | ||
| "gpt-4o-mini"; | ||
| const fallbackModels = preset?.fallbackModels ?? []; |
| readonly label: string; | ||
| readonly model: string; | ||
| private readonly fallbackModels: string[]; | ||
| private activeModel: string; | ||
| private readonly client: OpenAI; |
| known-good fallback model for that provider. You'll see a warning like: | ||
|
|
||
| ``` | ||
| ⚠ Model "openrouter/free" failed, falling back to "google/gemini-2.0-flash-exp:free" |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
still dont wanna check it out @ni5arga |
|
It becomes hard for me to review large PRs and this is a very medium to large sized PR (+/- 600 LoC). I'll recommend you to close this PR and make new PRs in chunks, so it'll be easier for me to review them in a proper manner. I'll just end up running Claude Code review or Copilot on PRs of this size and merge them - that'll just degrade code optimization and I don't want to push code which I haven't manually reviewed. Hope this makes sense. |
Also, tests are failing - you might want to check that. |
I mean it worked on my system when i ran it but def checking it rn |
Also, please try to take this into consideration. Thanks. |
|
Closed in favor of breaking this down into smaller, easier-to-review PRs as requested! I'll be opening them shortly. |
istg vro its good . trust me.