Contribution/01 - #21
Merged
Merged
Conversation
…ling - Introduced `ordered_initial_queue` to reorder agents based on `depends_on` relationships, ensuring stable execution order. - Added `topological_queue` function to manage dependencies and detect cycles. - Updated `QueueProcessor` and `OrchestratorBase` to utilize the new ordering logic. - Implemented tests for dependency ordering and validation of execution flow. - Enhanced `ResponseNormalizer` to preserve confidence and cost from wrapped OrkaResponses. - Improved `StreamingOrchestrator` to feed accumulated conversation history to satellites. - Added environment variable handling for concurrency limits and operation timeouts. - Created regression tests for semantic recall and episode outcome recording in the Brain module.
- Refactor agent YAML configurations to include model_url and provider parameters. - Adjust timeout settings for specific agents in self-assessment workflows. - Modify InvariantValidatorAgent to read execution data from the formatted prompt. - Enhance response normalization to preserve node outputs without a response key. - Add unit tests for regression scenarios in invariant execution data handling. - Polish dashboard layout styles for improved UI presentation.
- Updated SkillTransferEngine to clarify scoring weights and semantic matching behavior. - Improved semantic similarity computation by prioritizing task descriptions and domain hints. - Refined loop node configuration to allow shallow merging of YAML configurations, preserving default strategies. - Enhanced LoopScoreExtractor with detailed documentation on score extraction strategies and fallback mechanisms. - Modified ResponseNormalizer to ensure rich fields are preserved and surfaced correctly in normalized payloads. - Removed outdated scoring presets file and added tests to ensure scoring preset weights are well-formed and thresholds are valid. - Hardened FastAPI server with CORS configurations and API key validation for /api/run endpoint. - Added comprehensive unit tests for boolean parser, episode embedding cache, semantic recall, loop score extraction, and server hardening.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @marcosomma, your pull request is larger than the review limit of 150000 diff characters
…r tests The litellm removal dropped aiohttp from the resolved dependency set (it was a transitive dep), but orka/agents/local_llm_agents.py and orka/orchestrator/dry_run/llm_providers.py import it directly — so a fresh `pip install -e .` no longer provided it, collapsing the whole test suite with 148 collection-time ModuleNotFoundError. Declare aiohttp explicitly. Also fix the two mypy errors the Lint job caught: - episode_store._store_embedding: guard self._embedder is None before .encode - llm_agents: annotate _client_kwargs as dict[str, Any] so **kwargs to AsyncOpenAI typechecks And update tests/unit/test_server.py to the hardened run_execution/main contract (reads request.body() + json.loads, binds 127.0.0.1 by default) rather than the pre-hardening request.json()/0.0.0.0 expectations.
tiktoken is imported at module level in local_llm_agents.py, so its absence broke test collection (148 errors) the same way aiohttp did — both were litellm transitives. requests is imported (lazily) by search tools and the plan-validator HTTP client; declare it too for parity.
conftest's autouse mock_external_services fixture patched litellm.completion, which imports litellm — now uninstalled — raising ModuleNotFoundError in 1409 tests on a fresh CI install. It passed locally only because of a stale litellm in the dev venv. No orka code imports litellm anymore, so all litellm mocking (the patch, the async-cleanup neutralization, the MOCK_EXTERNAL_SERVICES entry) is dead and removed. Verified by uninstalling litellm locally then running the full suite: 3181 passed, coverage 85.07%.
…eport check.sh previously hard-coded 7 workflows in an && chain that died on the first failure and produced no summary. Now it globs every *.yml in the folder (so new assessments need no edits), runs them all, and prints a report table + PASS/FAIL/ RAN/ERROR summary. Verdict combines the process exit code with a best-effort parse of the final report agent's overall PASS/FAIL (ignoring the prompt template). Exits non-zero if anything errored or explicitly failed.
marcosomma
force-pushed
the
contribution/01
branch
from
June 28, 2026 19:52
d8328a8 to
b585971
Compare
…candidates GraphScout discovery emitted duplicate paths: _ensure_terminal_path appends a response builder to prefix paths ([A,B] -> [A,B,rb]), colliding with the [A,B,rb] already produced, and recursion re-emits paths via different branches — nothing deduplicated. It also seeded DFS from response-builder nodes, producing nonsensical rb -> X paths that route out of a terminal. - discover_paths now deduplicates candidates by full path tuple (first wins, keeps richest metadata) and bounds the set to max_candidates (default 50, configurable; truncation is logged, never silent) since universal-routing DFS is combinatorial. - _explore_extended_paths skips response-builder start nodes. - 4 real tests build a GraphState and assert: no duplicate paths, response builders only terminal, the cap is honored, and the dedup helper preserves first-occurrence.
Under require_terminal (the default), make_decision committed to the top-scored terminal path the instant any terminal path existed — so the configurable commit_margin / shortlist logic below was dead whenever a terminal path was present (which is almost always, since discovery ensures terminal paths). Now the commit margin applies AMONG terminal paths: commit only when the best terminal path beats the runner-up terminal by the (dynamic) margin, or when it is the sole terminal path; otherwise return a shortlist of the top-k terminal paths. The terminal preference is preserved (still only commits to terminal paths), but the documented knob is now functional. 3 tests: sole terminal commits, clear-winner commits, near-tie shortlists.
…nly no-op dry_run_engine: the three getattr(self.config, 'llm_evaluation_enabled', ...) reads defaulted to True, contradicting GraphScoutConfig's real default (False). In the live path the attribute is always set so behavior was unchanged, but any caller with a bare config would silently flip LLM eval ON and then crash on the missing model config. Default to False so it fails safe to the deterministic evaluator. Test: a config lacking the flag now falls back to deterministic. streaming: the runtime only calls a model when ORKA_STREAMING_HTTP_ENABLE=1. Without it, 'orka streaming run/chat' ran as a silent no-op (structure events, no response text). main() now prints a clear stdout warning in that case. Test drives the real CLI path (mocking only the orchestrator) and asserts the warning appears when HTTP is disabled and not when enabled.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.