Skip to content

mameen/AI_Digest

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

80 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

AI Digest — agentic daily AI news digest

AI Digest logo
Showcase demo: ORIO — agentic architecture for a daily AI news digest
Open Research Intelligence Observatory · Hermes crew + Oreo mascot
Live site · Architecture · Run the POC


From Multi-Agent Crew to Single-Agent-with-Skills: ORIO's evolution preserves the "cute" role mascots from its legacy Hermes multi-agent phase.

Current Direction (July 2026)

Multi-agent orchestration in Hermes is still a valid architecture for ORIO, but in practice it adds coordination latency, prompt overhead, and token cost. The project direction is to preserve Hermes artifacts as a reference implementation while redesigning the active runtime toward a modern single-agent architecture with dynamic context loading and deterministic software boundaries.

Concierge
Concierge
Your single point of contact.
Keeps the standing topic list and schedule; tells GO from "add a topic."
Assembles the kanban board — never fetches sources or writes stories.
SOUL · Roles & responsibilities
Researcher
Researcher
Parallel worker — one target per task
(category, feed cluster, or source bundle).
Fetches pages, extracts facts, returns structured notes with URLs.
Reflects and grounds its own artifact — downstream agents trust that work.
Does not merge across topics or write the digest.
SOUL · Roles & responsibilities
Librarian
Librarian
Fan-in after all researchers finish.
Resolves overlap and maps every article/data point to topics.
Outputs a curated skeleton + knowledge graph — not final prose.
Synthesizer should not redo this curatorial work.
SOUL · Roles & responsibilities
Synthesizer
Synthesizer
Reads the librarian skeleton — overlap and topic mapping are done.
Focuses on format, schema, and writing: takeaway, summary, narratives → digest JSON.
Does not re-fetch, reclassify, or resolve overlap; grounding runs downstream.
SOUL · Roles & responsibilities

AI Digest (codename ORIOOpen Research Intelligence Observatory) turns noisy AI news into a polished daily briefing — HTML archive, heatmaps, leaderboards, and per-run diagnostics. The current production path is Hermes orchestration plus deterministic pipeline rendering (agentic/hermes/ + llm_pipeline/). A Skills-First Architecture track (agentic/single_hermes_agent/) is the active direction and is under implementation.

ORIO runs local LLMs via Ollama — no cloud API keys required. Every published story is deterministically grounded and traceable directly to its source.

How ORIO Evolved

  1. The Claude Skill: It started as a simple daily briefing prompt. This worked until gaps appeared; dedicated tool integration was needed for YouTube chapter parsing, leaderboard crawlers, and structured API feeds.
  2. The Staged LLM Pipeline (llm_pipeline/): We built a structured Python pipeline (ingest $\rightarrow$ enrich $\rightarrow$ validate $\rightarrow$ render) to verify formatting and grounding rules, but the sequential batch execution grew hard to debug and scale.
  3. The Multi-Agent Crew (agentic/hermes/): The staged pipeline was replaced by a four-role crew running on a Hermes kanban board (Concierge, Researcher, Librarian, Synthesizer). The mascot illustrations in the table above represent this phase. While highly decoupled, running multiple agents created high latency, orchestration complexity, and prompt attention competition.
  4. The Agent Skills Refactor (agentic/single_hermes_agent/): The active target architecture (in progress). ORIO is being refactored toward a Single-Agent-with-Skills pattern (inspired by the Agent Skills research paper). Instead of executing multiple conversational subagents, a single host agent will dynamically load modular skills (using progressive disclosure) and route state via a decoupled file message bus, reducing multi-agent orchestration overhead and context rot.

→ Current approach: agentic/single_hermes_agent/docs/ideation.md → Early pipeline exploration: docs/LLM_PIPELINE.md

AI Daily digest — categories, charts, and story cards
Daily digest — categories, leaderboards, charts, provenance on every story

Hermes agent diagnostics — stage waterfall
Agent diagnostics — kanban crew waterfall (research → librarian → synthesizer → render)

Digest archive — heatmap and topics by week
Archive analytics — activity heatmap and topic trends across runs


ORIO workflow (source of truth)

Do not change this graph, role split, or four-output contract without explicit maintainer approval. The diagram below matches agentic/hermes/docs/ARCHITECTURE.md — keep them in sync.

Production end-to-end flow (default GO)

flowchart TB
    GO["GO — Concierge"] --> C["Kanban board"]
    C --> R1["Researcher"]
    C --> R2["Researcher"]
    C --> R3["Researcher"]
    R1 & R2 & R3 --> L["Librarian"]
    L --> S["Synthesizer"]
    S --> P["grounding · validate · render"]
    P --> HTML["reports/<prefix>.html"]
    P --> JSON["reports/<prefix>.json"]
    P --> DH["diagnostics/<prefix>.diagnostics.html"]
    P --> DJ["diagnostics/<prefix>.diagnostics.json"]
Loading

What happens on GO:

  1. Concierge kicks off the run (digest_go / manage.py go) and assembles the kanban board — one Researcher task per topic (default: categories from the best known-good report; override via demo_topics in yaml).
  2. Ingest warm-up (deterministic) fills .preflight/ and .cache/<prefix>/.
  3. Researcher × N work in parallel — one topic each → output.md per task. Each researcher reflects and grounds its own artifact; downstream roles trust that work.
  4. Librarian waits for all researchers — resolves overlap, maps articles and data points to standing topics, dedupes/regroups → librarian.md.
  5. Synthesizer reads that skeleton — format, schema, and prose → digest.json.
  6. Grounding · validate · render — deterministic pipeline (not agent roles) → four files below.
  7. Diagnostics waterfall written for the run.

Four published files (example prefix 20260709120000):

File Path
Report HTML agentic/hermes/reports/<prefix>.html
Report JSON agentic/hermes/reports/<prefix>.json
Diagnostics HTML agentic/hermes/diagnostics/<prefix>.diagnostics.html
Diagnostics JSON agentic/hermes/diagnostics/<prefix>.diagnostics.json
Layer What happens
Orchestration Concierge kanban — research × N → librarian → synthesizer
Concierge control plane GO, board status/abort, assess, digest_open_report, deploy, publish (push only after you approve)
Board topics Auto from best known-good report (most stories); override via demo_topics in yaml
Ingest Warm cache (preflight, Crawl4AI, structured APIs) before researchers run
Workers Hermes LLM profiles with artifact gates per role
Invariants grounding.py + validate.py — deterministic, not agent-judged
Output Four files above + archive index updates on publish

Batch escape hatch (go --pipeline): same enrich_digest as run.py — debug/A/B only. Skips kanban workers entirely.

Agents propose; the pipeline disposes. Links, categories, and provenance tokens are stamped by deterministic code — never trusted from model output alone.

Where to read — role details (who does what)

What you want Where to read
Full role definitions — purpose, responsibilities, tools, what each profile must not do agentic/hermes/system_roles.md
Artifact shapes & handoffsoutput.md, librarian.md, digest.json, gates agentic/hermes/working_agreements.md
Concierge control plane (GO, status, publish) agentic/hermes/admin/config/souls/orio_concierge.md
Worker behavior in kanban SOUL files: orio_researcher.md, orio_librarian.md, orio_synthesizer.md
Showcase one-liner per role (mascot table) This file — four roles above

Split: system_roles.md = who and orchestration. working_agreements.md = what each role produces and which tools it may call.

Where to read — flow chart & runbook

What you want Where to read
High-level agent flow (this page) Production end-to-end flow above
Detailed E2E — numbered steps, ingest, kanban artifacts, approved design agentic/hermes/docs/ARCHITECTURE.md
How to run it agentic/hermes/POC.md
Admin commands + digest-tools agentic/hermes/admin/README.md

Fastest path: this README (story + diagram) → system_roles.md (each profile) → ARCHITECTURE.md (full pipeline + paths).

Quick commands

# Bootstrap (once)
python agentic/hermes/admin/manage.py bootstrap

# Full production run (agentic kanban — default)
python agentic/hermes/admin/manage.py go --start 2026-07-09 --history 10 --fresh

# Batch run.py parity (escape hatch only)
python agentic/hermes/admin/manage.py go --pipeline --start 2026-07-09

# Diagnostics waterfall for a run
python agentic/hermes/admin/manage.py diagnostics --prefix 20260707182407

# Publish to GitHub Pages
python scripts/deploy_app.py --agentic-hermes --one-day 20260707182407 --not-dry-run

Tests: python run_tests.py — real fixtures, no mocks (see AGENTS.md).


Documentation canon

Single source of truth: this file (README.md) — especially ORIO workflow (diagram, four outputs, role split).

Everything under agentic/hermes/ extends it. If anything conflicts, README wins. Legacy staged-pipeline notes live under docs/LLM_PIPELINE.md and llm_pipeline/ — not the product story.

More docs

Topic Doc
Slack front desk agentic/hermes/slack.md
Early staged pipeline (legacy) docs/LLM_PIPELINE.md
Agent onboarding (contributors) .agents/onboarding/

Role, workflow, and architecture pointers are in Where to read above.

Hermes profiles

Each role maps to a profile in the Hermes dashboard — seeded from hermes_roles.yaml via manage.py setup. Production runs on a dedicated server; the repo holds the bootstrap config.

Hermes dashboard — concierge, researcher, librarian, synthesizer profiles


Acknowledgments

Author: Ameen Demiry · Portfolio · GitHub

Editorial inspiration: the daily briefing format is inspired by theAIsearch — adapted here as a local-first, agentic, auditable pipeline rather than a broadcast show.

Agent platform: Hermes Agent by Nous Research — kanban orchestration, profiles, and tooling that this digest builds on. Docs · GitHub

Hermes Agent

Role mascots (Concierge, Researcher, Librarian, Synthesizer) are original artwork for this project. The AI Digest logo and banner are © Ameen Demiry.


Third-party software

AI Digest is released under the MIT License. It depends on and integrates the following open-source projects. Each retains its own license; see the linked project for full terms and attribution requirements.

Project Role in AI Digest License
Hermes Agent Agent orchestration (kanban, profiles, CLI) See upstream repo
Ollama Local LLM inference See upstream
Instructor Structured LLM output (Pydantic) MIT
OpenAI Python SDK Ollama-compatible API client Apache-2.0
Pydantic Schema validation MIT
PyYAML Configuration MIT
Crawl4AI JS-rendered page crawl (leaderboards) See upstream repo
Playwright Browser automation (Crawl4AI) Apache-2.0
yt-dlp YouTube chapter extraction Unlicense
D3.js Archive heatmaps & trend charts (CDN) ISC

Pinned Python versions: requirements-lock.txt.

Redistribution of this software must retain the MIT copyright notice in LICENSE and comply with the licenses of bundled third-party components listed above.


AI Digest · MIT License · Ameen Demiry