feat: schema extraction, LangChain/LlamaIndex integrations, reputation oracle#9
Merged
Conversation
- Add cheerio + yaml as runtime dependencies - Implement extractStructured() with full CSS selector pipeline: primary selectors → fallback selectors → transformations → derived fields - Add 30-minute schema cache keyed by schemaUrl - Fall back to basic og/meta tag extraction when no schema available - Extracted data includes _schema field indicating which schema matched Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Adds TollwayRetriever (extends BaseRetriever) and TollwayLoader for LangChain pipelines. Agents pass a URL as the query string or configure fixed urls at construction time. Structured schema data is promoted into Document.metadata. Peer dep on @langchain/core >=0.2.0. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Adds TollwayReader implementing BaseReader for LlamaIndex pipelines. loadData(urls[]) fetches via Tollway, promotes structured schema fields into Document metadata. Peer dep on llamaindex >=0.5.0. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Standalone Express server tracking agent DID scores from observations. - GET /v1/:did — public score lookup (neutral 50 for unknowns) - POST /v1/:did/observe — servers record observations (auth via X-Oracle-Key) - 8 observation types with score deltas (+0.5 to -20) - ReputationStore class importable for embedding in existing servers - npx @tollway/reputation to run standalone Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Add integrations/* to root workspaces - Update packages table and roadmap in README (3 new packages checked off) - Update project structure diagram - Add MOLTBOOK_POST.md: 3 targeted Moltbook posts with engagement strategy Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
@tollway/client— implements the long-standing TODO inextractStructured(): full YAML schema parsing + CSS selector extraction viacheerio. Supports primary selectors, fallback selectors, transformations, derived fields, and a 30-min schema cache. Falls back to og/meta tag extraction when no schema is available.@tollway/langchain— new package.TollwayRetriever(extendsBaseRetriever) andTollwayLoaderfor LangChain pipelines. Pass a URL as the query or configure fixedurlsat construction time.@tollway/llamaindex— new package.TollwayReaderimplementingBaseReader.loadData(urls[])fetches via Tollway and returnsDocument[]with structured metadata.@tollway/reputation— new package. Reference reputation oracle server.GET /v1/:didfor score lookup,POST /v1/:did/observefor server-side observation recording. Runnable standalone vianpx @tollway/reputationor embeddable as a library.README— updated packages table, roadmap (3 items checked off), and project structure diagram.MOLTBOOK_POST.md— outreach strategy + 3 drafted posts targetingm/Agent-Builders,m/Philosophy-and-Ponderings, andm/jailbreaksurvivors.Test plan
@tollway/client: verifyextractStructured()fetches and applies a schema fromschemaUrl, falls back to meta tags when schema unavailable@tollway/langchain: smoke testTollwayRetriever.invoke(url)returns aDocumentwith correctpageContentandmetadata@tollway/llamaindex: smoke testTollwayReader.loadData([url])returns aDocument@tollway/reputation:POST /v1/:did/observethenGET /v1/:didreturns updated score;npx @tollway/reputationstarts server on port 3100turbo build🤖 Generated with Claude Code