-
-
Notifications
You must be signed in to change notification settings - Fork 98
Expand file tree
/
Copy pathllms-full.txt
More file actions
144 lines (121 loc) · 9.66 KB
/
Copy pathllms-full.txt
File metadata and controls
144 lines (121 loc) · 9.66 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
# XActions
> The complete X/Twitter automation toolkit. Scrapers, an MCP server for AI agents, a CLI, browser console scripts and a browser extension. No X API key, no monthly fee, Apache-2.0. By nichxbt.
## Overview
XActions automates X/Twitter without official API access. It combines 95 browser-paste console scripts, HTTP and Puppeteer scrapers, an MCP server with 152 tools, a CLI with 56 commands, 50 agent skills, and a full-stack Express.js dashboard backed by PostgreSQL.
Two lanes lead into the same engine, and picking the right one matters for an agent: shell out to the CLI for a handful of reads, load the MCP server for long sessions and writes. AGENTS.md is the single source of truth on that choice and on the repository layout.
## Architecture
Three runtime contexts. Code that is correct in one is broken in another:
1. **Browser scripts**: IIFE scripts pasted into the DevTools console on x.com. DOM APIs and `sessionStorage` only, no Node.js.
2. **Node.js library / CLI / MCP**: runs locally or on a server. HTTP-first, with Puppeteer as the fallback for surfaces that need a real browser. Entry points: `src/cli/index.js`, `src/mcp/server.js`.
3. **API server**: Express.js backend at `api/server.js`. PostgreSQL via Prisma, Bull and Redis for the job queue, Socket.io for realtime.
## Tech Stack
- **Runtime**: Node.js >= 20 (CI runs the suite on 20, 22 and 24), ESM (`"type": "module"`)
- **Backend**: Express.js, Helmet, CORS, rate limiting, Morgan
- **Database**: PostgreSQL via Prisma ORM (`prisma/schema.prisma`)
- **Job Queue**: Bull + Redis
- **Browser Automation**: Puppeteer + puppeteer-extra-plugin-stealth
- **Testing**: Vitest 4.x (config: `vitest.config.js`); the suite is offline and runs in CI on Node 20, 22 and 24
- **MCP**: `@modelcontextprotocol/sdk`, stdio and Streamable HTTP transports
- **Payments**: Stripe + x402 crypto
- **Realtime**: Socket.io
## Project Structure
```
src/ Core library
cli/ CLI commands (commander.js): index.js plus commands/
mcp/ MCP server, tool groups, draft-approval gate, action caps
scrapers/ Scrapers: twitter/ (http/ and browser), bluesky/, mastodon/, threads/
client/ HTTP-only Twitter client (no Puppeteer)
automation/ Browser automation scripts (paste core.js first)
agents/ Thought leader agent, persona engine
analytics/ Engagement analytics
streaming/ Real-time tweet, follower and mention streams, plus the live pipeline
portability/ X archive import, export, diff, migrate
notifications/ Email, Slack, Discord, Telegram and signed webhook channels
plugins/ Plugin system
ai/ LLM integrations
a2a/ Agent-to-Agent protocol
workflows/ Multi-step automation
spaces/ AI voice agent for live X Spaces
utils/ Shared utilities
api/ Express.js backend
server.js Main entry
routes/ 40 REST route modules
services/ Business logic, job queue, payments
middleware/ Auth, rate limiting, error handling
realtime/ Socket.io handlers
scripts/ 95 browser console scripts, plus build and maintenance scripts
dashboard/ Static HTML frontend
extension/ Chrome and Edge extension (Manifest V3)
skills/ 50 Agent Skills (skills/*/SKILL.md)
tests/ Vitest tests
types/ TypeScript declarations (index.d.ts)
prisma/ Database schema + migrations
docs/agents/ DOM selectors, browser-script patterns, contributing guide
python/ xeepy, the Python twin of the library
xspace-agents/ The Spaces agent monorepo
```
## Key Features
- **Reading without an account**: profiles, timelines, threads, media and hashtags come from X's internal GraphQL API with a guest token. No login, no Chromium download.
- **Query-ID auto-discovery**: GraphQL query IDs are read from x.com's own JavaScript bundles and cached, so a rotated ID self-heals instead of turning every read into a 404 (`src/scrapers/twitter/http/queryIds.js`).
- **Request signing**: reads and writes carry an `x-client-transaction-id` header computed the way x.com's client computes it (`src/scrapers/twitter/http/transactionId.js`).
- **Cookie import**: `xactions login --from-browser` reads x.com cookies from a locally installed Chrome, Chromium, Brave, Edge, Arc or Firefox profile; `--cookies-file` imports a Netscape `cookies.txt`, a Cookie-Editor or EditThisCookie JSON export, a Playwright or Puppeteer `storageState`, or a raw cookie string.
- **Account pool**: several sessions in a SQLite database, each with its own proxy and per-operation rate-limit window read from X's own `x-rate-limit-*` headers. A pooled client rotates on a 429 and locks an account on a 401 (`src/scrapers/twitter/http/accountPool.js`).
- **Resumable scrapes**: a checkpoint file written after every page lets a killed 50,000-follower scrape restart from its last cursor instead of page one (`src/scrapers/twitter/http/checkpoint.js`).
- **Unfollow management**: mass unfollow, unfollow non-followers, unfollower detection, smart unfollow after a grace period.
- **Growth automation**: auto-like, auto-follow, keyword follow, follow engagers, and sweeps that engage every post on a profile or every result of a search.
- **Content posting**: tweets, threads, polls, replies, quotes, reposts, scheduled posts, DMs.
- **Daily action caps**: every MCP write is charged against a rolling 24-hour per-account budget stored on disk, so the cap survives a restart and a call that would exceed it is refused before anything reaches X (`src/mcp/action-caps.js`).
- **Draft-approval gate**: with `XACTIONS_MCP_REQUIRE_APPROVAL=1` every write tool becomes a draft a human releases, from the MCP tools or from `xactions drafts` (`src/mcp/drafts.js`).
- **MCP tool groups**: `XACTIONS_MCP_TOOLS` and `XACTIONS_MCP_EXCLUDE` filter the advertised tool list by group, and a filtered tool is neither advertised nor callable (`src/mcp/tool-groups.js`).
- **MCP transports**: stdio by default; `--http` serves Streamable HTTP on `/mcp`, with optional bearer auth via `XACTIONS_MCP_TOKEN`.
- **`.mcpb` bundle**: a one-drag install for Claude Desktop, built by `scripts/build-mcpb.mjs` and attached to each release.
- **Live event streaming**: x.com's own event pipeline delivered as a chunked newline-delimited JSON response, with mid-session subscription changes and automatic reconnect. It is not a WebSocket; the endpoint never answers with a 101 (`src/streaming/livePipeline.js`).
- **Signed webhooks**: outbound deliveries carry an HMAC-SHA256 signature, a timestamp, an event type and a stable delivery id, with three retries and a replayable delivery log (`src/notifications/webhook.js`).
- **X archive import**: read the official GDPR export zip for counts, date ranges and top hashtags, write it out as JSON, CSV, Markdown and an HTML viewer, or migrate it to Bluesky or Mastodon (`src/portability/twitter-archive.js`).
- **Agent skills**: 50 skills under `skills/`, installable into Claude Code, Cursor, Codex, Windsurf or the current project with `xactions skills install`.
- **Agent output contract**: `--compact` prints one record per line as tab-separated `key=value` pairs, `--fields` narrows that to named columns, and `--json` prints the full structured object.
- **Analytics**: engagement metrics, best time to post, audience overlap, competitor intelligence, reputation reports.
- **Cross-platform**: Bluesky, Mastodon and Threads scrapers behind one normalised interface.
- **A2A protocol**: Agent-to-Agent communication server.
- **Spaces**: discover, scrape and join Spaces, including an AI voice agent that listens and speaks.
- **Browser extension**: Chrome and Edge, Manifest V3. There is no Firefox build.
## Database Models (Prisma)
- `User` (auth via X session cookies, credits, admin flags)
- `Subscription` (Stripe billing tiers)
- `Operation` (automation job tracking: type, status, progress, result)
- `Payment` / `CryptoPayment` (Stripe and x402 payments)
- `JobQueue` (Bull job persistence)
- `License` (license management)
- `AccountSnapshot` / `FollowerSnapshot` / `FollowerChange` (follower monitoring)
- `UnfollowerSchedule` (scheduled unfollower checks)
## npm Package Exports
```
xactions -> ./src/index.js (main)
xactions/scrapers -> ./src/scrapers/index.js
xactions/scrapers/twitter -> ./src/scrapers/twitter/index.js
xactions/scrapers/twitter/http -> ./src/scrapers/twitter/http/index.js
xactions/scrapers/bluesky -> ./src/scrapers/bluesky/index.js
xactions/scrapers/mastodon -> ./src/scrapers/mastodon/index.js
xactions/scrapers/threads -> ./src/scrapers/threads/index.js
xactions/streaming -> ./src/streaming/index.js
xactions/analytics -> ./src/analytics/index.js
xactions/plugins -> ./src/plugins/index.js
xactions/mcp -> ./src/mcp/server.js
xactions/cli -> ./src/cli/index.js
xactions/client -> ./src/client/index.js
xactions/spaces -> ./src/spaces/agent.js
xactions/portability -> ./src/portability/index.js
```
## CLI Binaries
- `xactions` -> CLI tool (`src/cli/index.js`)
- `xactions-mcp` -> MCP server (`src/mcp/server.js`)
- `xactions-agent` -> Thought leader agent (`src/agents/thoughtLeaderAgent.js`)
## Repository
- URL: https://github.com/nirholas/XActions
- Website: https://xactions.app
- Language: JavaScript (ESM)
- License: Apache-2.0
- Author: nirholas (@nichxbt)
- Node.js >= 20 required
## Contributing
Contributions are welcome. See CONTRIBUTING.md for the three commands to run before opening a PR, and AGENTS.md for how the repository is laid out.