feat(wren): add Xquik dlt loader

[kriptoburak]

Summary

Adds a bundled load_xquik script to the dlt connector Skill. It loads a bounded X search result into DuckDB and feeds that database into Wren’s existing introspection flow.

What failure does this repair?

This is a feature, not a bug fix. The dlt connector supports REST sources, but users had no runnable Xquik source. The new script configures authentication, contract versioning, cursor pagination, row limits, merge keys, and the DuckDB destination.

How is it tested?

• 27 focused Skill delivery tests pass.
• 1,196 CI-equivalent unit tests pass, with 2 existing skips.
• Ruff formatting and lint checks pass.
• Python 3.11 build and wheel-content checks pass.
• A local dlt 1.30 fixture verified 2 cursor pages, request parameters, headers, and 3 records.
• A full local pipeline wrote and queried xquik_data.tweets in DuckDB.

No real API key or production request was used.

Duplicate check

Reviewed all 261 open issues and 74 open pull requests. No Xquik or dlt-connector item overlaps this work. No open pull request changes these files.

Summary by CodeRabbit

• New Features

  • Added support for loading public X search results from Xquik into DuckDB.
  • Added cursor-based pagination, result limits, and ID-based updates for reliable data loading.
  • Added API-key configuration and a bundled command-line loader for Xquik data.

• Documentation

  • Documented Xquik as a verified data source, including setup, usage, generated tables, and Wren introspection guidance.
  • Updated bundled-resource documentation to include the new loader.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7e3f60bb-9752-4747-8877-651516ddcc96

📥 Commits

Reviewing files that changed from the base of the PR and between c222885 and dfd83e1.

📒 Files selected for processing (1)

• core/wren/tests/unit/test_skills_cli.py

🚧 Files skipped from review as they are similar to previous changes (1)

• core/wren/tests/unit/test_skills_cli.py

Included review availability: Your plan provides up to 10 included reviews per hour; 8 remain after this review.

---

Walkthrough

The dlt connector skill now includes a bundled load_xquik script. The script loads paginated public X search results into DuckDB. Documentation covers setup and usage. Unit tests cover configuration, validation, and script discovery.

Changes

Xquik dlt integration

Layer / File(s)	Summary
Usage documentation core/wren/src/wren/skills_content/dlt-connector/SKILL.md, core/wren/src/wren/skills_content/dlt-connector/references/dlt_sources.md	The skill documents Xquik support, script retrieval, API-key setup, command options, pagination, DuckDB output, and introspection.
Xquik loader implementation core/wren/src/wren/skills_content/dlt-connector/scripts/load_xquik.py	The bundled script validates inputs, builds the dlt REST configuration, requests paginated X search results, merges rows by tweet ID, and runs a DuckDB pipeline.
Loader validation core/wren/tests/unit/test_skills_cli.py	Tests cover isolated script execution, Xquik configuration, invalid inputs, and the bundled script listing.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to dfd83

The PR adds a bounded Xquik-to-DuckDB loader and documents its use without any identified merge-blocking correctness, security, deployment, or runtime risk; it is merge-ready after normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant load_xquik
  participant Xquik
  participant DuckDB
  CLI->>load_xquik: Provide query and result limit
  load_xquik->>Xquik: Request paginated X search results
  Xquik-->>load_xquik: Return search results and cursor
  load_xquik->>DuckDB: Merge rows by tweet ID

Loading

Poem

A rabbit loads tweets through the data stream,
With cursors that hop like a bright little dream.
Into DuckDB the results descend,
IDs merge neatly from start to end.
“X marks the query!” the rabbit cheers.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the primary change: adding an Xquik dlt loader to Wren.
Description check	✅ Passed	The description includes the required summary, testing details, rationale, and duplicate check information.
Docstring Coverage	✅ Passed	Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 2 files.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}