Add optional autodoc integration to convert docstring Example blocks to interactive py-repl

[cursor]

Summary

Add optional autodoc integration so Sphinx projects can automatically convert docstring Example: doctest blocks (rendered via sphinx.ext.autodoc) into interactive py-repl widgets with replay UX — without duplicating examples into RST files or stripping expected outputs from Python source.

This is needed for downstream projects like chrizzFTD/naming, where PR #29 already migrates Overview tutorial blocks to .. py-repl::, but autoclass API pages still render static doctest code from docstrings (e.g. naming.Name, File, Pipe, PipeFile in naming/__init__.py).

Motivation

Today, sphinx-pyrepl-web only exposes the .. py-repl:: directive. Autodoc docstring examples render as static literal blocks (often styled by sphinx_toggleprompt). There is no bridge between autodoc docstrings and the pyrepl-web replay pipeline.

Goals:

• Keep full doctests (inputs and expected outputs) in Python source for IDE hover docs and optional doctest runs
• Strip expected outputs at Sphinx build time only, producing input-only replay scripts (same transformation naming PR #29 applied manually to Overview)
• Reuse existing replay file generation, strip_doctest_prompts(), and <py-repl> HTML emission

Non-goals:

• Converting non-Example: code blocks
• Modifying consumer project docstrings to remove outputs
• Duplicating examples into RST below .. autoclass::

Proposed approach

Add an opt-in autodoc-process-docstring hook (default off) that:

1. Detects configurable section headers (default: Example, Examples)
2. Extracts doctest input lines from each section (stripping expected output)
3. Replaces the static doctest block with an embedded .. py-repl:: directive in the processed docstring lines
4. Lets the existing PyRepl directive handle replay script writing and JS loading

Architecture

Docstring (unchanged in .py, full doctest)
  → autodoc-process-docstring hook
  → inject .. py-repl:: directive (input-only body)
  → existing PyRepl directive
  → replay script + <py-repl replay-src="...">
  → pyrepl-web replay UX

Concrete implementation tasks

1. Refactor shared replay builder

Extract replay registration and HTML generation from PyRepl.run() in sphinx_pyrepl_web/__init__.py into a shared helper, e.g.:

def build_pyrepl_html(env, docname, body_lines, options: dict) -> str:
    ...

Used by both:

• PyRepl directive (existing behavior)
• autodoc hook (new)

The helper should centralize:

• strip_doctest_prompts() application
• replay script naming ({docname}-{counter}.py)
• writing to env.metadata[docname][REPLAY_FILES_KEY]
• setting env.metadata[docname]["pyrepl"] = True
• emitting <py-repl ... replay-src="_static/pyrepl/..."> HTML

2. Add extract_doctest_inputs() utility

New function, e.g. in sphinx_pyrepl_web/doctest.py:

def extract_doctest_inputs(lines: list[str]) -> list[str]:
    """Return input-only doctest lines, dropping expected output."""

Rules:

• Keep lines starting with >>> or ... (including bare ... terminators — already handled downstream by strip_doctest_prompts() in v0.1.1)
• Drop expected output lines (non-prompt lines between inputs): '{base}', {}, Name("..."), multi-line list output, etc.
• Drop traceback output blocks: lines starting with Traceback through the error message; the live REPL shows the real traceback

Reference patterns from naming docstrings:

• naming/__init__.py — Name, File, Pipe, PipeFile class examples
• PipeFile includes a ValueError traceback block that must become input-only

3. Add autodoc hook module

New file: sphinx_pyrepl_web/autodoc.py

Register on autodoc-process-docstring when pyrepl_autodoc = True.

Handler logic:

1. Scan docstring lines for section headers matching pyrepl_autodoc_sections
2. Within each section, collect contiguous indented doctest lines
3. Run extract_doctest_inputs()
4. Replace the doctest block with:

Example:

.. py-repl::
   :packages: naming
   :no-banner:

   >>> from naming import Name
   >>> class MyName(Name):
   ...     config = dict(base=r'\w+')
   ...
   >>> n = MyName()
   ...

5. Apply default directive options from pyrepl_autodoc_options
6. Leave docstrings without matching sections untouched

Wire up in setup():

if app.config.pyrepl_autodoc:
    app.connect("autodoc-process-docstring", process_autodoc_docstring)

Only register if autodoc is loaded (or document that autodoc must be in extensions).

4. New Sphinx config values

Config	Default	Description
pyrepl_autodoc	False	Enable autodoc docstring → REPL conversion
pyrepl_autodoc_packages	None	Value for :packages: on generated directives
pyrepl_autodoc_sections	["Example", "Examples"]	Section titles to convert
pyrepl_autodoc_options	{}	Default directive flags, e.g. {"no-banner": True}

Optional enhancement: auto-derive packages from documented object name (naming.Name → naming) when pyrepl_autodoc_packages is unset.

Example consumer conf.py:

pyrepl_autodoc = True
pyrepl_autodoc_packages = "naming"
pyrepl_autodoc_sections = ["Example"]
pyrepl_autodoc_options = {"no-banner": True}

5. Tests

Add unit/integration tests for:

• extract_doctest_inputs() with naming-like patterns (output lines, tracebacks, bare ..., multi-line list output)
• autodoc hook: given sample docstring lines, assert correct .. py-repl:: injection shape and indentation
• regression: docstrings without Example: sections are unchanged
• replay scripts generated from autodoc examples compile cleanly (python -m py_compile)

Acceptance criteria

When enabled in a consumer project with autodoc:

• Class/function docstring Example: blocks render as <py-repl packages="..." replay-src="_static/pyrepl/..."> elements
• Replay scripts contain input-only lines (no expected output)
• Python source docstrings remain unchanged (full doctests preserved)
• Existing .. py-repl:: directive behavior is unchanged (backward compatible)
• Feature is opt-in (pyrepl_autodoc = False by default)
• Documented in README with config examples

Downstream validation (naming)

After release (target: v0.2.0), naming will:

1. Bump dependency to sphinx-pyrepl-web>=0.2.0
2. Enable config in docs/source/conf.py
3. Validate on Name.html, File.html, Pipe.html, PipeFile.html:
   • 4 <py-repl> elements on autoclass pages
   • 4 replay scripts under _static/pyrepl/
   • Overview page still has its 9 existing REPLs

Related links

• Consumer migration PR: Add interactive REPL to Overview docs naming#29
• Example docstrings: https://github.com/chrizzFTD/naming/blob/develop/naming/__init__.py
• Current directive implementation: sphinx_pyrepl_web/__init__.py (PyRepl, strip_doctest_prompts, replay file pipeline)

Suggested release

Ship as v0.2.0 (new feature, backward compatible, opt-in).

[chrizzFTD]

Done in #16