Skip to content

Python: expose Parse RPC on PythonRewriteRpc for explicit file lists#7987

Open
knutwannheden wants to merge 1 commit into
mainfrom
expose-parse-in-pythonrewriterpc
Open

Python: expose Parse RPC on PythonRewriteRpc for explicit file lists#7987
knutwannheden wants to merge 1 commit into
mainfrom
expose-parse-in-pythonrewriterpc

Conversation

@knutwannheden

Copy link
Copy Markdown
Contributor

Motivation

PythonRewriteRpc.parseProject(...) walks a whole directory and roots ty (the type resolver) at that directory. That cannot serve a build that wants to parse one unit at a time — an explicit list of files — while resolving first-party imports against a broader workspace root.

The concrete driver is a Bazel-monorepo Python build (in the Moderne CLI) that parses one Bazel target at a time: the target's .py files are an explicit list, but from pkg.other_module import X must resolve against the workspace root. Rooting parseProject at the monorepo root instead would also parse tens of thousands of unrelated files.

The Python server already has a Parse handler that takes explicit inputs and roots ty at relativeTo (and accepts a dependencyPath). It was already consumed over the wire by PythonParser via the inherited RewriteRpc.parse(...), but that path has no way to forward dependencyPath, and there was no client entry point for "parse these files, resolve types against that root." This adds one.

Note: the server's Parse handler is left unchanged — it returns bare id strings, which the inherited RewriteRpc.parse() / PythonParser path already depends on. Changing its return shape would have broken that path, so the new client reuses the existing bare-id contract with a uniform Py.CompilationUnit type hint (exactly as PythonParser already does).

Examples

// Parse a Bazel target's files; resolve first-party imports against the workspace root.
Stream<SourceFile> sources = PythonRewriteRpc.getOrStart().parse(
        List.of(workspaceRoot.resolve("pkg/a.py"), workspaceRoot.resolve("pkg/b.py")),
        workspaceRoot,   // relativeTo: ty is rooted here, so `from pkg.a import X` resolves
        venvPath,        // dependencyPath (optional): third-party type resolution
        ctx);

// Without a dependency env, and with per-parse options:
PythonRewriteRpc.getOrStart().parse(
        files, workspaceRoot, null,
        Map.of("languageLevel", "2.7"), ctx);

Summary

  • Add PythonRewriteRpc.parse(List<Path> inputs, @Nullable Path relativeTo, @Nullable Path dependencyPath, ExecutionContext ctx) plus an overload taking per-parse options. Sends the existing "Parse" RPC and lazily streams each returned id back as a SourceFile via getObject(...), mirroring parseProject's plumbing.
  • Add a Parse request class (mirroring ParseProject) carrying inputs (each { "path": ... }), relativeTo, dependencyPath, and options. The core Parse request is left untouched, so the JS contract is unaffected.
  • parseProject and the Python server's handle_parse are unchanged.

Test plan

  • New PythonRewriteRpcParseTest: parses an explicit list of two files where derived.py imports a class from base.py, with relativeTo set to their common root. Asserts both come back as attributed Py.CompilationUnits (not ParseError) and that the cross-file first-party supertype resolves (the self receiver in Derived is assignable to Base).
  • Existing PythonParserTest (the inherited parse() RPC path) still passes — confirms the Parse handler's bare-id contract is unchanged.
  • licenseFormat clean.

Expose the existing Parse RPC handler on the Java client so callers can
parse an explicit list of Python files with ty rooted at a separate
workspace root, rather than walking a whole directory as parseProject
does. This lets a build parse one unit at a time (e.g. a single Bazel
target) while cross-package first-party imports still resolve against
the monorepo root.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant