Skip to content

Scala: never throw from parseInputs and self-heal compiler core libraries#7980

Open
knutwannheden wants to merge 1 commit into
mainfrom
scalaparser-never-throw-fix-empty-classpath-with
Open

Scala: never throw from parseInputs and self-heal compiler core libraries#7980
knutwannheden wants to merge 1 commit into
mainfrom
scalaparser-never-throw-fix-empty-classpath-with

Conversation

@knutwannheden

Copy link
Copy Markdown
Contributor

Motivation

ScalaParser.parseInputs() could throw instead of producing per-file ParseError source files, which kills entire builds in downstream tools (observed on apache/flink, where a resource build step died mid-build). Two related problems:

  1. parseInputs() could throw. Batch compilation (compilerContext.compileAll(...)) ran outside the per-input try/catch, so any exception from it propagated to the caller and failed the whole parse stream. Inside the bridge, the phase-1 parse loop also had no per-entry handling, so one file that failed to parse aborted the entire batch. Other parsers (e.g. JavaParser, KotlinParser) yield a ParseError for the failing file and keep going.

  2. Parsing A with B types threw MissingCoreLibraryException without the Scala stdlib on the classpath. Parsing a self-type like class Service { self: Logging with Config => } with an empty classpath inside a fat-jar application threw dotty.tools.dotc.MissingCoreLibraryException: Could not find package scala from compiler core libraries (withTypeRestuntpd.makeAndTypeDefinitions.andTypeDenotations.staticRef). Dotty's Run construction needs the Scala stdlib to initialize Definitions; when the caller's classpath lacks it — or overrides dotty's default lookup, as a fat jar does — the bridge fell back to a bare context where with types "fail loudly". This didn't reproduce in the test JVM because dotty finds the stdlib via the JVM classpath there.

Summary

  • ScalaParser.parseInputs() wraps batch compilation in a catch-all; on failure it reports via ctx.getOnError() and degrades to the existing per-file fallback, where each input independently yields a real tree or a ParseError.
  • ScalaCompilerBridge.compileAll catches per-entry parse failures in its phase-1 loop and omits the entry, so the caller's single-file fallback surfaces a ParseError for that file only.
  • When Run construction fails, the bridge retries it with the compiler's own core library jars appended (located via ProtectionDomain, which works inside a fat jar). The retry needs an entirely new compiler context: the failed attempt has already cached the ContextBase's platform classpath, so updating the setting on a fresh of the same base has no effect.
  • The healed Run is used for parsing only — no type attribution. Narrow-classpath callers like ScalaTemplate intentionally avoid type resolution of user types; they only need the initialized ContextBase so with types parse.
  • Extracted configuredContext(...) in the bridge to share compiler context setup between compileAll and parse.

Test plan

  • New test: one unparseable input (stack-overflowing nesting) among good inputs → good inputs parse, bad one comes back as ParseError (previously the error killed the whole stream)
  • New test: a batch failure (source whose first read throws) degrades to per-file parsing instead of throwing out of parseInputs
  • New test: with self-type parses to a real S.CompilationUnit with a classpath containing only a non-Scala jar, which overrides dotty's default stdlib lookup to simulate the fat-jar environment (previously MissingCoreLibraryException)
  • New test: precondition check that the same self-type parses with the full runtime classpath
  • ./gradlew :rewrite-scala:check passes (full module test suite, license check)

…ries

ScalaParser.parseInputs() could throw instead of producing per-file
ParseError source files, killing entire builds in downstream tools:

- Batch compilation ran outside the per-input try/catch, so any exception
  from compileAll propagated to the caller. Now a batch failure degrades
  to per-file parsing, and a per-file parse failure inside the bridge's
  parse loop no longer aborts the whole batch.
- With a classpath lacking the Scala stdlib (e.g. an empty classpath
  inside a fat-jar application), dotty's Run construction failed and
  parsing `A with B` types threw MissingCoreLibraryException. The bridge
  now retries Run construction with the compiler's own core library jars
  (located via ProtectionDomain) appended, using the result for parsing
  only so narrow-classpath callers like ScalaTemplate still get no type
  attribution.
@github-project-automation github-project-automation Bot moved this to In Progress in OpenRewrite Jun 11, 2026
@knutwannheden knutwannheden changed the title Scala: never throw from parseInputs and self-heal compiler core libraries Scala: never throw from parseInputs and self-heal compiler core libraries Jun 11, 2026

@greg-at-moderne greg-at-moderne left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement.

@github-project-automation github-project-automation Bot moved this from In Progress to Ready to Review in OpenRewrite Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Ready to Review

Development

Successfully merging this pull request may close these issues.

2 participants