From fff1889a732a630150fa3e6d23840cd308932738 Mon Sep 17 00:00:00 2001 From: woksin Date: Tue, 28 Jul 2026 20:08:52 +0200 Subject: [PATCH] Compile every generated document and report one the language rejects RoundTrip printed a document, compiled it and reprinted it, and its own comment called it the correctness gate for everything the generator produces. It lived in the spec project, so it only ever ran against hand-written fixtures - the generator running against a real application verified nothing. That is how a command property called Description shipped a document that did not compile: no fixture happened to use that name, and 690 green specs said nothing. Move the capability into the package and read every printed document back through the Screenplay compiler. A rejected document is the generator being wrong rather than anything the source declared, so it is reported at error severity and a host exits non zero. The text is returned regardless so the rejected line can be read. The verifier is held concretely rather than injected. A host that could substitute it could switch it off, and a check nobody runs is the situation this exists to end. SP0024 suppresses it, because a model recovered from source the compiler never accepted describes an application that does not exist. Reprinting and comparing stays a spec-level gate. Compiling is the cheap, decisive check; byte-identical reprint is a stronger claim about information loss and belongs where fixtures are known. Co-Authored-By: Claude Opus 5 --- Documentation/generating-a-screenplay.md | 17 ++++++ Source/DotNET/Screenplay.Specs/RoundTrip.cs | 34 ----------- .../when_emitting/a_library_application.cs | 1 + .../a_projection_using_every_block_kind.cs | 1 + ...rojection_with_an_unmappable_expression.cs | 1 + ...ice_naming_its_members_after_directives.cs | 1 + .../a_slice_that_declares_nothing.cs | 1 + ...slice_whose_screens_are_all_it_declares.cs | 1 + .../given/a_document_the_language_rejects.cs | 49 ++++++++++++++++ .../and_the_document_it_printed_compiles.cs | 34 +++++++++++ ...he_document_it_printed_does_not_compile.cs | 43 ++++++++++++++ .../and_the_source_it_read_did_not_compile.cs | 45 +++++++++++++++ .../Screenplay/ScreenplayDiagnosticCodes.cs | 13 +++++ .../DotNET/Screenplay/ScreenplayGenerator.cs | 57 ++++++++++++++++++- .../Verification/IScreenplayVerifier.cs | 22 +++++++ .../Screenplay/Verification/RoundTrip.cs | 33 +++++++++++ .../Verification}/RoundTripResult.cs | 25 +++++--- .../Verification/ScreenplayVerification.cs | 29 ++++++++++ .../Verification/ScreenplayVerifier.cs | 34 +++++++++++ 19 files changed, 397 insertions(+), 44 deletions(-) delete mode 100644 Source/DotNET/Screenplay.Specs/RoundTrip.cs create mode 100644 Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/given/a_document_the_language_rejects.cs create mode 100644 Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_compiles.cs create mode 100644 Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_does_not_compile.cs create mode 100644 Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_source_it_read_did_not_compile.cs create mode 100644 Source/DotNET/Screenplay/Verification/IScreenplayVerifier.cs create mode 100644 Source/DotNET/Screenplay/Verification/RoundTrip.cs rename Source/DotNET/{Screenplay.Specs => Screenplay/Verification}/RoundTripResult.cs (54%) create mode 100644 Source/DotNET/Screenplay/Verification/ScreenplayVerification.cs create mode 100644 Source/DotNET/Screenplay/Verification/ScreenplayVerifier.cs diff --git a/Documentation/generating-a-screenplay.md b/Documentation/generating-a-screenplay.md index 2d4354a88..021813dd3 100644 --- a/Documentation/generating-a-screenplay.md +++ b/Documentation/generating-a-screenplay.md @@ -50,6 +50,23 @@ transported rather than what it is, so the query was left out (Library.Messaging Diagnostics come in three severities. **Information** means something is worth knowing but the document is complete. **Warning** means something was left out. **Error** means the document should not be trusted at all — the most common cause being source that did not compile. +## The generator checks its own output + +Every diagnostic above names something *your application* declared that the language cannot hold. There is one that names a defect in the generator instead. + +After the document is written, the generator hands it straight back to the Screenplay compiler. If the compiler rejects it, `SP0034` is reported as an error — because a `.play` that does not compile is output nobody can use, and there is no way of writing an application that avoids it. This is not a mode you turn on: it runs on every generation, since the only way a rejected document is ever found is by reading each one back. + +```text +Error SP0034: The generated document did not compile - 1 error(s), the first being +'Invalid description 'description RequestDescription' - expected 'description ""'' +on line 6. That is the generator being wrong rather than anything the source declared, +and the document is returned as it stands so the line can be read (Library) +``` + +The document is still written out, so you can open it at the reported line and see what happened. If you hit this, it is a bug worth [reporting](https://github.com/Cratis/Arc/issues) — include the line, and the C# declaration it came from. + +Source that did not compile (`SP0024`) suppresses `SP0034`. A model recovered from symbols the compiler never accepted describes an application that does not exist, so a poor document made from it is a consequence of the broken build rather than a second, separate defect. Fix the build and generate again. + ## Screens A [vertical slice](./vertical-slices.md) puts the React component that realizes a slice's screen in the same folder as its C#. Roslyn syntax trees carry real file paths, so the generator knows where each slice's source lives and declares a `screen` for every `.tsx` component sitting next to it, named after the file: diff --git a/Source/DotNET/Screenplay.Specs/RoundTrip.cs b/Source/DotNET/Screenplay.Specs/RoundTrip.cs deleted file mode 100644 index 8d2a0b5db..000000000 --- a/Source/DotNET/Screenplay.Specs/RoundTrip.cs +++ /dev/null @@ -1,34 +0,0 @@ -// Copyright (c) Cratis. All rights reserved. -// Licensed under the MIT license. See LICENSE file in the project root for full license information. - -using Cratis.Screenplay; -using Cratis.Screenplay.Printing; -using Cratis.Screenplay.Syntax; - -namespace Cratis.Arc.Screenplay; - -/// -/// Prints a document, compiles the printed source and prints the result again. -/// -/// -/// This is the correctness gate for everything the generator produces. Output that does not compile means the -/// generator is wrong, and output that does not print identically the second time means information was lost. -/// -public static class RoundTrip -{ - /// - /// Runs the round trip for a document. - /// - /// The document to round trip. - /// The result of the round trip. - public static RoundTripResult For(ApplicationSyntax application) - { - var printer = new ScreenplayPrinter(); - var compiler = new ScreenplayCompiler(); - var printed = printer.Print(application); - var compilation = compiler.Compile(printed); - var reprinted = compilation.Value is null ? string.Empty : printer.Print(compilation.Value); - - return new(printed, reprinted, [.. compilation.Diagnostics]); - } -} diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_library_application.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_library_application.cs index 5c3818b2c..6edb472cb 100644 --- a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_library_application.cs +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_library_application.cs @@ -2,6 +2,7 @@ // Licensed under the MIT license. See LICENSE file in the project root for full license information. using Cratis.Arc.Screenplay.Emission; +using Cratis.Arc.Screenplay.Verification; using Cratis.Screenplay.Syntax; namespace Cratis.Arc.Screenplay.for_ScreenplayEmitter.when_emitting; diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_using_every_block_kind.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_using_every_block_kind.cs index 036de1b17..3e28ebb78 100644 --- a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_using_every_block_kind.cs +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_using_every_block_kind.cs @@ -4,6 +4,7 @@ using Cratis.Arc.Screenplay.Emission; using Cratis.Arc.Screenplay.Library; using Cratis.Arc.Screenplay.Model; +using Cratis.Arc.Screenplay.Verification; namespace Cratis.Arc.Screenplay.for_ScreenplayEmitter.when_emitting; diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_with_an_unmappable_expression.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_with_an_unmappable_expression.cs index 9a8cbb4f2..c1f4b5f09 100644 --- a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_with_an_unmappable_expression.cs +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_projection_with_an_unmappable_expression.cs @@ -4,6 +4,7 @@ using Cratis.Arc.Screenplay.Emission; using Cratis.Arc.Screenplay.Library; using Cratis.Arc.Screenplay.Model; +using Cratis.Arc.Screenplay.Verification; namespace Cratis.Arc.Screenplay.for_ScreenplayEmitter.when_emitting; diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_naming_its_members_after_directives.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_naming_its_members_after_directives.cs index be8ae95d8..084bcda56 100644 --- a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_naming_its_members_after_directives.cs +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_naming_its_members_after_directives.cs @@ -4,6 +4,7 @@ using Cratis.Arc.Screenplay.Emission; using Cratis.Arc.Screenplay.Library; using Cratis.Arc.Screenplay.Model; +using Cratis.Arc.Screenplay.Verification; namespace Cratis.Arc.Screenplay.for_ScreenplayEmitter.when_emitting; diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_that_declares_nothing.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_that_declares_nothing.cs index bdeba4ec3..043b4df66 100644 --- a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_that_declares_nothing.cs +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_that_declares_nothing.cs @@ -4,6 +4,7 @@ using Cratis.Arc.Screenplay.Emission; using Cratis.Arc.Screenplay.Library; using Cratis.Arc.Screenplay.Model; +using Cratis.Arc.Screenplay.Verification; namespace Cratis.Arc.Screenplay.for_ScreenplayEmitter.when_emitting; diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_whose_screens_are_all_it_declares.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_whose_screens_are_all_it_declares.cs index 722aa264d..281372a8a 100644 --- a/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_whose_screens_are_all_it_declares.cs +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayEmitter/when_emitting/a_slice_whose_screens_are_all_it_declares.cs @@ -4,6 +4,7 @@ using Cratis.Arc.Screenplay.Emission; using Cratis.Arc.Screenplay.Library; using Cratis.Arc.Screenplay.Model; +using Cratis.Arc.Screenplay.Verification; namespace Cratis.Arc.Screenplay.for_ScreenplayEmitter.when_emitting; diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/given/a_document_the_language_rejects.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/given/a_document_the_language_rejects.cs new file mode 100644 index 000000000..914bde8a0 --- /dev/null +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/given/a_document_the_language_rejects.cs @@ -0,0 +1,49 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Arc.Screenplay.Emission; +using Cratis.Arc.Screenplay.Emission.Naming; +using Cratis.Screenplay.Printing; +using Cratis.Screenplay.Syntax; + +namespace Cratis.Arc.Screenplay.for_ScreenplayGenerator.given; + +/// +/// Stands in for the printer, writing text that was prepared rather than rendered, so that a document the Screenplay +/// compiler rejects can be put in front of the generator. +/// +/// +/// Every way of emitting a rejected document that anyone has found has since been fixed, which is exactly why the +/// check exists - it has to hold for the one nobody has found yet. Replacing the printed text is the smallest seam +/// that produces one: the analysis, the emitter, the syntax tree the emitter builds and the compiler reading the +/// text back are all the real ones, and nothing on the path a host takes is substituted. The text is the defect that +/// shipped, which is a command property called Description written out as the directive of the same name. +/// +public class a_document_the_language_rejects : Specification +{ + /// + /// The text the stand-in printer writes in place of the document. + /// + protected const string Rejected = """ + domain Library + module Library + feature Authors + slice StateChange Registration + command RegisterAuthor + description RequestDescription + """; + + /// + /// The line of the text the Screenplay compiler rejects. + /// + protected const int RejectedLine = 6; + + protected IScreenplayEmitter _emitter; + + void Establish() + { + var printer = Substitute.For(); + printer.Print(Arg.Any()).Returns(Rejected); + _emitter = new ScreenplayEmitter(printer, new ScreenplayNaming()); + } +} diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_compiles.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_compiles.cs new file mode 100644 index 000000000..cb8e4b7f0 --- /dev/null +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_compiles.cs @@ -0,0 +1,34 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Arc.Screenplay.Emission; +using Cratis.Arc.Screenplay.Library; +using Cratis.Screenplay; + +namespace Cratis.Arc.Screenplay.for_ScreenplayGenerator.when_generating; + +/// +/// Reading every generated document back is only worth doing if it stays quiet about the documents that are fine. +/// A check that cries wolf on a real application is a check people learn to ignore, so this generates the whole +/// library - every slice kind, every construct the language holds - and expects nothing said about it. +/// +public class and_the_document_it_printed_compiles : given.a_compilation +{ + ScreenplayGenerator _generator; + ScreenplayGenerationResult _result; + CompilationResult _compiled; + + void Establish() => _generator = new(new given.a_recovered_model(LibraryApplication.Build()), new ScreenplayEmitter()); + + void Because() + { + _result = _generator.Generate(_compilation, new ScreenplayOptions()); + _compiled = new ScreenplayCompiler().Compile(_result.Source); + } + + [Fact] void should_be_generating_a_document_that_really_does_compile() => _compiled.Success.ShouldBeTrue(); + [Fact] void should_be_generating_the_whole_application() => _result.Source.Contains("slice StateChange Registration", StringComparison.Ordinal).ShouldBeTrue(); + [Fact] void should_not_report_that_the_document_did_not_compile() => _result.Diagnostics.Select(_ => _.Code).ShouldNotContain(ScreenplayDiagnosticCodes.DocumentDidNotCompile); + [Fact] void should_report_nothing_at_all() => _result.Diagnostics.ShouldBeEmpty(); + [Fact] void should_be_successful() => _result.IsSuccess.ShouldBeTrue(); +} diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_does_not_compile.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_does_not_compile.cs new file mode 100644 index 000000000..757dba1f5 --- /dev/null +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_document_it_printed_does_not_compile.cs @@ -0,0 +1,43 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Arc.Screenplay.Analysis; +using Cratis.Screenplay; +using Microsoft.CodeAnalysis; + +namespace Cratis.Arc.Screenplay.for_ScreenplayGenerator.when_generating; + +/// +/// A document the Screenplay compiler rejects is output nobody can use, and no way of writing an application avoids +/// it - it is the generator being wrong. Nothing but reading every generated document back finds one, which is how +/// a property named after a directive shipped with 690 green specifications behind it. So every generation compiles +/// what it printed, reports a rejected document as an error - which is what makes a host exit non zero - and hands +/// the text back regardless, because the line that was rejected is the only place to start looking. +/// +public class and_the_document_it_printed_does_not_compile : given.a_document_the_language_rejects +{ + Compilation _compilation; + ScreenplayGenerator _generator; + ScreenplayGenerationResult _result; + + void Establish() + { + _compilation = Analyzed.Compile(("Library/Authors/Registration/Registration.cs", "namespace Library.Authors.Registration;")); + _generator = new(new ApplicationModelAnalyzer(), _emitter); + } + + void Because() => _result = _generator.Generate(_compilation, new ScreenplayOptions()); + + ScreenplayDiagnostic Reported => _result.Diagnostics.First(_ => _.Code == ScreenplayDiagnosticCodes.DocumentDidNotCompile); + + [Fact] void should_be_printing_a_document_the_language_really_rejects() => new ScreenplayCompiler().Compile(Rejected).Success.ShouldBeFalse(); + [Fact] void should_report_that_the_document_did_not_compile() => _result.Diagnostics.Select(_ => _.Code).ShouldContain(ScreenplayDiagnosticCodes.DocumentDidNotCompile); + [Fact] void should_report_it_as_an_error() => Reported.Severity.ShouldEqual(ScreenplayDiagnosticSeverity.Error); + [Fact] void should_report_it_only_once() => _result.Diagnostics.Count(_ => _.Code == ScreenplayDiagnosticCodes.DocumentDidNotCompile).ShouldEqual(1); + [Fact] void should_quote_the_first_thing_the_compiler_said() => Reported.Message.Contains("description RequestDescription", StringComparison.Ordinal).ShouldBeTrue(); + [Fact] void should_say_which_line_was_rejected() => Reported.Message.Contains($"on line {RejectedLine}", StringComparison.Ordinal).ShouldBeTrue(); + [Fact] void should_say_it_is_the_generator_that_is_wrong() => Reported.Message.Contains("the generator being wrong", StringComparison.Ordinal).ShouldBeTrue(); + [Fact] void should_still_return_the_document() => _result.Source.ShouldEqual(Rejected); + [Fact] void should_not_be_successful() => _result.IsSuccess.ShouldBeFalse(); + [Fact] void should_not_claim_the_source_it_read_failed_to_compile() => _result.Diagnostics.Select(_ => _.Code).ShouldNotContain(ScreenplayDiagnosticCodes.SourceDidNotCompile); +} diff --git a/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_source_it_read_did_not_compile.cs b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_source_it_read_did_not_compile.cs new file mode 100644 index 000000000..19845d8bc --- /dev/null +++ b/Source/DotNET/Screenplay.Specs/for_ScreenplayGenerator/when_generating/and_the_source_it_read_did_not_compile.cs @@ -0,0 +1,45 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Arc.Screenplay.Analysis; +using Cratis.Screenplay; +using Microsoft.CodeAnalysis; + +namespace Cratis.Arc.Screenplay.for_ScreenplayGenerator.when_generating; + +/// +/// Source the compiler never accepted still yields symbols, and a model recovered from them describes an application +/// that does not exist - so a document made from it being poor is the consequence already reported rather than a +/// second defect. Saying both would send whoever reads the diagnostics hunting for a bug in the generator when the +/// thing to fix is the build, which is why the same suppression the empty document already gets applies here. +/// +public class and_the_source_it_read_did_not_compile : given.a_document_the_language_rejects +{ + const string Source = """ + namespace Library.Authors.Registration; + + public record RegisterAuthor(string Name) + { + public string Handle() => ThisDoesNotExist; + } + """; + + Compilation _compilation; + ScreenplayGenerator _generator; + ScreenplayGenerationResult _result; + + void Establish() + { + _compilation = Analyzed.Compile((Analyzed.SlicePath, Source)); + _generator = new(new ApplicationModelAnalyzer(), _emitter); + } + + void Because() => _result = _generator.Generate(_compilation, new ScreenplayOptions()); + + [Fact] void should_be_generating_from_source_that_really_does_not_compile() => Analyzed.ErrorsIn((Analyzed.SlicePath, Source)).ShouldNotBeEmpty(); + [Fact] void should_be_printing_a_document_the_language_really_rejects() => new ScreenplayCompiler().Compile(Rejected).Success.ShouldBeFalse(); + [Fact] void should_report_that_the_source_did_not_compile() => _result.Diagnostics.Select(_ => _.Code).ShouldContain(ScreenplayDiagnosticCodes.SourceDidNotCompile); + [Fact] void should_not_report_the_document_on_top_of_it() => _result.Diagnostics.Select(_ => _.Code).ShouldNotContain(ScreenplayDiagnosticCodes.DocumentDidNotCompile); + [Fact] void should_still_return_the_document() => _result.Source.ShouldEqual(Rejected); + [Fact] void should_not_be_successful() => _result.IsSuccess.ShouldBeFalse(); +} diff --git a/Source/DotNET/Screenplay/ScreenplayDiagnosticCodes.cs b/Source/DotNET/Screenplay/ScreenplayDiagnosticCodes.cs index 9ebab52cc..f7dc7eddc 100644 --- a/Source/DotNET/Screenplay/ScreenplayDiagnosticCodes.cs +++ b/Source/DotNET/Screenplay/ScreenplayDiagnosticCodes.cs @@ -211,4 +211,17 @@ public static class ScreenplayDiagnosticCodes /// number is written as it stands and what it names is left unsaid. /// public const string UnnamedEnumerationValue = "SP0033"; + + /// + /// The document that was generated does not compile, which is the generator being wrong rather than the source. + /// + /// + /// Every other code here names something the application declared that the language has no counterpart for, + /// which is a gap the reader can see and work around. This one names a defect in the generator - a document the + /// Screenplay compiler rejects is output nobody can use, and no way of writing an application avoids it. It + /// exists because such a document is only ever found by reading every generated one back: a property named + /// after a directive shipped once precisely because no fixture happened to use that name. The text is returned + /// as it stands so the line that was rejected can be read. + /// + public const string DocumentDidNotCompile = "SP0034"; } diff --git a/Source/DotNET/Screenplay/ScreenplayGenerator.cs b/Source/DotNET/Screenplay/ScreenplayGenerator.cs index 3e78ffacb..2140740c1 100644 --- a/Source/DotNET/Screenplay/ScreenplayGenerator.cs +++ b/Source/DotNET/Screenplay/ScreenplayGenerator.cs @@ -3,6 +3,7 @@ using Cratis.Arc.Screenplay.Analysis; using Cratis.Arc.Screenplay.Emission; +using Cratis.Arc.Screenplay.Verification; using Microsoft.CodeAnalysis; namespace Cratis.Arc.Screenplay; @@ -15,10 +16,14 @@ namespace Cratis.Arc.Screenplay; /// /// The package ships no dependency injection of its own, so a consumer that just wants to generate a document says /// new ScreenplayGenerator() and gets everything wired. The constructor taking collaborators exists for -/// specifications and for hosts that want to substitute one half. +/// specifications and for hosts that want to substitute one half. Reading the generated document back is not one of +/// the halves - a host that could substitute it could switch it off, and a check nobody runs is the situation this +/// exists to end. /// public class ScreenplayGenerator(IApplicationModelAnalyzer analyzer, IScreenplayEmitter emitter) : IScreenplayGenerator { + readonly ScreenplayVerifier _verifier = new(); + /// /// Initializes a new instance of the class with everything wired up. /// @@ -38,6 +43,54 @@ public ScreenplayGenerationResult Generate(Compilation compilation, ScreenplayOp var analysis = analyzer.Analyze(compilation, resolved); var emission = emitter.Emit(analysis.Model, resolved); - return new(emission.Source, analysis.Model, [.. analysis.Diagnostics, .. emission.Diagnostics]); + var diagnostics = new ScreenplayDiagnostics(); + diagnostics.AddRange(analysis.Diagnostics); + diagnostics.AddRange(emission.Diagnostics); + ReportDocumentThatDoesNotCompile(emission.Source, analysis.Diagnostics, diagnostics, compilation.AssemblyName); + + return new(emission.Source, analysis.Model, diagnostics.All); + } + + /// + /// Reads the printed document back and reports one the Screenplay compiler rejects. + /// + /// The printed document. + /// What analysis reported, which says whether the source it read compiled. + /// The diagnostics to report to. + /// Where to report against. + /// + /// Everything else reported names something the application declared that the language cannot hold. This names + /// the generator being wrong, which is why it runs on every generation rather than on request - the only way a + /// rejected document is ever found is by reading each one back, and a document nobody happened to look at is + /// exactly how one shipped. Reporting it as an error is what makes a host exit non zero, and the text is + /// returned regardless so the line that was rejected can be read - the same bargain + /// strikes. That code also suppresses this one, the + /// way it suppresses : a model recovered from symbols + /// the compiler never accepted describes an application that does not exist, so a poor document made from it is + /// the consequence already reported rather than a second defect. + /// + void ReportDocumentThatDoesNotCompile( + string source, + IEnumerable analyzed, + ScreenplayDiagnostics diagnostics, + string? location) + { + if (analyzed.Any(_ => _.Code == ScreenplayDiagnosticCodes.SourceDidNotCompile)) + { + return; + } + + var verification = _verifier.Verify(source); + if (verification.Compiles) + { + return; + } + + var first = verification.Errors[0]; + + diagnostics.Error( + ScreenplayDiagnosticCodes.DocumentDidNotCompile, + $"The generated document did not compile - {verification.Errors.Count} error(s), the first being '{first.Message}' on line {first.Location.Line}. That is the generator being wrong rather than anything the source declared, and the document is returned as it stands so the line can be read", + location); } } diff --git a/Source/DotNET/Screenplay/Verification/IScreenplayVerifier.cs b/Source/DotNET/Screenplay/Verification/IScreenplayVerifier.cs new file mode 100644 index 000000000..451fed405 --- /dev/null +++ b/Source/DotNET/Screenplay/Verification/IScreenplayVerifier.cs @@ -0,0 +1,22 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +namespace Cratis.Arc.Screenplay.Verification; + +/// +/// Defines a system that reads a printed Screenplay document back to establish that it is a document at all. +/// +/// +/// This is the half of the generator that trusts nothing the other two halves said. It never sees a compilation or +/// a model - only the text and the Screenplay language - which is what makes a document nobody can use a fact +/// rather than an opinion. +/// +public interface IScreenplayVerifier +{ + /// + /// Compiles a printed Screenplay document. + /// + /// The printed .play text to compile. + /// The . + ScreenplayVerification Verify(string source); +} diff --git a/Source/DotNET/Screenplay/Verification/RoundTrip.cs b/Source/DotNET/Screenplay/Verification/RoundTrip.cs new file mode 100644 index 000000000..49b2fd27a --- /dev/null +++ b/Source/DotNET/Screenplay/Verification/RoundTrip.cs @@ -0,0 +1,33 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Screenplay.Printing; +using Cratis.Screenplay.Syntax; + +namespace Cratis.Arc.Screenplay.Verification; + +/// +/// Prints a document, compiles the printed text and prints the result again. +/// +/// +/// Every generation compiles what it printed, because output the language rejects is output nobody can use. This +/// goes one step further and prints the recompiled document a second time, which asks whether anything was lost on +/// the way through the text - a stronger claim, and one that can only be made where a document is small enough to +/// be read and compared. That is why generating uses only the compiling half of this and specifying uses all of it. +/// +public static class RoundTrip +{ + /// + /// Runs the round trip for a document. + /// + /// The document to round trip. + /// The result of the round trip. + public static RoundTripResult For(ApplicationSyntax application) + { + var printer = new ScreenplayPrinter(); + var verification = new ScreenplayVerifier().Verify(printer.Print(application)); + var reprinted = verification.Application is null ? string.Empty : printer.Print(verification.Application); + + return new(verification, reprinted); + } +} diff --git a/Source/DotNET/Screenplay.Specs/RoundTripResult.cs b/Source/DotNET/Screenplay/Verification/RoundTripResult.cs similarity index 54% rename from Source/DotNET/Screenplay.Specs/RoundTripResult.cs rename to Source/DotNET/Screenplay/Verification/RoundTripResult.cs index 9ef2bfaf1..973b90032 100644 --- a/Source/DotNET/Screenplay.Specs/RoundTripResult.cs +++ b/Source/DotNET/Screenplay/Verification/RoundTripResult.cs @@ -3,25 +3,34 @@ using Cratis.Screenplay.Diagnostics; -namespace Cratis.Arc.Screenplay; +namespace Cratis.Arc.Screenplay.Verification; /// /// Represents the outcome of printing, compiling and reprinting a document. /// -/// The printed source. -/// The source printed from the recompiled document. -/// Everything the compiler reported. -public record RoundTripResult(string Printed, string Reprinted, IEnumerable Diagnostics) +/// What reading the printed text back produced. +/// The text printed from the recompiled document. +public record RoundTripResult(ScreenplayVerification Verification, string Reprinted) { + /// + /// Gets the printed text. + /// + public string Printed => Verification.Source; + + /// + /// Gets everything the compiler reported. + /// + public IReadOnlyList Diagnostics => Verification.Diagnostics; + /// /// Gets everything the compiler reported as an error. /// - public IEnumerable Errors => Diagnostics.Where(_ => _.Severity == DiagnosticSeverity.Error); + public IReadOnlyList Errors => Verification.Errors; /// - /// Gets a value indicating whether the printed source compiles. + /// Gets a value indicating whether the printed text compiles. /// - public bool Compiles => !Errors.Any(); + public bool Compiles => Verification.Compiles; /// /// Gets a value indicating whether the second print is identical to the first. diff --git a/Source/DotNET/Screenplay/Verification/ScreenplayVerification.cs b/Source/DotNET/Screenplay/Verification/ScreenplayVerification.cs new file mode 100644 index 000000000..fe7d1169a --- /dev/null +++ b/Source/DotNET/Screenplay/Verification/ScreenplayVerification.cs @@ -0,0 +1,29 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Screenplay.Diagnostics; +using Cratis.Screenplay.Syntax; + +namespace Cratis.Arc.Screenplay.Verification; + +/// +/// Represents everything reading a printed Screenplay document back produced. +/// +/// The printed .play text that was compiled. +/// The document the text compiled to, null when it did not compile at all. +/// Everything the Screenplay compiler reported about the text. +public record ScreenplayVerification( + string Source, + ApplicationSyntax? Application, + IReadOnlyList Diagnostics) +{ + /// + /// Gets everything the Screenplay compiler reported as an error. + /// + public IReadOnlyList Errors => [.. Diagnostics.Where(_ => _.Severity == DiagnosticSeverity.Error)]; + + /// + /// Gets a value indicating whether the printed text compiles. + /// + public bool Compiles => Errors.Count == 0; +} diff --git a/Source/DotNET/Screenplay/Verification/ScreenplayVerifier.cs b/Source/DotNET/Screenplay/Verification/ScreenplayVerifier.cs new file mode 100644 index 000000000..6c688fc71 --- /dev/null +++ b/Source/DotNET/Screenplay/Verification/ScreenplayVerifier.cs @@ -0,0 +1,34 @@ +// Copyright (c) Cratis. All rights reserved. +// Licensed under the MIT license. See LICENSE file in the project root for full license information. + +using Cratis.Screenplay; + +namespace Cratis.Arc.Screenplay.Verification; + +/// +/// Represents an implementation of . +/// +/// The the printed document is read back with. +/// +/// The compiler used is the one the language ships, not a second reading of the grammar written here. Anything +/// less would prove that the document satisfies this package rather than that it satisfies Screenplay, which is +/// the only claim worth making. +/// +public class ScreenplayVerifier(IScreenplayCompiler compiler) : IScreenplayVerifier +{ + /// + /// Initializes a new instance of the class using the compiler the language ships. + /// + public ScreenplayVerifier() + : this(new ScreenplayCompiler()) + { + } + + /// + public ScreenplayVerification Verify(string source) + { + var compilation = compiler.Compile(source); + + return new(source, compilation.Value, [.. compilation.Diagnostics]); + } +}