diff --git a/DIATAXIS_REVIEW.md b/DIATAXIS_REVIEW.md new file mode 100644 index 000000000..3066956ea --- /dev/null +++ b/DIATAXIS_REVIEW.md @@ -0,0 +1,221 @@ +# Diátaxis Review — docs-pact-io-diataxis-refactor + +**Scope:** `/Users/matthew.fellows/development/public/docs-pact-io-diataxis-refactor/website/docs/` +**Date:** 2026-03-20 +**Reviewer:** Claude Sonnet 4.6 + +--- + +## Executive Summary + +This is a Diátaxis refactor branch of the Pact documentation site. Compared to a typical legacy codebase, it shows significant intentional improvement: the top-level navigation has been explicitly reorganised around the four quadrants, with dedicated `concepts.md`, `tutorials.md`, and the existing `consumer/` and `provider/` sections positioned as how-to content. The structural intent is clear and largely correct. However, several important documents still contain type-mixing anti-patterns, and two of the four quadrants remain underdeveloped or mislocated within the overall structure. + +--- + +## 1. What Adheres Well + +### The top-level navigation model (`getting_started.md`, `concepts.md`, `tutorials.md`) + +The landing page (`getting_started.md`) is the strongest structural win. Its "Where would you like to go?" table maps cleanly onto the four Diátaxis types: + +| User intent | Destination | +|---|---| +| Follow a tutorial | Tutorials | +| Understand how Pact works | Concepts | +| Solve a specific problem | How-to Guides | +| Look up API/spec details | Pact Docs | + +This is correct Diátaxis navigation architecture. The reader is routed to the right quadrant from the first page they land on. + +`concepts.md` correctly acts as a signpost page for Explanation content. It labels itself clearly as "the why and how behind Pact" and routes users to conceptual sub-pages. The framing — "If you're new to contract testing or want to deepen your understanding" — is the right register for Explanation. + +`tutorials.md` is a well-structured Tutorial hub. It names concrete outcomes ("5-minute quick start", "Introduction to Pact — 13 steps", "Async messages") and links to the workshop steps. The framing ("hands-on tutorials to learn Pact by doing") is correct. The "After the tutorials" section correctly guides users onward to How-to and Reference material — this cross-linking pattern is exactly what Diátaxis recommends. + +### University workshop steps (`university/introduction/`, `university/message-pact-async/`) + +These are genuine Tutorials in execution. They are sequential, step-based, learning-by-doing exercises. Each step (`01_Step1.md` through `13_Step13.md`) follows the learner-led progression that defines a tutorial. The scaffolding files (`00_1_Intro.md`, `00_3_LearningObjectives.md`, etc.) support the tutorial's promise of a guided learning journey. + +### Reference material (`pact_broker/configuration/settings.md`, `pact_broker/api/webhooks.md`) + +`pact_broker/configuration/settings.md` is correctly structured as Reference. It is auto-generated, structured by the product's own parameter names, and contains no procedural steps. Each entry lists environment variable name, YAML key, default, and allowed values — consistent patterns across all items. This is textbook Reference. + +`pact_broker/api/webhooks.md` follows the same pattern — paths, allowed methods, request/response schemas — with no instructions or opinions embedded. Correct placement. + +`pact_broker/advanced_topics/consumer_version_selectors.md` is a strong Reference document. It defines each selector property neutrally, lists them consistently, and provides code examples structured by language. The inline comments explaining _why_ not to use `latest: true` drift briefly toward opinion, but this is minor. The overall structure is correct for a document that users will consult during work, not read through. + +`implementation_guides/overview.md` correctly functions as a Reference index — tables of languages, CLI tools, Docker images, and Homebrew packages, structured by the product's own taxonomy. + +### How-to guides (`consumer.md`, `provider/how_to_fix_failing_verification_tests.md`) + +`consumer.md` (Writing Consumer Tests) is close to a correct How-to. It is organised around practitioner rules ("Use Pact for isolated unit tests", "Avoid using Pact for tests that involve the UI", "Ensure all calls to the Provider go through tested classes"). The Best Practice Guide section with good/bad code examples is well-suited to a How-to audience. + +`provider/how_to_fix_failing_verification_tests.md` is a correctly scoped How-to: a specific problem ("my pact verification tests are failing"), with concrete steps, per-language. It is numbered, goal-directed, and assumes the user knows they have a problem to solve. + +--- + +## 2. What Needs Improvement + +### `5-minute-getting-started-guide.md` — Tutorial with anti-patterns + +**Classification:** Intended as Tutorial; partially functioning as one. + +**Issues:** +- Lines 40–53: Instead of guiding the learner through actions, the guide pauses to explain _scope_ — "Ideally, the Pact tests should be 'unit tests' for your client class… Remember that pact is for testing the contract used for communication." This is Explanation embedded in a Tutorial. A Tutorial learner does not need to understand _why_ scope matters during their first steps; they need to be told what to do next. +- Lines 237–242: The aside about 404/400/500 status codes ("you would rinse and repeat") breaks the tutorial's linear journey with open-ended suggestions. Tutorials should not offer choices or branches — they lead. +- The title says "5 minutes" but the guide includes significant conceptual passages and requires environment setup, forking, and cloning. The learning contract with the reader is broken. + +**Suggestion:** Split this into two documents. Retain a true 5-minute tutorial that guides the learner through exactly one consumer-provider test end-to-end with no explanatory digressions. Move the explanatory content about test scope and status codes to the relevant Concepts or How-to pages (some of this already exists in `consumer.md`). + +--- + +### `getting_started/how_pact_works.md` — Explanation with embedded How-to + +**Classification:** Primarily Explanation; contains how-to instructions. + +**Issues:** +- Lines 105–193 ("How to write message pact tests?") embed a full walkthrough of Ports and Adapters architecture, code samples, and procedural steps for consumer and provider sides. This is How-to material. Its presence in what should be a conceptual overview document breaks the Explanation quadrant. +- The title "How Pact works" signals Explanation; the final third of the document delivers instruction. + +**Suggestion:** Extract lines 105–193 ("How to write message pact tests?") into a dedicated How-to guide under `consumer/` or `provider/` — something like "How to test async message integrations." Retain the conceptual description of Message Pact (what it is, why it abstracts protocols) in `how_pact_works.md`. + +--- + +### `getting_started/conceptual_overview.md` — Explanation with procedural leakage + +**Classification:** Primarily Explanation; anti-patterns present. + +**Issues:** +- Line 44: "I'll discuss later how we use branches and environments to help answer the question: can I deploy?" — first-person narrative is appropriate for Explanation, but the phrase "I'll discuss later" suggests a tutorial's sequential structure rather than a stable conceptual reference. Explanation does not have a "later." +- The document mixes the right conceptual content (domain model tables, how branches and versions relate) with forward references that make it feel like a chapter in a guide rather than a standalone explanation. +- There are no explicit heading anchors or a clear scope boundary. The page stops abruptly at "pacticipant." + +**Suggestion:** Make this document self-contained and explicitly scoped. Add a brief intro paragraph naming what the page explains and its boundaries. Remove the "I'll discuss later" forward references. Replace them with links to the relevant how-to or reference documents. + +--- + +### `pact_nirvana.md` / `pact_nirvana/step_*.md` — How-to mixed with Tutorial framing + +**Classification:** Mixed — part CI/CD How-to, part Tutorial walkthrough. + +**Issues:** +- The intro (`pact_nirvana.md`) is framed as a how-to guide: "a technical guide for developers and testers who want to use Pact to implement consumer driven contract testing as part of your ongoing CI/CD workflow." This is correct how-to positioning. +- However, the step files (`step_1.md` through `step_7.md`) are sequential and prescriptive, resembling a Tutorial. `step_1.md` reads: "Before you read this document, you should have read..." — this prerequisite structure belongs to Tutorial, not How-to. +- The "steps" metaphor and the linear Bronze→Silver→Gold→Platinum progression blur the line between "stages of a guide" and "tutorial steps." +- `step_1.md` contains almost no actionable content — it is a list of prerequisites and links, not a guide step. + +**Suggestion:** Clarify the document type explicitly. If this is a How-to, reorganise around tasks the reader needs to accomplish ("How to set up Pact Broker integration in CI", "How to configure can-i-deploy"). If it is a Tutorial, rewrite it with first-person plural framing ("Let's configure...") and ensure it reaches a satisfying conclusion. A hybrid is difficult to maintain and confuses both types of users. + +--- + +### `getting_started/matching.md` — Reference with embedded Best Practice + +**Classification:** Mixed — partly Reference, partly How-to opinion. + +**Issues:** +- Lines 183–199 ("Best practice — Request matching / Response matching / Random data") embed opinionated guidance ("You generally want to use exact matching... You want to be _as loose as possible_...") into what is otherwise a factual description of matching features. +- The first ~180 lines correctly describe matcher syntax and behaviour (Reference). The last section adds opinion and judgement — which is How-to territory. + +**Suggestion:** Move the "Best practice" section to `consumer.md` or a dedicated how-to page on choosing matchers. Leave `matching.md` as a neutral description of the matching features available. + +--- + +### `pact_broker/can_i_deploy.md` — How-to with Explanation embedded + +**Classification:** Primarily How-to; good but contains conceptual explanation that interrupts the task. + +**Issues:** +- Lines 7–13: The explanation of what the Matrix is ("When a pact is published, the version of the consumer that generated the pact is recorded...") is Explanation. A user consulting can-i-deploy to accomplish a deployment task does not need this background explained inline. It belongs in the Pact Broker Overview (`pact_broker/overview.md`), which already covers the Matrix concept. +- The document improves significantly once it starts with concrete commands. The Summary at line 93 is exactly right for a How-to — it restates the two commands the user needs. + +**Suggestion:** Replace the explanatory opening paragraph with a one-sentence statement of the problem ("Before deploying, you need to confirm the version you're deploying is compatible with what's already in the environment") and a link to the Matrix concept in the overview. The bulk of the document is solid. + +--- + +### `recipes/kafka.md` — How-to without How-to structure + +**Classification:** Intended as How-to (recipe); reads as an informal tutorial walkthrough. + +**Issues:** +- The document uses first-person narrative ("We'll start with...", "Lets take a look at..."), which is tutorial voice, not how-to voice. +- It walks through code chronologically ("Next we define...", "Moving on...") in a way that teaches the pattern rather than efficiently directing someone to accomplish a task. +- There is no clear problem statement ("You need to test a Kafka consumer using Pact — here's how") or assumed competence level. +- Code blocks are not preceded by explanatory headings that tell the reader what they're about to do. + +**Suggestion:** Restructure as a How-to guide: state the goal upfront, use imperative headings ("Configure your consumer test", "Write the provider verification"), and remove the narrative walkalong voice. The technical content is good; the framing needs to shift from "let me show you" to "here is how to do it." + +--- + +### `consumer.md` — Structural boundary issue: Best Practice Guide section + +**Classification:** Mostly How-to; Best Practice Guide section is appropriate but very long. + +**Issues:** +- The "Best Practice Guide" section (lines 110–640) is a comprehensive, well-organised set of rules. However, its depth and structure (numbered sections with subsections, rule boxes, code examples) makes it feel like a standalone reference document rather than integrated how-to guidance. +- It would benefit from being its own page (e.g., `consumer/best_practices.md`) with a clear link from `consumer.md`, reducing the length of the main how-to page. + +**Suggestion:** Extract the Best Practice Guide into its own document. This would make `consumer.md` a proper compact how-to, and the Best Practice Guide page would be findable on its own as a reference or how-to companion. + +--- + +## 3. Significant Gaps + +### Gap 1: No standalone Explanation for core Pact concepts + +The `concepts.md` page correctly positions itself as an Explanation hub, and `how_pact_works.md` and `conceptual_overview.md` are attempts at Explanation. However, there is no dedicated Explanation page that addresses the fundamental question a newcomer has: **"Why does contract testing exist, and why is Pact's consumer-driven approach the right design?"** + +`getting_started/what_is_pact_good_for.md` is a good start — it explains trade-offs and design reasoning. But the questions of _why consumer-driven contracts rather than schema testing_, _why the pact file is generated rather than hand-written_, and _how this fits into a microservices testing strategy_ are scattered across FAQs, the landing page, and blog links. These are precisely the "distance from the work" topics that Explanation is for — the user is not trying to do anything; they are trying to understand. + +**User need left unmet:** A developer new to contract testing who has finished the tutorial and wants to understand why Pact works the way it does cannot find a single, coherent page that explains the design philosophy, trade-offs, and mental model. They leave without the conceptual foundation needed to make good decisions when Pact behaves unexpectedly. + +**Suggestion:** Create `concepts/why-pact.md` or `concepts/design-philosophy.md` that brings together the rationale for consumer-driven contracts, the difference from schema testing, the philosophy of test independence, and why the broker model exists. This page should be read away from the computer, not during work. + +--- + +### Gap 2: No Reference index for the Pact file format / specification + +The `getting_started/specification.md` links to the external GitHub specification repository. There is no on-site Reference documentation describing the structure of a pact file — what fields it contains, what the schema is, what each field means, and how different specification versions differ. + +**User need left unmet:** A developer who is debugging a pact file, writing a custom publisher, or building tooling around the pact format has no on-site reference to consult. They must go to GitHub and read raw specification files. + +**Suggestion:** Create a Reference page `reference/pact-file-format.md` that describes the JSON structure of a pact file — top-level fields, interaction structure, matching rules, and metadata — across specification versions. + +--- + +### Gap 3: Shallow provider how-to coverage compared to consumer + +The `consumer.md` page is extensive (640+ lines with best practices). The `provider.md` equivalent is much thinner. The provider how-to section contains: +- `provider/recommended_configuration.md` +- `provider/handling_auth.md` +- `provider/how_to_fix_failing_verification_tests.md` +- `provider/using_provider_states_effectively.md` + +There is no equivalent of the consumer Best Practice Guide for providers. There is no "how to write good provider tests" with the depth and structure that `consumer.md` provides for consumers. + +**User need left unmet:** A provider team picking up Pact for the first time has deep consumer guidance available but must figure out provider verification best practices by reading implementation-guide documentation for their specific language, rather than finding general principled guidance. + +**Suggestion:** Create a `provider/best_practices.md` that mirrors the structure of the consumer best practice guide — covering test isolation, stubbing dependencies, provider state design, and verification scope. + +--- + +### Gap 4: Plugin documentation has no Explanation-level page + +The `plugins/` section contains `directory.md` (a reference list) and `quick_start.md`, plus the `create-a-plugin` workshop tutorial. There is no page that explains the conceptual model of the plugin system: what plugins are, how they integrate with the FFI layer, when you would build one versus using an existing one, and what the protocol boundaries are. + +The design documents under `implementation_guides/pact_plugins/docs/` (content-matcher-design, plugin-driver-design) are Explanation content buried in the implementation guides section, inaccessible to users who approach from the top-level plugins menu. + +**User need left unmet:** A developer evaluating whether to build a Pact plugin cannot find a conceptual overview in the user-facing plugins section. They either do the tutorial (too much) or read implementation design docs written for contributors (too much the other way). + +**Suggestion:** Create `plugins/concepts.md` that explains the plugin model at a conceptual level — suitable for someone evaluating whether plugins address their use case, before they commit to a tutorial. + +--- + +## After the Report + +The most impactful next step would be addressing Gap 1 — the missing Explanation page for Pact's design philosophy. This single gap affects every new user who completes a tutorial and then struggles to understand why Pact behaves the way it does. I can draft `concepts/why-pact.md` if that would be useful. + +Alternatively, if the priority is fixing existing content rather than creating new pages, the highest-impact improvements would be: +1. Extracting the Message Pact how-to walkthrough out of `getting_started/how_pact_works.md` (it contaminates what should be a clean Explanation page) +2. Splitting the `5-minute-getting-started-guide.md` to remove the Explanation digressions +3. Giving the `pact_nirvana/` section a clear type declaration and restructuring accordingly + +Which of these would you like to tackle first? diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 000000000..5a0df25c5 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,139 @@ +# Diataxis Refactor Plan — docs.pact.io + +## Status: In Progress +Last updated: 2026-03-20 + +--- + +## 1. Audit Summary + +### Current navigation structure + +| Nav Item | Sidebar key | Description | +|---|---|---| +| Getting Started | `getting-started` | Landing, How Pact Works, 5-min guide, Pact Concepts, Workshops, FAQ | +| Guides | `guides` | Consumer best practices, Provider best practices, Pact Nirvana (CI/CD), Recipes | +| Pact Docs | `docs` | Implementation guides per language, CLI tools, Plugins, Feature support | +| Pact Broker Docs | `pact_broker` | Broker admin, webhooks, can-i-deploy, etc. | +| Resources | `resources` | History, blogs, books, events, case studies | +| Contributing | `contributing` | Code, CI, docs, contributors | +| Blog | (separate) | | +| Help | `help` | How to ask for help, debug logging, SmartBear resources | +| University | `university` | Intro to Pact (13 steps), Async messages (4 steps), Plugins workshop | + +--- + +## 2. Key constraints + +- **DO NOT modify** anything under `website/docs/implementation_guides/` (Pact Docs) +- **DO NOT modify** anything under `website/docs/pact_broker/` (Pact Broker Docs) +- **DO NOT modify** anything under `website/blog/` (Blog) +- **Add redirects** in `netlify.toml` for any URL changes + +--- + +## 3. Implementation plan + +### Phase 1–13: Navigation and structural restructure ✅ COMPLETE +All phases completed 2026-03-19. Summary: +- Added Concepts sidebar (extracted from Getting Started) +- Added Tutorials sidebar (replacing University nav item) +- Renamed Guides → How-to Guides in nav + footer +- Slimmed Getting Started; added Diataxis signposting table to landing page +- Added `/university` → `/tutorials` redirect in netlify.toml +- Fixed pre-existing broken Step file references in tutorials sidebar +- `further_reading` moved to Resources sidebar +- `concepts.md` and `tutorials.md` landing pages created + +--- + +### Phase 14: Diataxis review + content gaps ✅ COMPLETE (2026-03-20) + +Full Diataxis review conducted. Four content gaps identified and filled: + +- [x] **Gap 1** — `getting_started/why-pact.md` created: design philosophy explanation (consumer-driven contracts, generated pact files, broker model, trade-offs) +- [x] **Gap 2** — `getting_started/pact-file-format.md` created: Reference page for pact JSON schema across spec versions V1–V4 +- [x] **Gap 3** — `provider/best_practices.md` created: provider how-to guide mirroring consumer.md depth +- [x] **Gap 4** — `plugins/concepts.md` created: Explanation of the plugin model and system actors +- [x] `concepts.md` updated with new entries for why-pact and pact-file-format +- [x] All four pages wired into `sidebars.json` + +--- + +### Phase 15: Structural fixes to existing content ✅ COMPLETE (2026-03-20) + +Eight type-mixing and framing issues fixed: + +- [x] **Fix 1** — `getting_started/how_pact_works.md`: extracted Message Pact how-to walkthrough → `consumer/how_to_test_async_messages.md` +- [x] **Fix 2** — `5-minute-getting-started-guide.md`: removed embedded Explanation digressions (consumer scope, provider scope, status code advice) +- [x] **Fix 3** — `pact_nirvana.md` + `step_1.md`: declared as how-to guide; removed tutorial prerequisite voice +- [x] **Fix 4** — `getting_started/matching.md`: moved Best practice section to `consumer.md`; added "avoid random data" guidance +- [x] **Fix 5** — `pact_broker/can_i_deploy.md`: replaced Matrix explanation with 2-sentence problem statement + link +- [x] **Fix 6** — `recipes/kafka.md`: full rewrite with imperative headings and goal-first framing +- [x] **Fix 7** — `consumer.md`: extracted 530-line Best Practice Guide → `consumer/best_practices.md` +- [x] **Fix 8** — `getting_started/conceptual_overview.md`: added scoped intro; replaced "I'll discuss later" with direct link + +--- + +### Phase 16: Tone and style consistency pass ✅ COMPLETE (2026-03-20) + +Rewrite in-scope pages for consistent tone and style: +- Second-person voice throughout ("you", not "we" or "I") +- Imperative framing in How-to pages +- Neutral descriptive framing in Reference/Explanation pages +- Remove hedging language ("you might want to", "it can be tempting to") +- Remove filler openers ("Remember that...", "As you would hopefully have read") +- Remove first-person author voice ("I'll discuss", "Let's walk through") +- Consistent heading capitalisation (sentence case) + +Scope excludes `implementation_guides/`, `pact_broker/`, `blog/`, and pages written fresh in phases 14–15 (already consistent). + +#### Getting Started section +- [x] **T1** — `getting_started.md` (landing page) +- [x] **T2** — `getting_started/what_is_pact_good_for.md` +- [x] **T3** — `getting_started/how_pact_works.md` +- [x] **T4** — `getting_started/terminology.md` (already clean) +- [x] **T5** — `getting_started/testing-scope.md` +- [x] **T6** — `getting_started/verifying_pacts.md` +- [x] **T7** — `getting_started/provider_states.md` +- [x] **T8** — `getting_started/sharing_pacts.md` +- [x] **T9** — `getting_started/comparisons.md` (already clean) + +#### FAQ +- [x] **T10** — `faq.md` + `faq/convinceme.md` + +#### Consumer how-to +- [x] **T11** — `consumer.md` + `consumer/contract_tests_not_functional_tests.md` + `consumer/using_pact_to_support_ui_testing.md` + `consumer/recommended_configuration.md` + +#### Provider how-to +- [x] **T12** — `provider.md` + `provider/using_provider_states_effectively.md` + `provider/handling_auth.md` + `provider/recommended_configuration.md` + `provider/how_to_fix_failing_verification_tests.md` + +#### CI/CD Guide +- [x] **T13** — `pact_nirvana/step_2.md` through `step_7.md` + +#### Recipes +- [x] **T14** — `recipes.md` + `recipes/optional.md` + `recipes/graphql.md` + `recipes/apigateway.md` + `recipes/awssignedrequests.md` + `recipes/lambdaasync.md` + `recipes/lambdahttp.md` + `recipes/cypress.md` + `recipes/munit.md` + +#### Plugins +- [x] **T15** — `plugins/quick_start.md` + `plugins/directory.md` + +--- + +### Phase 17: Housekeeping ✅ COMPLETE (2026-03-20) + +- [x] Commit `DIATAXIS_REVIEW.md` (committed 2026-03-20) +- [x] Final build verification — 2 pre-existing broken links remain in `implementation_guides/` (out of scope); 3 links fixed in concepts.md and step_1.md +- [x] Review for any remaining orphaned pages — none found +- [ ] PR / merge to master + +--- + +## Progress log + +- [x] **2026-03-19**: Worktree created, PLAN.md written +- [x] **2026-03-19**: Phases 2–13 — navigation restructure complete +- [x] **2026-03-20**: Sidebar step ID regression fixed (`01_Step1` → `Step1`); site confirmed running +- [x] **2026-03-20**: Phase 14 — four content gaps filled (4 new pages) +- [x] **2026-03-20**: Phase 15 — eight structural fixes applied +- [x] **2026-03-20**: Phase 16 — tone and style pass complete (T1–T15, 30+ files) +- [x] **2026-03-20**: Phase 17 — housekeeping complete; 3 broken links fixed; PR pending diff --git a/netlify.toml b/netlify.toml index bc346fb6a..43012bc74 100644 --- a/netlify.toml +++ b/netlify.toml @@ -225,3 +225,9 @@ from = "/implementation_guides/pact_specification" to = "/getting-started/specification" status = 301 + +# Diataxis refactor: /university landing page now redirects to /tutorials +[[redirects]] + from = "/university" + to = "/tutorials" + status = 301 diff --git a/website/docs/5-minute-getting-started-guide.md b/website/docs/5-minute-getting-started-guide.md index 4274ba564..0d508176e 100644 --- a/website/docs/5-minute-getting-started-guide.md +++ b/website/docs/5-minute-getting-started-guide.md @@ -9,7 +9,7 @@ description: From zero to running Pact tests in 5 mins ::: -This getting started guide can be run purely in the browser or on your local machine, with the intention to get you across the key concepts quickly. As we go through, there will be code snippets to demonstrate concepts. These are not runnable, but are there to assist with explanation. When there is code to run, you will be presented with a REPL as per below. Simply hit the green ▶ button, and the output will be displayed in the terminal below. Give it a shot now: +This guide walks you through writing your first consumer and provider Pact tests end-to-end using a simple Order API example. By the end, you will have a generated pact file and a passing provider verification. ## Ways to run @@ -41,16 +41,6 @@ Note that to create a pact, you _do_ need to write the code that executes the HT ## Testing the Order Web \(consumer\) project -### Scope of a Consumer Pact Test - -Ideally, the Pact tests should be "unit tests" for your client class, and they should just focus on ensuring that the request creation and response handling are correct. If you use pact for your UI tests, you'll end up with an explosion of redundant interactions that will make the verification process tedious. Remember that pact is for testing the contract used for communication, and not for testing particular UI behaviour or business logic. - -Usually, your application will be broken down into a number of sub-components, depending on what type of application your consumer is \(e.g. a Web application or another API\). This is how you might visualise the coverage of a consumer Pact test: - -![Scope of a consumer Pact test](/img/consumer-test-coverage.png) - -Here, a _Collaborator_ is a component whose job is to communicate with another system. In our case, this is the `OrderApiClient`communicating with the external `Order Api` system. This is what we want our consumer test to inspect. - ### 1. Start with your model Imagine a simple model class that looks something like this \(order.js\). The attributes for an Order live on a remote server, and will need to be retrieved by an HTTP call to the Order API. @@ -234,14 +224,7 @@ Running the passing Order API spec will generate a pact file in the configured p You now have a pact file that can be used to verify your expectations of the Order API provider project. -Now, in real life you would rinse and repeat for other likely status codes that may be returned. For example, consider how you want your client to respond to a: - -* 404 \(return null, or raise an error?\) -* 400 \(how should validation errors be handled, what will the body look like when there is one?\) -* 500 \(specifying that the response body should contain an error message, and ensuring that your client logs that error message will make your life much easier when things go wrong. Note that it may be hard to force your provider to generate a 500 error on demand. You may need to collaborate with your provider team to create a known provider state that will artificially return a 500 error, or you may just wish to use a standard unit test without a pact to test this.\) -* 401/403 if there is authorisation. - -#### **Run the consumer Tests!** +#### **Run the consumer tests!** OK enough talk - let's run the consumer test. If you like, click around the project to see the files from above in context. The most interesting file is the consumer test in `./consumer/consumer.spec.js` . @@ -267,20 +250,6 @@ After publishing the pact, we can now verify that the Provider meets these expec ## Testing the Order API \(provider\) project -### Scope of a Provider Pact Test - -On the Provider side, Pact needs to replay all of the interactions \(usually HTTP requests\) against your service. There are a number of choices that can be made here, but usually these are the choices: - -* Invoke just the controller layer \(in an MVC app, or the "Adapter" in our diagram\) and stub out layers beneath -* Choosing a real vs mocked out database -* Choosing to hit mock HTTP servers or mocks for external services - -Generally speaking, we test the entire service and mock out external services such as downstream APIs \(which would need their own set of Pact tests\) and databases. This gives you some of the benefits of an integration test without the high costs of maintenance. - -This is how you might visualise the coverage of a provider Pact test: - -![Provider side Pact test scope](/img/provider-test-coverage.png) - ### 1. Create the Order API Below we have created a simple API using [Express JS](https://expressjs.com). diff --git a/website/docs/concepts.md b/website/docs/concepts.md new file mode 100644 index 000000000..629511309 --- /dev/null +++ b/website/docs/concepts.md @@ -0,0 +1,33 @@ +--- +id: concepts +title: Pact Concepts +description: Understanding how and why Pact works — the key concepts behind consumer-driven contract testing. +--- + +This section explains the **why** and **how** behind Pact. If you're new to contract testing or want to deepen your understanding of how Pact works, start here. + +## What you'll find here + +| Topic | Description | +|---|---| +| [Why Pact?](/getting_started/why-pact) | The design philosophy behind consumer-driven contracts, generated pact files, and the broker model | +| [Conceptual overview](/getting_started/conceptual_overview) | A visual and narrative overview of the full Pact workflow | +| [Terminology](/getting_started/terminology) | Definitions of key Pact terms: consumer, provider, pact, interaction, and more | +| [Testing scope](/getting_started/testing-scope) | What Pact tests cover, and what it deliberately does not | +| [Matching](/getting_started/matching) | How Pact compares requests and responses, and the types of matchers available | +| [Provider states](/getting_started/provider_states) | How to set up the provider data needed for each interaction | +| [Verifying pacts](/getting_started/verifying_pacts) | What happens when the provider runs verification | +| [Sharing pacts](/getting_started/sharing_pacts) | How pact files flow between consumer and provider teams | + +## Reference + +| Topic | Description | +|---|---| +| [Pact specification](/getting_started/specification) | The Pact specification versions and what they support | +| [Pact file format](/getting_started/pact-file-format) | The JSON structure of a pact file, including fields, matching rules, and generators | + +## Not sure where to start? + +- **New to Pact?** Start with the [Introduction](/) and [How Pact works](/getting_started/how_pact_works). +- **Ready to write code?** Jump straight to [Tutorials](/tutorials). +- **Solving a specific problem?** See [How-to Guides](/consumer). diff --git a/website/docs/consumer.md b/website/docs/consumer.md index 2c3141e09..ff19c10a4 100644 --- a/website/docs/consumer.md +++ b/website/docs/consumer.md @@ -51,12 +51,20 @@ Avoid the temptation to make assertions about general business rules that you kn Typically, exact matching is most appropriate for Pact tests on the consumer side that are running at the unit test level. The same person is responsible for both the expectation and the actual request, so making sure that they match should be straightforward. -If you are using Pact for tests that cover more than just the "client" code, then you might want to use a looser matching (eg. type based matching and regular expressions). Remember that the more of the consumer code you cover with your Pact tests, the less maintainable it is, so do this with caution, and make sure you understand the trade-offs. +If you are using Pact for tests that cover more than just the "client" code, then you might want to use a looser matching (eg. type based matching and regular expressions). The more consumer code you cover with Pact tests, the less maintainable they become — proceed with caution, and make sure you understand the trade-offs. For the response, loose matching is generally the recommended option, however, the exact/loose matching decision should be made on a field by field basis. Most of the time, the consumer should not really care about the contents of a response field, and should just care about the type. However, there will be times when the contents do matter, so use exact matching when this is the case. The question to ask yourself when writing the response expectations is "if I made this looser/tighter what bugs would I miss/prevent?" If you find you are putting in strict matchers on the contents of a field because you need to extract a particular piece of data out of it (eg. parsing a street number out of an address, or an ID out of a URL) or you are trying to infer a fact about it (e.g codes that start with `VIC-` are from the state of Victoria) then that tends to mean that the API should be providing that field for you separately itself, rather than pushing that work into the consumer side. Your tests are sending you message! Creating explicit separate fields for things like this will make the API more usable and the consumer code much less brittle. Highlighting issues like this is exactly what the "consumer driven" part of "consumer driven contracts" is about. +For a full reference on the available matcher types, see [Matching](/getting_started/matching). + +## Avoid random data in interactions + +If you are using a Pact Broker, avoid using random data in your pact interactions. When a new pact is published that is identical to a previously verified version, the existing verification results are automatically applied — which means the consumer can deploy immediately without waiting for a new provider verification run. + +Random data makes every pact publication look like a new contract version, even when nothing meaningful has changed. This breaks the optimisation and forces an unnecessary provider verification on every consumer build. + ## Use `Pact` for isolated \(unit\) tests * as a mock \(calls to mocks are verified after a test\) not a stub \(calls to stubs are not verified\). Using `Pact` as a stub defeats the purpose of using `Pacts`. @@ -107,535 +115,5 @@ Sure, you’ve checked that your client deserialises the HTTP response into the ## Best Practice Guide -This guide defines best practices for writing high-quality Pact consumer unit tests. Each rule includes an example of good and bad practice to help users lint and review their own code. - -### 🧱 1. Test Structure and Naming - -#### ✅ Rule: Use meaningful test descriptions -These should describe the client’s intent, not just the request type. - -**Explanation:** -Descriptive names improve readability and make it easier for consumers and providers to understand what’s being tested. Avoid repeating technical details already expressed in the test code. - -**✅ Good:** -```ts -.uponReceiving("a request to get all products") -``` - -**❌ Bad:** -```ts -.uponReceiving("GET /products") // Redundant information already described by the Pact test -``` - -#### ✅ Rule: Use human-readable test cases (BDD-style) -Combine the provider state, request description, and response to form a natural-language sentence. - -**Explanation:** -This helps others (including non-developers) understand test intent, and is particularly useful when reviewing contracts in the Pact Broker UI. - -**✅ Good:** -```ts -.addInteraction() - .given("there are active products in the catalog") - .uponReceiving("a request to get all products") - .withRequest("GET", "/products") - .willRespondWith(200, ...) -``` - -Reads as: -> **Given** there are active products in the catalog, **upon receiving** a request to get all products, **the provider will respond with** HTTP 200 with a list of products. - -**❌ Bad:** -```ts -.addInteraction() - .uponReceiving("GET /products") - .withRequest("GET", "/products") - .willRespondWith(200, ...) -``` - - -#### ✅ Rule: Use a consistent test template -Adopt a known structure across your test suite. - -**Explanation:** -A consistent structure makes tests easier to follow, review, and maintain — especially in large teams or codebases. - -**✅ Good:** -```ts -// Top level - name of the API -describe("Thing API", () => { - const pact = new PactV4({ - consumer: "ThingConsumer", - provider: "ThingProvider", - spec: SpecificationVersion.SPECIFICATION_VERSION_V4, - }); - - // Level 2 - Describe block for the specific API endpoint - describe("GET /thing/:id", () => { - - // Level 3 - Test block for the specific test case - test("given a valid thing, returns 200", async () => { - await pact - .addInteraction() - .given("a thing with id 1 exists") - .uponReceiving("a request for thing with ID 1") - .withRequest("GET", "/thing/1", (builder) => { - builder.headers({ Accept: "application/json" }); - }) - .willRespondWith(200, (builder) => { - builder.jsonBody( - like({ - id: 1, - name: "Thing 1", - price: 100, - }) - ); - }) - .executeTest(async (mockserver) => { - const ThingAPI = new ThingAPI(mockserver.url); - const Thing = await ThingAPI.getThingById(1); - expect(Thing).toEqual({ - id: 1, - name: "Some 1", - price: 100, - }); - }); - }); - }); -}); -``` - -**❌ Bad:** -```ts -// Random test structure per case, hard to follow -describe("Pact Tests", () => { - const pact = new PactV4({ - consumer: "ThingConsumer", - provider: "ThingProvider", - spec: SpecificationVersion.SPECIFICATION_VERSION_V4, - }); - - // Not organised well, hard to see what has been tested - test("can call the API", async () => { - await pact - .addInteraction() - .uponReceiving("get a thing") - .withRequest("GET", "/thing/1", (builder) => { - builder.headers({ Accept: "application/json" }); - }) - ... - }); -}); -``` - -### 🧪 2. Test Design and Scope - -#### ✅ Rule: Bug catcher - -Only include tests that protect against real consumer breakages. - -**Explanation**: -Each test example in your contract should exist for a reason: to detect a breaking change in the provider that would actually impact the consumer. A good rule of thumb is to ask: *“If I remove this test, is there a type of provider change that could break the consumer and go unnoticed?”* If the answer is “no,” the test is unnecessary. Avoid adding examples just for coverage or realism — focus instead on what the consumer truly depends on. This keeps your contracts lean, meaningful, and maintainable. - -✅ Good example: -The consumer behaves differently depending on the `status` field returned by a decision API. Removing any of these tests would leave a branch of consumer logic untested. - -```ts -// decisionClient.ts – actual consumer logic -export function handleDecision(response: { status: string }) { - switch (response.status) { - case 'approved': - return '✅ Proceed'; - case 'rejected': - return '❌ Stop'; - case 'pending': - return '⏳ Await further input'; - default: - throw new Error(`Unhandled status: ${response.status}`); - } -} -``` - -```ts -it('handles approved decision', async () => { - await provider.addInteraction({ - uponReceiving: 'an approved decision', - withRequest: { method: 'GET', path: '/decision' }, - willRespondWith: { - status: 200, - body: { status: 'approved' } - } - }); - - const res = await getDecision(provider.mockService); - expect(handleDecision(res)).toBe('✅ Proceed'); -}); - -it('handles rejected decision', async () => { - await provider.addInteraction({ - uponReceiving: 'a rejected decision', - withRequest: { method: 'GET', path: '/decision' }, - willRespondWith: { - status: 200, - body: { status: 'rejected' } - } - }); - - const res = await getDecision(provider.mockService); - expect(handleDecision(res)).toBe('❌ Stop'); -}); - -it('handles pending decision', async () => { - await provider.addInteraction({ - uponReceiving: 'a pending decision', - withRequest: { method: 'GET', path: '/decision' }, - willRespondWith: { - status: 200, - body: { status: 'pending' } - } - }); - - const res = await getDecision(provider.mockService); - expect(handleDecision(res)).toBe('⏳ Await further input'); -}); -``` - -❌ Bad practice: -If we only tested `'approved'` responses, the `'rejected'` and `'pending'` branches would be untested. If the provider changed those values or response structure, the consumer would break without the contract test noticing. - - -#### ✅ Rule: Test the actual API Client -Always exercise the real consumer code in your contract tests. - -**Explanation**: -Contract tests should verify that your actual consumer code (e.g. your API client or data-access layer) behaves correctly when interacting with the provider. If you use a generic HTTP client (e.g. `fetch`, `axios`) directly in the test instead of invoking the real consumer logic, you're not truly testing the consumer. This creates blind spots: the contract may be valid, but your consumer might still fail at runtime. Worse, it may give the false impression that the contract is safe to verify on the provider side, even though the consumer has never been properly exercised. - -✅ Good example: -A test that imports your app’s API client and verifies how it behaves with mock provider responses. - -```js -// consumer.js – your actual app code -export async function getUser(apiClient, id) { - const response = await apiClient.get(`/users/${id}`); - return response.data; -} - -// consumer.pact.test.js – contract test -import { getUser } from './consumer'; - -it('gets a user', async () => { - await provider.addInteraction({ - uponReceiving: 'a request for a user', - withRequest: { - method: 'GET', - path: '/users/123', - }, - willRespondWith: { - status: 200, - body: { - id: '123', - name: 'Alice' - } - } - }); - - const result = await getUser(provider.mockService, '123'); - expect(result).toEqual({ id: '123', name: 'Alice' }); -}); -``` - -❌ Bad example: -A test that uses `axios` or `fetch` directly, bypassing your actual consumer logic. - -```js -import axios from 'axios'; - -it('gets a user (but bypasses consumer logic)', async () => { - await provider.addInteraction({ - uponReceiving: 'a request for a user', - withRequest: { - method: 'GET', - path: '/users/123', - }, - willRespondWith: { - status: 200, - body: { - id: '123', - name: 'Alice' - } - } - }); - - const response = await axios.get(`${provider.mockService.baseUrl}/users/123`); - expect(response.data).toEqual({ id: '123', name: 'Alice' }); -}); -``` - -Even though this test may pass and a contract is generated, the actual consumer code isn’t tested — which defeats the purpose of contract testing. - -#### ✅ Rule: Add one interaction per test -Each test should represent a single, isolated contract scenario - an *interaction*. - -**Explanation:** -This makes tests clearer, failures easier to diagnose and ensures interactions are independently verifiable. - - -**✅ Good:** -```ts -test("returns 200 when product exists", async () => { - await pact - .addInteraction() - .given("product with ID 1 exists") - .uponReceiving("a request for product with ID 1") - .withRequest("GET", "/products/1") - .willRespondWith(200, ...); -}); - -test("returns 404 when product does not exist", async () => { - await pact - .addInteraction() - .given("no product with ID 999 exists") - .uponReceiving("a request for product with ID 999") - .withRequest("GET", "/products/999") - .willRespondWith(404, ...); -}); -``` - -**❌ Bad:** -```ts -test("product endpoints", async () => { - // Multiple unrelated interactions in the same test block - await pact - .addInteraction() - .given("product with ID 1 exists") - .uponReceiving("a request for product with ID 1") - .withRequest("GET", "/products/1") - .willRespondWith(200, ...); - - await pact - .addInteraction() - .given("no product with ID 999 exists") - .uponReceiving("a request for product with ID 999") - .withRequest("GET", "/products/999") - .willRespondWith(404, ...); - - // Then run both assertions - const existing = await api.getProduct("1"); - const missing = await api.getProduct("999"); - - expect(existing).toEqual(...); - expect(missing).toEqual(...); -}); -``` - -#### ✅ Rule: Minimize assertions — assert contract, not behavior -Only assert what’s needed to verify the contract. - -**Explanation:** -Avoid including unrelated assertions (e.g. logging, side effects), which can introduce flakiness and clutter. - -**✅ Good:** -```ts -expect(response).toEqual([...]); -``` - -**❌ Bad:** -```ts -expect(response).toEqual([...]); -expect(logger.debug).toHaveBeenCalled(); -``` - -#### ✅ Rule: Only include data and operations used by the consumer -Test only what your consumer relies on. - -**Explanation:** -This keeps the contract minimal and avoids unnecessary coupling to unrelated parts of the API. Additional redundant assertions on the provider unnecessarily constrains change. - -**✅ Good:** -```ts -builder.jsonBody({ id: like("123"), name: like("Product 1"), price: like(37.25) }); -``` - -**❌ Bad:** -```ts -builder.jsonBody({ - id: like("123"), - name: like("Product 1"), - price: like(37.25), - availableStores: [...], - ... // Unused fields -}); -``` - -#### ✅ Rule: Focus on contract validation, not functional testing -Contract tests verify HTTP request/response structures, not business logic. - -**Explanation:** -System behavior and business logic should be tested separately. Pact tests should focus solely on API client behaviour. - -**✅ Good:** -```ts -test("returns products in correct format", async () => { - await pact - .addInteraction() - .uponReceiving("a request to get all products") - .withRequest("GET", "/products") - .willRespondWith(200, (builder) => { - builder.jsonBody([{ id: "1", name: "Product 1" }]); - }) - .executeTest(async (mockserver) => { - const products = await new API(mockserver.url).getAllProducts(); - expect(products).toEqual([{ id: "1", name: "Product 1" }]); - }); -}); -``` - -**❌ Bad:** -```ts -test("calculates total cart value", () => { - const cart = new Cart(); - cart.addItem({ price: 10 }); - expect(cart.total()).toBe(10); // functional test, not contract test -}); -``` - -### 🧭 3. Clarity and Maintainability - -#### ✅ Rule: Use provider states for clarity and maintainability -State setup should be explicit. - -**Explanation:** -This improves test clarity and helps the provider team replicate test conditions during verification. - -**✅ Good:** -```ts -.given("product with ID 1 exists") -.uponReceiving("a request for product with ID 1") -.withRequest("GET", "/products/1") -.willRespondWith(200, ...) -``` - -**❌ Bad:** -```ts -.uponReceiving("a request for product with ID 1") -.withRequest("GET", "/products/1") -.willRespondWith(200, ...) -// No provider state set — makes provider testing harder, reduces comprehension -``` - -#### ✅ Rule: Use matchers to reduce test fragility -Avoid asserting on exact values unless necessary. - -**Explanation:** -Using matchers like `like()` or `eachLike()` ensures the contract is flexible and tolerant to non-breaking changes. - -**✅ Good:** -```ts -eachLike({ id: like("123"), name: like("Product 1") }) -``` - -**❌ Bad:** -```ts -[ { id: "123", name: "Product 1" } ] // Makes the provider tests more brittle and coupled -``` - -#### 🧱 Avoid over-constraining the provider - -Only assert on formats or constraints that your consumer actually depends on. - -**Explanation**: -Avoid over-constraining the provider by specifying overly strict formats (e.g. using regex patterns) unless your consumer truly relies on them. While you might know how the provider currently behaves, asserting on things your consumer doesn’t need — like a specific ID format — can create unnecessary fragility. This can lead to false test failures if the provider changes in ways that don’t affect your consumer. Keep your assertions just strict enough to ensure your consumer continues to work as expected — and no stricter. - - -✅ OK: -```json -{ - "customerId": "ABC-123" -} -``` - -❌ Over-constrained: -```ts -{ - "customerId": regex("[A-Z]{3}-\d{3}", "ABC-123") // forces provider to constrain to a regex -} -``` - -Instead, keep your assertions just strict enough to ensure your consumer works as expected — and no stricter. This helps the provider evolve safely without false test failures. - -#### ✅ Rule: Avoid duplicate interaction descriptions -Make each interaction description unique. - -**Explanation:** -The Pact Broker de-duplicates interactions based on description + provider state. Re-using the same description can result in dropped tests. - -**✅ Good:** -```ts -.given("there are active products in the catalog") -.uponReceiving("a request for a single product") -.withRequest("GET", "/products/1") -.willRespondWith(200, ...) -... -.given("there are no active products in the catalog") -.uponReceiving("a request for a single product") // OK: provider state differentiates scenario -.withRequest("GET", "/products/1") -.willRespondWith(404, ...) -``` - -**❌ Bad:** -```ts -.uponReceiving("a request for a single product") // BAD: no provider state -.withRequest("GET", "/products/1") -.willRespondWith(200, ...) -... -.uponReceiving("a request for a single product") // BAD: no provider state, overrides above -.withRequest("GET", "/products/1") -.willRespondWith(404, ...) -``` - -#### ✅ Rule: Ensure Pact files are deterministic -Avoid dynamic data without fixed examples. - -**Explanation:** -Dynamic data can lead to flaky tests and noisy diffs. Always fix example values when using generators. - -**✅ Good:** -```ts -builder.jsonBody({ - id: like("123"), // Example value provided - createdAt: dateTime("yyyy-MM-dd'T'HH:mm:ss.SSSXXX", "2024-01-01T00:00:00.000Z") -}); -``` - -**❌ Bad:** -```ts -builder.jsonBody({ - id: uuid(), // No example value provided — new value on every run - createdAt: dateTime("yyyy-MM-dd'T'HH:mm:ss.SSSXXX") // Generates a new value each time -}); -``` - -> ℹ️ Tip: Always provide example values when using generators like `uuid()`, `timestamp()`, or `dateTime()` to ensure stable contract output. - - -### 🔒 4. Security - -#### ✅ Rule: Don’t include sensitive data in tests -Use matchers or fake data instead. - -**Explanation:** -Pact files may be published to shared brokers or VCS. Including secrets, credentials, or real customer data is a serious security risk. - -**✅ Good:** -```ts -builder.headers({ - Authorization: like("Bearer ") -}); -``` - -**❌ Bad:** -```ts -builder.headers({ - Authorization: "Bearer real-production-token" -}); -``` \ No newline at end of file +For a comprehensive set of rules covering test structure, naming, scope, maintainability, and security — with good and bad examples for each — see the [Consumer Best Practices guide](/consumer/best_practices). + diff --git a/website/docs/consumer/best_practices.md b/website/docs/consumer/best_practices.md new file mode 100644 index 000000000..a9dffd57f --- /dev/null +++ b/website/docs/consumer/best_practices.md @@ -0,0 +1,538 @@ +--- +title: Consumer best practices +--- + +## Best Practice Guide + +This guide defines best practices for writing high-quality Pact consumer unit tests. Each rule includes an example of good and bad practice to help users lint and review their own code. + +### 🧱 1. Test Structure and Naming + +#### ✅ Rule: Use meaningful test descriptions +These should describe the client’s intent, not just the request type. + +**Explanation:** +Descriptive names improve readability and make it easier for consumers and providers to understand what’s being tested. Avoid repeating technical details already expressed in the test code. + +**✅ Good:** +```ts +.uponReceiving("a request to get all products") +``` + +**❌ Bad:** +```ts +.uponReceiving("GET /products") // Redundant information already described by the Pact test +``` + +#### ✅ Rule: Use human-readable test cases (BDD-style) +Combine the provider state, request description, and response to form a natural-language sentence. + +**Explanation:** +This helps others (including non-developers) understand test intent, and is particularly useful when reviewing contracts in the Pact Broker UI. + +**✅ Good:** +```ts +.addInteraction() + .given("there are active products in the catalog") + .uponReceiving("a request to get all products") + .withRequest("GET", "/products") + .willRespondWith(200, ...) +``` + +Reads as: +> **Given** there are active products in the catalog, **upon receiving** a request to get all products, **the provider will respond with** HTTP 200 with a list of products. + +**❌ Bad:** +```ts +.addInteraction() + .uponReceiving("GET /products") + .withRequest("GET", "/products") + .willRespondWith(200, ...) +``` + + +#### ✅ Rule: Use a consistent test template +Adopt a known structure across your test suite. + +**Explanation:** +A consistent structure makes tests easier to follow, review, and maintain — especially in large teams or codebases. + +**✅ Good:** +```ts +// Top level - name of the API +describe("Thing API", () => { + const pact = new PactV4({ + consumer: "ThingConsumer", + provider: "ThingProvider", + spec: SpecificationVersion.SPECIFICATION_VERSION_V4, + }); + + // Level 2 - Describe block for the specific API endpoint + describe("GET /thing/:id", () => { + + // Level 3 - Test block for the specific test case + test("given a valid thing, returns 200", async () => { + await pact + .addInteraction() + .given("a thing with id 1 exists") + .uponReceiving("a request for thing with ID 1") + .withRequest("GET", "/thing/1", (builder) => { + builder.headers({ Accept: "application/json" }); + }) + .willRespondWith(200, (builder) => { + builder.jsonBody( + like({ + id: 1, + name: "Thing 1", + price: 100, + }) + ); + }) + .executeTest(async (mockserver) => { + const ThingAPI = new ThingAPI(mockserver.url); + const Thing = await ThingAPI.getThingById(1); + expect(Thing).toEqual({ + id: 1, + name: "Some 1", + price: 100, + }); + }); + }); + }); +}); +``` + +**❌ Bad:** +```ts +// Random test structure per case, hard to follow +describe("Pact Tests", () => { + const pact = new PactV4({ + consumer: "ThingConsumer", + provider: "ThingProvider", + spec: SpecificationVersion.SPECIFICATION_VERSION_V4, + }); + + // Not organised well, hard to see what has been tested + test("can call the API", async () => { + await pact + .addInteraction() + .uponReceiving("get a thing") + .withRequest("GET", "/thing/1", (builder) => { + builder.headers({ Accept: "application/json" }); + }) + ... + }); +}); +``` + +### 🧪 2. Test Design and Scope + +#### ✅ Rule: Bug catcher + +Only include tests that protect against real consumer breakages. + +**Explanation**: +Each test example in your contract should exist for a reason: to detect a breaking change in the provider that would actually impact the consumer. A good rule of thumb is to ask: *“If I remove this test, is there a type of provider change that could break the consumer and go unnoticed?”* If the answer is “no,” the test is unnecessary. Avoid adding examples just for coverage or realism — focus instead on what the consumer truly depends on. This keeps your contracts lean, meaningful, and maintainable. + +✅ Good example: +The consumer behaves differently depending on the `status` field returned by a decision API. Removing any of these tests would leave a branch of consumer logic untested. + +```ts +// decisionClient.ts – actual consumer logic +export function handleDecision(response: { status: string }) { + switch (response.status) { + case 'approved': + return '✅ Proceed'; + case 'rejected': + return '❌ Stop'; + case 'pending': + return '⏳ Await further input'; + default: + throw new Error(`Unhandled status: ${response.status}`); + } +} +``` + +```ts +it('handles approved decision', async () => { + await provider.addInteraction({ + uponReceiving: 'an approved decision', + withRequest: { method: 'GET', path: '/decision' }, + willRespondWith: { + status: 200, + body: { status: 'approved' } + } + }); + + const res = await getDecision(provider.mockService); + expect(handleDecision(res)).toBe('✅ Proceed'); +}); + +it('handles rejected decision', async () => { + await provider.addInteraction({ + uponReceiving: 'a rejected decision', + withRequest: { method: 'GET', path: '/decision' }, + willRespondWith: { + status: 200, + body: { status: 'rejected' } + } + }); + + const res = await getDecision(provider.mockService); + expect(handleDecision(res)).toBe('❌ Stop'); +}); + +it('handles pending decision', async () => { + await provider.addInteraction({ + uponReceiving: 'a pending decision', + withRequest: { method: 'GET', path: '/decision' }, + willRespondWith: { + status: 200, + body: { status: 'pending' } + } + }); + + const res = await getDecision(provider.mockService); + expect(handleDecision(res)).toBe('⏳ Await further input'); +}); +``` + +❌ Bad practice: +If we only tested `'approved'` responses, the `'rejected'` and `'pending'` branches would be untested. If the provider changed those values or response structure, the consumer would break without the contract test noticing. + + +#### ✅ Rule: Test the actual API Client +Always exercise the real consumer code in your contract tests. + +**Explanation**: +Contract tests should verify that your actual consumer code (e.g. your API client or data-access layer) behaves correctly when interacting with the provider. If you use a generic HTTP client (e.g. `fetch`, `axios`) directly in the test instead of invoking the real consumer logic, you're not truly testing the consumer. This creates blind spots: the contract may be valid, but your consumer might still fail at runtime. Worse, it may give the false impression that the contract is safe to verify on the provider side, even though the consumer has never been properly exercised. + +✅ Good example: +A test that imports your app’s API client and verifies how it behaves with mock provider responses. + +```js +// consumer.js – your actual app code +export async function getUser(apiClient, id) { + const response = await apiClient.get(`/users/${id}`); + return response.data; +} + +// consumer.pact.test.js – contract test +import { getUser } from './consumer'; + +it('gets a user', async () => { + await provider.addInteraction({ + uponReceiving: 'a request for a user', + withRequest: { + method: 'GET', + path: '/users/123', + }, + willRespondWith: { + status: 200, + body: { + id: '123', + name: 'Alice' + } + } + }); + + const result = await getUser(provider.mockService, '123'); + expect(result).toEqual({ id: '123', name: 'Alice' }); +}); +``` + +❌ Bad example: +A test that uses `axios` or `fetch` directly, bypassing your actual consumer logic. + +```js +import axios from 'axios'; + +it('gets a user (but bypasses consumer logic)', async () => { + await provider.addInteraction({ + uponReceiving: 'a request for a user', + withRequest: { + method: 'GET', + path: '/users/123', + }, + willRespondWith: { + status: 200, + body: { + id: '123', + name: 'Alice' + } + } + }); + + const response = await axios.get(`${provider.mockService.baseUrl}/users/123`); + expect(response.data).toEqual({ id: '123', name: 'Alice' }); +}); +``` + +Even though this test may pass and a contract is generated, the actual consumer code isn’t tested — which defeats the purpose of contract testing. + +#### ✅ Rule: Add one interaction per test +Each test should represent a single, isolated contract scenario - an *interaction*. + +**Explanation:** +This makes tests clearer, failures easier to diagnose and ensures interactions are independently verifiable. + + +**✅ Good:** +```ts +test("returns 200 when product exists", async () => { + await pact + .addInteraction() + .given("product with ID 1 exists") + .uponReceiving("a request for product with ID 1") + .withRequest("GET", "/products/1") + .willRespondWith(200, ...); +}); + +test("returns 404 when product does not exist", async () => { + await pact + .addInteraction() + .given("no product with ID 999 exists") + .uponReceiving("a request for product with ID 999") + .withRequest("GET", "/products/999") + .willRespondWith(404, ...); +}); +``` + +**❌ Bad:** +```ts +test("product endpoints", async () => { + // Multiple unrelated interactions in the same test block + await pact + .addInteraction() + .given("product with ID 1 exists") + .uponReceiving("a request for product with ID 1") + .withRequest("GET", "/products/1") + .willRespondWith(200, ...); + + await pact + .addInteraction() + .given("no product with ID 999 exists") + .uponReceiving("a request for product with ID 999") + .withRequest("GET", "/products/999") + .willRespondWith(404, ...); + + // Then run both assertions + const existing = await api.getProduct("1"); + const missing = await api.getProduct("999"); + + expect(existing).toEqual(...); + expect(missing).toEqual(...); +}); +``` + +#### ✅ Rule: Minimize assertions — assert contract, not behavior +Only assert what’s needed to verify the contract. + +**Explanation:** +Avoid including unrelated assertions (e.g. logging, side effects), which can introduce flakiness and clutter. + +**✅ Good:** +```ts +expect(response).toEqual([...]); +``` + +**❌ Bad:** +```ts +expect(response).toEqual([...]); +expect(logger.debug).toHaveBeenCalled(); +``` + +#### ✅ Rule: Only include data and operations used by the consumer +Test only what your consumer relies on. + +**Explanation:** +This keeps the contract minimal and avoids unnecessary coupling to unrelated parts of the API. Additional redundant assertions on the provider unnecessarily constrains change. + +**✅ Good:** +```ts +builder.jsonBody({ id: like("123"), name: like("Product 1"), price: like(37.25) }); +``` + +**❌ Bad:** +```ts +builder.jsonBody({ + id: like("123"), + name: like("Product 1"), + price: like(37.25), + availableStores: [...], + ... // Unused fields +}); +``` + +#### ✅ Rule: Focus on contract validation, not functional testing +Contract tests verify HTTP request/response structures, not business logic. + +**Explanation:** +System behavior and business logic should be tested separately. Pact tests should focus solely on API client behaviour. + +**✅ Good:** +```ts +test("returns products in correct format", async () => { + await pact + .addInteraction() + .uponReceiving("a request to get all products") + .withRequest("GET", "/products") + .willRespondWith(200, (builder) => { + builder.jsonBody([{ id: "1", name: "Product 1" }]); + }) + .executeTest(async (mockserver) => { + const products = await new API(mockserver.url).getAllProducts(); + expect(products).toEqual([{ id: "1", name: "Product 1" }]); + }); +}); +``` + +**❌ Bad:** +```ts +test("calculates total cart value", () => { + const cart = new Cart(); + cart.addItem({ price: 10 }); + expect(cart.total()).toBe(10); // functional test, not contract test +}); +``` + +### 🧭 3. Clarity and Maintainability + +#### ✅ Rule: Use provider states for clarity and maintainability +State setup should be explicit. + +**Explanation:** +This improves test clarity and helps the provider team replicate test conditions during verification. + +**✅ Good:** +```ts +.given("product with ID 1 exists") +.uponReceiving("a request for product with ID 1") +.withRequest("GET", "/products/1") +.willRespondWith(200, ...) +``` + +**❌ Bad:** +```ts +.uponReceiving("a request for product with ID 1") +.withRequest("GET", "/products/1") +.willRespondWith(200, ...) +// No provider state set — makes provider testing harder, reduces comprehension +``` + +#### ✅ Rule: Use matchers to reduce test fragility +Avoid asserting on exact values unless necessary. + +**Explanation:** +Using matchers like `like()` or `eachLike()` ensures the contract is flexible and tolerant to non-breaking changes. + +**✅ Good:** +```ts +eachLike({ id: like("123"), name: like("Product 1") }) +``` + +**❌ Bad:** +```ts +[ { id: "123", name: "Product 1" } ] // Makes the provider tests more brittle and coupled +``` + +#### 🧱 Avoid over-constraining the provider + +Only assert on formats or constraints that your consumer actually depends on. + +**Explanation**: +Avoid over-constraining the provider by specifying overly strict formats (e.g. using regex patterns) unless your consumer truly relies on them. While you might know how the provider currently behaves, asserting on things your consumer doesn’t need — like a specific ID format — can create unnecessary fragility. This can lead to false test failures if the provider changes in ways that don’t affect your consumer. Keep your assertions just strict enough to ensure your consumer continues to work as expected — and no stricter. + + +✅ OK: +```json +{ + "customerId": "ABC-123" +} +``` + +❌ Over-constrained: +```ts +{ + "customerId": regex("[A-Z]{3}-\d{3}", "ABC-123") // forces provider to constrain to a regex +} +``` + +Instead, keep your assertions just strict enough to ensure your consumer works as expected — and no stricter. This helps the provider evolve safely without false test failures. + +#### ✅ Rule: Avoid duplicate interaction descriptions +Make each interaction description unique. + +**Explanation:** +The Pact Broker de-duplicates interactions based on description + provider state. Re-using the same description can result in dropped tests. + +**✅ Good:** +```ts +.given("there are active products in the catalog") +.uponReceiving("a request for a single product") +.withRequest("GET", "/products/1") +.willRespondWith(200, ...) +... +.given("there are no active products in the catalog") +.uponReceiving("a request for a single product") // OK: provider state differentiates scenario +.withRequest("GET", "/products/1") +.willRespondWith(404, ...) +``` + +**❌ Bad:** +```ts +.uponReceiving("a request for a single product") // BAD: no provider state +.withRequest("GET", "/products/1") +.willRespondWith(200, ...) +... +.uponReceiving("a request for a single product") // BAD: no provider state, overrides above +.withRequest("GET", "/products/1") +.willRespondWith(404, ...) +``` + +#### ✅ Rule: Ensure Pact files are deterministic +Avoid dynamic data without fixed examples. + +**Explanation:** +Dynamic data can lead to flaky tests and noisy diffs. Always fix example values when using generators. + +**✅ Good:** +```ts +builder.jsonBody({ + id: like("123"), // Example value provided + createdAt: dateTime("yyyy-MM-dd'T'HH:mm:ss.SSSXXX", "2024-01-01T00:00:00.000Z") +}); +``` + +**❌ Bad:** +```ts +builder.jsonBody({ + id: uuid(), // No example value provided — new value on every run + createdAt: dateTime("yyyy-MM-dd'T'HH:mm:ss.SSSXXX") // Generates a new value each time +}); +``` + +> ℹ️ Tip: Always provide example values when using generators like `uuid()`, `timestamp()`, or `dateTime()` to ensure stable contract output. + + +### 🔒 4. Security + +#### ✅ Rule: Don’t include sensitive data in tests +Use matchers or fake data instead. + +**Explanation:** +Pact files may be published to shared brokers or VCS. Including secrets, credentials, or real customer data is a serious security risk. + +**✅ Good:** +```ts +builder.headers({ + Authorization: like("Bearer ") +}); +``` + +**❌ Bad:** +```ts +builder.headers({ + Authorization: "Bearer real-production-token" +}); +``` \ No newline at end of file diff --git a/website/docs/consumer/contract_tests_not_functional_tests.md b/website/docs/consumer/contract_tests_not_functional_tests.md index 0cb8282fc..6220c1809 100644 --- a/website/docs/consumer/contract_tests_not_functional_tests.md +++ b/website/docs/consumer/contract_tests_not_functional_tests.md @@ -6,7 +6,7 @@ Contract tests focus on the messages that flow between a consumer and provider, A more subtle distinction is required when it comes to contract testing interactions that don't have side effects, like validation error responses. -Imagine that we have a simple _User Service_ that allows Consumers to register new users, typically with a POST request containing the details of the created user in the body. +Consider a simple _User Service_ that allows consumers to register new users, typically with a POST request containing the details of the created user in the body. A simple happy-path scenario for that interaction might look like: @@ -28,11 +28,11 @@ Then Expected Response is 409 Conflict ``` -So far so good, we're covering a new behaviour, with a different response code. +So far so good — this covers a new behaviour with a different response code. -Now we've been talking to the Team managing the _User Service_ and they tell us that username has a maximum length of 20 characters, also they only allow letters in the username and a blank username is obviously not valid. Maybe that's something we should add in our contract? +Now, the team managing the _User Service_ has told you that usernames have a maximum length of 20 characters, allow only letters, and cannot be blank. That sounds like something to add to the contract. -This is where we get on the slippery slope... it's very tempting to now add 3 scenarios to our contract, something like: +This is the slippery slope — it is tempting to now add 3 scenarios to the contract, something like: ```text When "creating a user with a blank username" @@ -58,7 +58,7 @@ Then Expected Response body is { "error": "username can only contain letters" } ``` -We've gone past the contract testing at this point, we're actually testing that the _User Service_ implements the validation rules correctly: this is functional testing, and it should be covered by the _User Service_ in its own codebase. +At this point, you have gone past contract testing and are actually testing that the _User Service_ implements validation rules correctly: this is functional testing, and it should be covered by the _User Service_ in its own codebase. What is the harm in this... more testing is good, right? The issue here is that these scenarios are going too far and create an unnecessarily tight contract - what if the _User Service_ Team decides that actually 20 characters is too restrictive for username and increases it to 50 characters? What if now numbers are allowed in the username? Any Consumer should be unaffected by any of these changes, unfortunately the _Users Service_ will break our Pact just by loosening the validation rules. These are not breaking changes, but by over-specifying our scenarios we are stopping the _User Service_ Team from implementing them. @@ -72,7 +72,7 @@ Then Response body is { "error": "" } ``` -Subtle, but so much more flexible! Now the _User Service_ Team can change \(most\) of their validation rules without breaking the Pact we give them... we don't really care about each individual business rule, we only care that if we send something wrong, then we understand the way the _User Service_ responds to us. +Subtle, but so much more flexible! Now the _User Service_ team can change \(most\) of their validation rules without breaking the pact — the contract doesn't need to care about each individual business rule, only that if you send something wrong, you understand how the _User Service_ will respond. When writing a test for an interaction, ask yourself what you are trying to cover. Contracts should be about catching: @@ -80,7 +80,7 @@ When writing a test for an interaction, ask yourself what you are trying to cove * misunderstanding from the consumer about end-points or payload * breaking changes by the provider on end-points or payload -In short, your Pact scenarios should not dig into the business logic of the Provider but should stick with verifying that Consumer and Provider have a shared understanding of what requests and responses will be. In our example of validation, write scenarios about _how_ the validation fails, not _why_ the validation fails. +In short, your Pact scenarios should not dig into the business logic of the provider but should stick with verifying that consumer and provider have a shared understanding of what requests and responses will be. In the validation example above, write scenarios about _how_ validation fails, not _why_ it fails. ## Which test is responsible for what? diff --git a/website/docs/consumer/how_to_test_async_messages.md b/website/docs/consumer/how_to_test_async_messages.md new file mode 100644 index 000000000..a7cf90f81 --- /dev/null +++ b/website/docs/consumer/how_to_test_async_messages.md @@ -0,0 +1,130 @@ +--- +title: How to test async message integrations +--- + +This guide explains how to structure your code and your Pact tests to contract-test asynchronous message integrations — events and messages sent over queues or event buses such as AWS SNS, SQS, Kafka, RabbitMQ, or Kinesis. + +Before reading this, you should understand the [Message Pact concept](/getting_started/how_pact_works#non-http-testing-message-pact) and have checked [feature support](/roadmap/feature_support) for your language. + +## Separate your protocol adapter from your message handler + +The key prerequisite for testable async message code is a clean separation between: + +- **The adapter** — the code that knows about the specific queuing technology (SNS event shape, Kafka wrapper, AMQP envelope). This is untestable with Pact because Pact does not speak those protocols directly. +- **The port** — the code that handles the domain payload, unaware of the transport. This is what Pact tests. + +This is the Ports and Adapters pattern (also called Hexagonal Architecture). + +![Ports and Adapters architecture](/img/ports-and-adapters.png) + +Without this separation, your Pact test would have to unwrap the transport-specific envelope before it could test the message content — and that couples your test to infrastructure details that don't belong in a contract. + +## Consumer side + +### Identify your port + +Given a product event published through AWS SNS, the raw Lambda handler is the adapter: + +```js +const handler = async (event) => { + console.info(event); + + // Unwrap the SNS envelope and pass the payload to the handler + const results = event.Records.map((e) => receiveProductUpdate(JSON.parse(e.Sns.Message))); + + return Promise.all(results); +}; +``` + +The `receiveProductUpdate` function is the port — it deals only with the domain payload: + +```js +const receiveProductUpdate = (product) => { + console.log('received product:', product) + + // Handle the product event, e.g. store in the database + return repository.insert(new Product(product.id, product.type, product.name, product.version)) +} +``` + +Your Pact test targets `receiveProductUpdate`, not `handler`. This function expects a message of the following shape: + +```json +{ + "id": "some-uuid-1234-5678", + "type": "spare", + "name": "3mm hex bolt", + "version": "v1", + "event": "UPDATED" +} +``` + +### Write the consumer Pact test + +In your consumer test, describe the message your port expects to receive, then invoke the port with that message. Pact verifies that the message shape is recorded in the pact file: + +```js +// Example structure — exact syntax depends on your Pact library +const { message } = pact; + +describe('receiveProductUpdate', () => { + it('handles a product updated event', () => { + return message + .given('a product updated event exists') + .expectsToReceive('a product updated event') + .withContent({ + id: like('some-uuid-1234-5678'), + type: like('spare'), + name: like('3mm hex bolt'), + version: like('v1'), + event: 'UPDATED' + }) + .withMetadata({ contentType: 'application/json' }) + .verify(receiveProductUpdate); // pass the port function directly + }); +}); +``` + +The `verify` call invokes your port with the message content and confirms it does not throw. If the function handles the message without error, the interaction is recorded in the pact file. + +## Provider (producer) side + +On the provider side, find the port responsible for producing the message. In this example, `ProductEventService` publishes product events: + +```js +class ProductEventService { + async create(event) { + const product = productFromJson(event); + return this.publish(createEvent(product, "CREATED")); + } + + async update(event) { + const product = productFromJson(event); + return this.publish(createEvent(product, "UPDATED")); + } + + async publish(message) { + const SNS = new AWS.SNS({ + endpoint: process.env.AWS_SNS_ENDPOINT, + region: process.env.AWS_REGION + }); + + const params = { + Message: JSON.stringify(message), + TopicArn: TOPIC_ARN, + }; + + return SNS.publish(params).promise(); + } +} +``` + +The `update` method is the port — it produces the domain event. The `publish` method is the adapter — it knows how to send it to SNS. + +In the provider verification test, register a handler that calls `update` (the port) and returns the message it would produce. Pact replays the recorded consumer interaction against this handler and checks that the output matches the consumer's expectation. + +## Further reading + +- [Example consumer project (AWS SNS)](https://docs.pactflow.io/docs/examples/aws/sns/consumer) and [example provider project](https://docs.pactflow.io/docs/examples/aws/sns/provider) +- [Introduction to Async Messages workshop](/university/message-pact-async/00_1_Intro) — a step-by-step tutorial using Kafka +- [Message Pact concept](/getting_started/how_pact_works#non-http-testing-message-pact) diff --git a/website/docs/consumer/using_pact_to_support_ui_testing.md b/website/docs/consumer/using_pact_to_support_ui_testing.md index 4ba0b4abd..d387165e9 100644 --- a/website/docs/consumer/using_pact_to_support_ui_testing.md +++ b/website/docs/consumer/using_pact_to_support_ui_testing.md @@ -2,7 +2,7 @@ title: Using Pact to support UI testing --- -While we don't recommend you generate pacts from your UI tests, you can use Pact to support your UI testing using one of a few approaches. +Generating pacts from UI tests is not recommended, but you can use Pact to support your UI testing using one of a few approaches. ## 1. Use the generated pact file to run a Pact HTTP stub server diff --git a/website/docs/faq.md b/website/docs/faq.md index 0ff7ca3c8..7beb93c88 100644 --- a/website/docs/faq.md +++ b/website/docs/faq.md @@ -8,7 +8,7 @@ You can find the answers to some more technical and complex questions [here](/fa ::: -You might also like to look at our [Ask Me Anything videos](/help/amas), which often cover these questions in more detail. +You might also like to look at the [Ask Me Anything videos](/help/amas), which often cover these questions in more detail. ### What is Pact good for? @@ -24,9 +24,9 @@ Pact is most valuable for designing and testing integrations where you \(or your ### Who would typically implement Pact? -Pact is generally implemented by developers, during development. While some testers do write Pact tests, the code-first, white-box nature of Pact testing means that whoever writes the tests needs to have a strong understanding of the code under test, how to write code, how to use the existing testing libraries (eg. Jest, JUnit) and build tools (eg. Maven, npm), how to create and inject stubs, as well as how to use the Pact library itself. Testers who do not have this experience struggle with Pact, and we suggest pairing with a developer in this situation. Business analysts and testers can still benefit from the presence of contracts by using them to understand the underlying interactions between the applications. +Pact is generally implemented by developers, during development. While some testers do write Pact tests, the code-first, white-box nature of Pact testing means that whoever writes the tests needs to have a strong understanding of the code under test, how to write code, how to use the existing testing libraries (eg. Jest, JUnit) and build tools (eg. Maven, npm), how to create and inject stubs, as well as how to use the Pact library itself. Testers who do not have this experience struggle with Pact — pairing with a developer is recommended in this situation. Business analysts and testers can still benefit from the presence of contracts by using them to understand the underlying interactions between the applications. -The consumer team is responsible for implementing the Pact tests in the consumer codebase that will generate the contract, and for publishing it to a shared location \(usually a [Pact Broker](/pact_broker)\). The provider team is responsible for setting up the Pact verification task in the provider codebase, and for writing the code that sets up the correct data for each `provider state` described in the contract. Both teams are responsible for collaborating and communicating about the API and its usage! Remember that contracts are not a substitute for good communication between teams. +The consumer team is responsible for implementing the Pact tests in the consumer codebase that will generate the contract, and for publishing it to a shared location \(usually a [Pact Broker](/pact_broker)\). The provider team is responsible for setting up the Pact verification task in the provider codebase, and for writing the code that sets up the correct data for each `provider state` described in the contract. Both teams are responsible for collaborating and communicating about the API and its usage! Contracts are not a substitute for good communication between teams. ### What is the difference between contract testing and functional testing? @@ -53,18 +53,18 @@ Pact was written by a team that was using microservices that had read/write REST ### Why is there no support for specifying optional attributes? :::note -Refer to our [guide](/recipes/optional) on handling optional values. +Refer to the [guide](/recipes/optional) on handling optional values. ::: Firstly, it is assumed that you have control over the provider's data \(and consumer's data\) when doing the verification tests. If you don't, then maybe Pact is [not the best tool for your situation](/getting_started/what_is_pact_good_for). -Secondly, if Pact supports making an assertion that element `$.body.name` may be present in a response, then you write consumer code that can handle an optional `$.body.name`, but in fact, the provider gives `$.body.firstname`, no test will ever fail to tell you that you've made an incorrect assumption. Remember that a provider may return extra data without failing the contract, but it must provide at minimum the data you expect. +Secondly, if Pact supports making an assertion that element `$.body.name` may be present in a response, then you write consumer code that can handle an optional `$.body.name`, but in fact, the provider gives `$.body.firstname`, no test will ever fail to tell you that you've made an incorrect assumption. Note that a provider may return extra data without failing the contract, but it must provide at minimum the data you expect. The same goes for specifying "SOME\_VALUE or null". If all your provider verification test data returned nulls for this key, you might think that you had validated the "SOME\_VALUE", but in fact, you never had. You could get a completely different "SOME\_VALUE" for this key in production, which may then cause issues. The same goes for specifying an array with length 0 or more. If all your provider verification data returned 0 length arrays, all your verification tests would pass without you ever having validated the contents of the array. This is why you can only specify an array with minimum length 1 OR a zero length array. -Remember that unlike a schema, which describes all possible states of a document, Pact is "contract by examples". If you need to assert that multiple variations are possible, then you need to provide an example for each of those variations. Consider if it's _really_ important to you before you do add a Pact test for each and every variation however. Remember that each interaction comes with a "cost" of maintenance and execution time, and you need to consider if it is worth the cost in your particular situation. You may be better off handling the common scenarios in the pact, and then writing your consumer to code to gracefully handle unexpected variations (eg. by ignoring that data and raising an alert). +Remember that unlike a schema, which describes all possible states of a document, Pact is "contract by examples". If you need to assert that multiple variations are possible, then you need to provide an example for each of those variations. Consider if it's _really_ important to you before you do add a Pact test for each and every variation however. Each interaction comes with a cost of maintenance and execution time — consider whether it is worth it in your particular situation. You may be better off handling the common scenarios in the pact, and then writing your consumer code to gracefully handle unexpected variations (eg. by ignoring that data and raising an alert). ### Why are the pacts generated and not static? @@ -107,7 +107,7 @@ Contract tests replace a certain class of system integration test \(the ones you The value of contract tests is that they allow you to shift effort from high maintenance, slow feedback tests to low maintenance, fast feedback tests, reducing the overall effort required to release. -We often see end-to-end integration tests used as a catch all across integration, functional and acceptance testing (in the pyramid below, this would be represented as a bigger portion of the "E2E" part of the triangle). Specifically, we see them used as a proxy for [_provider functional tests_](/consumer/contract_tests_not_functional_tests). Separating the integration from the functional aspects often relieves end-to-end tests of a lot of their duties, and in some cases they can be replaced altogether. Read on to see how. +End-to-end integration tests are often used as a catch-all across integration, functional and acceptance testing (in the pyramid below, this would be represented as a bigger portion of the "E2E" part of the triangle). Specifically, they are often used as a proxy for [_provider functional tests_](/consumer/contract_tests_not_functional_tests). Separating the integration from the functional aspects often relieves end-to-end tests of a lot of their duties, and in some cases they can be replaced altogether. Read on to see how. **Watch a video: replacing end-to-end integration tests** @@ -142,7 +142,7 @@ If you work in a more traditional "Big Bang Release" environment, choose end to #### The Four Key Metrics -Lastly, we find that the [four key metrics](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance) from DORA that correlate to high performing organisations are great ways to measure and report on your contract testing initiative. These metrics are: +The [four key metrics](https://cloud.google.com/blog/products/devops-sre/using-the-four-keys-to-measure-your-devops-performance) from DORA that correlate to high-performing organisations are a great way to measure and report on your contract testing initiative. These metrics are: * Deployment frequency: how often do you release? * Lead time for change: how long does it take to get a release from from commit to production? @@ -199,7 +199,7 @@ PactFlow is a commercial fork of the Pact Broker, and adds features required to Pact is "consumer driven contracts", not "dictator driven contracts". Just because it's called "consumer driven" doesn't mean that the team writing the consumer gets to write a pact and throw it at the provider team without talking about it. The pact should be the starting point of a collaborative effort. -The way Pact works, it's the pact verification task \(in the provider codebase\) that fails when a consumer expects things that are different from what a provider responds with, even if the consumer itself is "wrong". This is a little unfortunate, but it's the nature of the beast. (See this page on [pending pacts](https://pact.io/pending) for how we've fixed this problem.) +The way Pact works, it's the pact verification task \(in the provider codebase\) that fails when a consumer expects things that are different from what a provider responds with, even if the consumer itself is "wrong". This is a little unfortunate, but it's the nature of the beast. (See this page on [pending pacts](https://pact.io/pending) for how this problem is addressed.) Running the pact verification task that gets triggered by the "contract content changed" webhook in a separate CI build from the rest of the tests for the provider is a good idea - if you have it in the same build, someone is going to get cranky about another team being able to break their build. @@ -273,9 +273,9 @@ There are multiple situations where you need to traverse more than 2 systems in Another common example is where one system calls out to another system first to fetch an authentication token such as a JWT. In this case, there is an API call from consumer A to auth provider B, which is then able to call auth server C. -Where possible, you should try to isolate interactions between two services at any one time. We would generally recommend stubbing out these systems. +Where possible, try to isolate interactions between two services at any one time. Stubbing out these systems is generally recommended. -See [https://gist.github.com/bethesque/43eef1bf47afea4445c8b8bdebf28df0](https://gist.github.com/bethesque/43eef1bf47afea4445c8b8bdebf28df0) for some more detail on how you might achieve this, and read our advice on [dealing with auth services](/provider/handling_auth). +See [https://gist.github.com/bethesque/43eef1bf47afea4445c8b8bdebf28df0](https://gist.github.com/bethesque/43eef1bf47afea4445c8b8bdebf28df0) for more detail on how you might achieve this, and read the advice on [dealing with auth services](/provider/handling_auth). ### How do I test auth cookies? @@ -290,13 +290,13 @@ In this instance you have a few options: \(1\) doesn't fully represent the true contract, but \(2\) would require more code/effort. -In this case, we suggest you need to weigh up the pros/cons. From a purely theoretical perspective, the answer is “you should include it”. But taking a more balanced view, we say test what gives you value. If the cookie is always going to be implicitly added by the browser \(because that’s how browser’s behave\) and it’s a scenario unlikely to give your team more \(useful\) information about how the system behaves, whilst costing you effort in maintaining it. Then maybe it’s not worth it. +In this case, weigh up the pros/cons. From a purely theoretical perspective, the answer is “you should include it”. But taking a more balanced view, test what gives you value. If the cookie is always going to be implicitly added by the browser \(because that’s how browser’s behave\) and it’s a scenario unlikely to give your team more \(useful\) information about how the system behaves, whilst costing you effort in maintaining it. Then maybe it’s not worth it. Sorry, life isn't black and white! ### Should the database or any other part of the provider be stubbed? -The pact authors' experience with using pacts to test microservices has been that using the set_up hooks to populate the database, and running the verifications with all the real provider code has worked very well, and gives us full confidence that the end to end scenario will work in the deployed code. +Experience with using pacts to test microservices has shown that using the set_up hooks to populate the database and running the verifications with all the real provider code works very well, giving full confidence that the end-to-end scenario will work in deployed code. However, if you have a large and complex provider, you might decide to stub some of your application code. You will definitely need to stub calls to downstream systems or to set up error scenarios. Make sure, if you stub, that you don't stub the code that actually parses the request and pulls the expected data out, because otherwise the consumer could be sending absolute rubbish, and the verification task won't fail because that code won't get executed. If the validation happens when you insert a record into the datasource, either don't stub anything, or rethink your validation code. @@ -330,7 +330,7 @@ See below, or the [Pact JS example](https://github.com/pact-foundation/pact-js/t ### I use GraphQL, SOAP, Protobufs ... do I need contract tests? -This is a hard one. All we can do is provide some general advice, which can be easily summarised as this: +This is a hard one. Here is some general advice, which can be summarised as this: > if there is a possibility that the provider and consumer can get out of sync, then contract tests can help @@ -338,14 +338,14 @@ GraphQL is simply an abstraction over HTTP, and it is entirely possible that the SOAP is the same. Yes, there is a strongly defined schema, however if the provider changes that schema and deploys before a consumer has updated, boom - client down. -Protobufs is something we are still thinking about, and we've yet to test it with Pact in the wild. It does appear unnecessary as it has mechanisms to deal with backwards compatibility - but if you're willing to investigate, please chat to us and tell us how you go :\) +Protobufs support in Pact is still being considered. It does appear unnecessary as it has mechanisms to deal with backwards compatibility — but if you're willing to investigate, share your findings in the [Pact Slack](https://slack.pact.io). ### How can I tell if I have good contract test coverage of my provider API? Contract tests aren't intended to provide any particular percentage coverage of the _provider_ code - that's what the provider's own functional tests are for. The important question to be asking is "what proportion of the consumer code _that makes the calls to the provider_ is covered?". If you execute your consumer Pact tests in a separate step in your test suite, you can use standard code coverage tools to determine whether or not your Pact tests have covered a sufficient percentage of your consumer client code. -While the coverage metric can be helpful, it unfortunately won't be able to tell you whether or not you've covered every semantic variation of an endpoint. Determining that is currently beyond the scope of Pact, but is something that we would love to be able to solve in the future. +While the coverage metric can be helpful, it won't tell you whether you've covered every semantic variation of an endpoint. Determining that is currently beyond the scope of Pact. ### Why does computer say no in the can-i-deploy tool? diff --git a/website/docs/faq/convinceme.md b/website/docs/faq/convinceme.md index 9feecf84d..d1f9b2d47 100644 --- a/website/docs/faq/convinceme.md +++ b/website/docs/faq/convinceme.md @@ -47,7 +47,7 @@ Read the following to un-convince yourself: * [http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam](http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam) * and if you're really keen, [Defect Analysis and Prevention for Software Process Quality Improvement](http://www.ijcaonline.org/volume8/number7/pxc3871759.pdf) -Research tells us that integration tests are more costly in terms of time, effort and maintenance without giving us any more guarantees. +Research shows that integration tests are more costly in terms of time, effort and maintenance without providing additional guarantees. **Watch a video** @@ -144,11 +144,9 @@ In fact, you don't even have to use Pact to implement contract testing and gain ## OK, I'm convinced but I can't convince my friends -Are you just saying that so we don't feel bad? - Here are some suggestions to win them over: * Watch some of the great talks over lunch with your team and some popcorn * [Ask](https://slack.pact.io) one of the pact contributors to do a brown-bag talk at your office or over hangouts -* [Chat](https://slack.pact.io) to us in real-time and we'll see if we can debate as your proxy +* [Chat in real-time on Pact Slack](https://slack.pact.io) — someone from the community can often help make the case diff --git a/website/docs/getting_started.md b/website/docs/getting_started.md index a345422a5..9e842d706 100644 --- a/website/docs/getting_started.md +++ b/website/docs/getting_started.md @@ -21,9 +21,14 @@ To view an animated step-by-step explanation of how Pact works, check out this [ Or, watch the [full series on contract testing](https://www.youtube.com/embed/videoseries?list=PLwy9Bnco-IpfZ72VQ7hce8GicVZs7nm0i). -## Ready to jump into the code already? +## Where would you like to go? -Get started with our [5 minute guide](5-minute-getting-started-guide.md). +| I want to… | Go to… | +|---|---| +| Follow a step-by-step tutorial | [Tutorials](/tutorials) — 5-minute quick start, Pact University workshops | +| Understand how Pact works | [Concepts](/concepts) — terminology, matching, provider states, sharing pacts | +| Solve a specific problem | [How-to Guides](/consumer) — best practices, CI/CD setup, recipes | +| Look up API or specification details | [Pact Docs](/implementation_guides/overview) — language clients, CLI, plugins | ## What is contract testing? @@ -39,7 +44,7 @@ Contract testing is immediately applicable anywhere where you have two services ## Contract testing terminology -In general, a contract is between a _consumer_ \(for example, a client that wants to receive some data\) and a _provider_ \(for example, an API on a server that provides the data the client needs\). In microservice architectures, the traditional terms _client_ and _server_ are not always appropriate -- for example, when communication is achieved through message queues. For this reason, we stick to _consumer_ and _provider_ in this documentation. +In general, a contract is between a _consumer_ \(for example, a client that wants to receive some data\) and a _provider_ \(for example, an API on a server that provides the data the client needs\). In microservice architectures, the traditional terms _client_ and _server_ are not always appropriate -- for example, when communication is achieved through message queues. For this reason, this documentation uses _consumer_ and _provider_ throughout. ## Consumer Driven Contracts diff --git a/website/docs/getting_started/conceptual_overview.md b/website/docs/getting_started/conceptual_overview.md index ae7d50d4a..358346d45 100644 --- a/website/docs/getting_started/conceptual_overview.md +++ b/website/docs/getting_started/conceptual_overview.md @@ -1,7 +1,8 @@ # Conceptual Overview -Using Pact successfully requires you to understand how Pact works and the concepts and terminology behind Pact. -If you try to build your CI/CD without this foundational understanding, you may find yourself getting confused or getting unexpected results. +This page explains the core domain model behind Pact: how consumer versions, pact versions, branches, environments, and verification results relate to each other. Understanding these concepts will help you reason about your CI/CD pipeline and avoid unexpected behaviour when publishing or verifying pacts. + +For an introduction to how Pact tests work, see [How Pact works](/getting_started/how_pact_works). For terminology definitions, see [Terminology](/getting_started/terminology). ## Consumers and publishing pacts @@ -41,7 +42,7 @@ Branches in the Pact Broker are designed to model repository (git, svn etc) bran The general recommendation is to set the branch name when you publish a pact, and set with the environment when you deploy. -I'll discuss later how we use branches and environments to help answer the question: can I deploy? +Branches and environments are used together to answer the deployment safety question: "can I deploy this version?" — see [can-i-deploy](/pact_broker/can_i_deploy) for how this works in practice. For example, let's say you have a PetStoreConsumer application. Every time you change it that's a new version of the application. When you run contract tests for that version, you publish a new pact publication to the broker, diff --git a/website/docs/getting_started/how_pact_works.md b/website/docs/getting_started/how_pact_works.md index 33ac6b0ab..9462e7de8 100644 --- a/website/docs/getting_started/how_pact_works.md +++ b/website/docs/getting_started/how_pact_works.md @@ -2,7 +2,7 @@ title: How Pact works --- -Remember these definitions from the [introduction](/): +The following definitions are used throughout this page (and align with the [introduction](/)): * **Consumer**: An application that makes use of the functionality or data from another application to do its job. For applications that use HTTP, the consumer is always the application that initiates the HTTP request \(eg. the web front end\), regardless of the direction of data flow. For applications that use queues, the consumer is the application that reads the message from the queue. * **Provider**: An application \(often called a service\) that provides functionality or data for other applications to use, often via an API. For applications that use HTTP, the provider is the application that returns the response. For applications that use queues, the provider \(also called _producer_\) is the application that writes the messages to the queue. @@ -82,11 +82,9 @@ In many cases, your provider will need to be in a particular state \(such as "us ## Putting it all together -Here’s a repeat of the two diagrams above: - ![Pact test and verify](/img/pact-test-and-verify.png) -If we pair the consumer test and provider verification process for each interaction, the contract between the consumer and provider is fully tested without having to spin up the services together. +Pairing the consumer test and provider verification process for each interaction means the contract between the consumer and provider is fully tested without having to spin up the services together. ## Non-HTTP testing (Message Pact) @@ -94,7 +92,7 @@ Modern distributed architectures are increasingly integrated in a decoupled, asy There are some minor differences between how Pact works in these cases when compared to the HTTP use case. Pact supports messages by abstracting away the protocol and specific queuing technology (such as Kafka) and focusses on the messages passing between them. -Check our [feature support](/roadmap/feature_support) to ensure your language has this capability. +Check the [feature support](/roadmap/feature_support) page to ensure your language has this capability. :::info To reiterate: Pact does not know about the various message queueing technologies - there are simply too many! And more importantly, Pact is really about testing the messages that pass between them, you can still write your standard functional tests using other frameworks designed for such things. @@ -102,100 +100,13 @@ To reiterate: Pact does not know about the various message queueing technologies When writing tests, Pact takes the place of the intermediary (MQ/broker etc.) and confirms whether or not the consumer is able to _handle_ a given event, or that the provider will be able to _produce_ the correct message. -### How to write "message pact" tests? - -We recommend that you split the code that is responsible for handling the protocol specific things - for example an AWS lambda handler and the AWS SNS input body - and the piece of code that actually handles the payload. - -You're probably familiar with layered architectures such as Ports and Adapters (also referred to as a Hexagonal architecture). Following a modular architecture will allow you to do this much more easily: - -![Ports and Adapters architecture](/img/ports-and-adapters.png) - -Let's walk through an example using a `product event` published through AWS SNS as an example. - -#### Consumer side - -The consumer expects to receive a message of the following shape: - -```json -{ - "id": "some-uuid-1234-5678", - "type": "spare", - "name": "3mm hex bolt", - "version": "v1", - "event": "UPDATED" -} -``` - -With this view, the "Adapter" will be the code that deals with the specific queue implementation. For example, it might be the lambda `handler` that receives the SNS message that wraps this payload, or the function that can read the message from a Kafka queue (wrapped in a Kafka specific container). Here is the lambda version: - -```js -const handler = async (event) => { - console.info(event); - - // Read the SNS message and pass the contents to the actual message handler - const results = event.Records.map((e) => receiveProductUpdate(JSON.parse(e.Sns.Message))); - - return Promise.all(results); -}; -``` - -The "Port" is the code (here `receiveProductUpdate`) that is unaware of the fact it's talking to SNS or Kafka, and only deals in the domain itself - in this case the `product event`. - -```js -const receiveProductUpdate = (product) => { - console.log('received product:', product) - - // do something with the product event, e.g. store in the database - return repository.insert(new Product(product.id, product.type, product.name, product.version)) -} -``` - -This function is the target of the Pact test on the consumer side. - -#### Provider (Producer) side - -On the other side, we need to find the "Port" that is responsible for _producing_ the message. In our case, we have a `ProductEventService` that is responsible for this: - -```js -class ProductEventService { - async create(event) { - const product = productFromJson(event); - return this.publish(createEvent(product, "CREATED")); - } - - async update(event) { - const product = productFromJson(event); - return this.publish(createEvent(product, "UPDATED")); - } - - ... - - async publish(message) { - const SNS = new AWS.SNS({ - endpoint: process.env.AWS_SNS_ENDPOINT, - region: process.env.AWS_REGION - }); - - const params = { - Message: JSON.stringify(message), - TopicArn: TOPIC_ARN, - }; - - return SNS.publish(params).promise(); - } -} -``` - -The `publish` is the bit ("Adapter") that knows how to talk to AWS SNS, the `update` is the bit ("Port") that just deals in our domain and knows how to create the specific event structure. This is the function on the provider side that we'll test is able to _produce_ the correct message structure. - -#### Further Reading +The key to testable message pact code is separating the _adapter_ (the code that knows about your specific queue technology — SNS, Kafka, RabbitMQ) from the _port_ (the code that handles the domain payload, unaware of the transport). Pact tests the port, not the adapter. -- Take a look at an [example consumer project](https://docs.pactflow.io/docs/examples/aws/sns/consumer) and its [example provider project](https://docs.pactflow.io/docs/examples/aws/sns/provider) to see this in action. -- Try out the [Pact Introduction to Async Messages workshop](https://docs.pact.io/university/message-pact-async/00_1_Intro) to see this in action with a Kafka example. +For a step-by-step guide on structuring and writing these tests, see [How to test async message integrations](/consumer/how_to_test_async_messages). ## Next steps -_Contract tests should focus on the messages \(requests and responses\) rather than the behaviour_. It can be tempting to use contract tests to write general functional tests for the provider. Experience shows this to leads to painful experiences with brittle tests. See [this guide for contract testing best practices](/consumer/contract_tests_not_functional_tests). +_Contract tests should focus on the messages \(requests and responses\) rather than the behaviour_. Using contract tests as general functional tests for the provider leads to brittle tests. See [this guide for contract testing best practices](/consumer/contract_tests_not_functional_tests). _Pact tests should be data independent_. Pact tests are best when successful verification doesn’t depend on the specific data that the provider returns. See [this guide](/consumer) for best practices when describing interactions. diff --git a/website/docs/getting_started/matching.md b/website/docs/getting_started/matching.md index b3480dd8a..75b48bb55 100644 --- a/website/docs/getting_started/matching.md +++ b/website/docs/getting_started/matching.md @@ -180,20 +180,10 @@ Pact.service_consumer "Zoo App" do end ``` -## Best practice +## Choosing matchers in practice -### Request matching +For guidance on when to use exact vs loose matching, and pitfalls to avoid, see [Writing Consumer tests](/consumer#choose-the-right-type-of-matching-for-the-situation). -As a rule of thumb, you generally want to use exact matching when you're setting up the expectations for a request \(`upon_receiving(...).with(...)`\) because you're under control of the data at this stage, and according to Postel's Law, we want to be "strict" with what we send out. Note that the request matching does not allow "unexpected" values to be present in JSON request bodies or query strings. \(It does however allow extra headers, because we found that writing expectations that included the headers added by the various frameworks that might be used resulted in tests that were very fiddly to maintain.\) - -### Response matching - -You want to be _as loose as possible_ with the matching for the response \(`will_respond_with(...)`\) though. This stops the tests being brittle on the provider side. Generally speaking, it doesn't matter what value the provider actually returns during verification, as long as the types match. When you need certain formats in the values \(eg. URLS\), you can use `terms` \(see docs below\). Really really consider before you start introducing too many matchers however - for example, yes, the provider might be currently returning a GUID, but would anything in your consumer really break if they returned a different format of string ID? \(If it did, that's a nasty code smell!\) Note that during provider verification, following Postel's Law of being "relaxed" with what we accept, "unexpected" values in JSON response bodies are ignored. This is expected and is perfectly OK. Another consumer may have an expectation about that field. - -### Random data - avoid it - -If you are using a Pact Broker to exchange pacts, then avoid using random data in your pacts. If a new pact is published that is exactly the same as a previous version that has already been verified, the existing verification results will be applied to the new pact publication. This means that you don't have to wait for the provider verification to run before deploying your consumer - you can go straight to prod. Random data makes it look like the contract has changed, and therefore you lose this optimisation. - -#### NOTE - -_If you are writing tests on the_ `Consumer` _side to a different language on the_ `Provider` _side, you must ensure you use a common Pact Specification between them or you will be unable to validate._ +:::note +If you are writing tests on the `Consumer` side against a different language on the `Provider` side, you must ensure you use a common Pact Specification between them or you will be unable to validate. +::: diff --git a/website/docs/getting_started/pact-file-format.md b/website/docs/getting_started/pact-file-format.md new file mode 100644 index 000000000..cdba0ea72 --- /dev/null +++ b/website/docs/getting_started/pact-file-format.md @@ -0,0 +1,254 @@ +--- +title: Pact file format +--- + +A pact file is a JSON document that describes the interactions between a consumer and a provider. It is generated automatically by the consumer's Pact tests and used as the input for provider verification. This page describes the structure of a pact file across the supported specification versions. + +The authoritative specification is maintained at [github.com/pact-foundation/pact-specification](https://github.com/pact-foundation/pact-specification). + +## Top-level structure + +All pact files share the same top-level shape: + +```json +{ + "consumer": { "name": "string" }, + "provider": { "name": "string" }, + "interactions": [...], + "metadata": { + "pactSpecification": { "version": "string" } + } +} +``` + +| Field | Type | Description | +|---|---|---| +| `consumer.name` | string | The name of the consumer service that generated this pact. | +| `provider.name` | string | The name of the provider service this pact is for. | +| `interactions` | array | The list of interactions recorded during consumer tests. | +| `metadata.pactSpecification.version` | string | The Pact specification version used (e.g. `"2.0.0"`, `"3.0.0"`). | + +## HTTP interactions (V1 / V2) + +Pact specification V1 and V2 describe synchronous HTTP request/response interactions. + +```json +{ + "description": "A request for a product", + "providerState": "product 10 exists", + "request": { + "method": "GET", + "path": "/products/10", + "headers": { + "Accept": "application/json" + }, + "query": "flag=true" + }, + "response": { + "status": 200, + "headers": { + "Content-Type": "application/json" + }, + "body": { + "id": 10, + "name": "Wooden Spoon", + "type": "cutlery" + }, + "matchingRules": { + "$.body.id": { "match": "type" } + } + } +} +``` + +### Interaction fields (V1/V2) + +| Field | Type | Required | Description | +|---|---|---|---| +| `description` | string | Yes | Human-readable label for this interaction. Must be unique within the pact file. | +| `providerState` | string | No | A named state the provider must be in before this interaction is replayed. | +| `request` | object | Yes | The HTTP request the consumer expects to make. | +| `response` | object | Yes | The HTTP response the consumer expects to receive. | + +### Request fields + +| Field | Type | Description | +|---|---|---| +| `method` | string | HTTP method (e.g. `GET`, `POST`, `DELETE`). Case-insensitive. | +| `path` | string | The request path. May contain matching rules in V2+. | +| `headers` | object | Request headers as key-value pairs. | +| `query` | string (V1) / object (V2) | Query string. V1 stores as a raw string; V2 stores as a map of arrays. | +| `body` | any | The request body. May be a JSON object, array, string, or absent. | +| `matchingRules` | object | Matching rules applied to request fields. See [Matching rules](#matching-rules). | + +### Response fields + +| Field | Type | Description | +|---|---|---| +| `status` | integer | The expected HTTP status code. | +| `headers` | object | Expected response headers. Only headers listed here are verified. | +| `body` | any | The expected response body. May be a JSON object, array, string, or absent. | +| `matchingRules` | object | Matching rules applied to response fields. | + +## HTTP interactions (V3 / V4) + +V3 introduced several breaking changes and additions. V4 extended the format further to support plugins. + +Key changes from V2: + +- `providerState` (singular string) is replaced by `providerStates` (array of objects). +- `generators` are added alongside `matchingRules` to support dynamic value generation during verification. +- `query` is always an object (map of arrays), not a raw string. + +```json +{ + "description": "A request for a product", + "providerStates": [ + { + "name": "product 10 exists", + "params": { "id": 10 } + } + ], + "request": { + "method": "GET", + "path": "/products/10", + "matchingRules": { + "path": { "matchers": [{ "match": "regex", "regex": "/products/[0-9]+" }] } + }, + "generators": { + "path": { "type": "RandomInt", "min": 1, "max": 100 } + } + }, + "response": { + "status": 200, + "body": { + "id": 10, + "name": "Wooden Spoon" + }, + "matchingRules": { + "body": { + "$.id": { "matchers": [{ "match": "type" }] } + } + } + } +} +``` + +### Provider states (V3+) + +| Field | Type | Description | +|---|---|---| +| `name` | string | The name of the provider state. Must match a state registered in the provider's verification setup. | +| `params` | object | Optional key-value parameters passed to the provider state handler. | + +## Message interactions (V3) + +Message pacts describe asynchronous interactions — events and messages passed over a queue or event bus. There is no HTTP request; instead, the interaction describes only the message content. + +```json +{ + "description": "a product updated event", + "providerStates": [ + { "name": "product 10 exists" } + ], + "contents": { + "id": 10, + "event": "UPDATED", + "name": "Wooden Spoon" + }, + "metadata": { + "contentType": "application/json" + }, + "matchingRules": { + "body": { + "$.id": { "matchers": [{ "match": "type" }] } + } + }, + "generators": {} +} +``` + +| Field | Type | Description | +|---|---|---| +| `description` | string | Human-readable label for this message. | +| `providerStates` | array | Named states the provider must be in to produce this message. | +| `contents` | any | The expected message payload. | +| `metadata` | object | Message metadata, including `contentType`. | +| `matchingRules` | object | Matching rules applied to message fields. | +| `generators` | object | Generators for dynamic field values. | + +## Matching rules + +Matching rules control how Pact compares actual values to expected values during verification. Without a matching rule, Pact uses exact equality. + +### V1/V2 format + +```json +"matchingRules": { + "$.body.id": { "match": "type" }, + "$.body.name": { "match": "regex", "regex": "[A-Za-z ]+" } +} +``` + +Rules are keyed by a JSONPath expression. Supported match types: + +| `match` value | Description | +|---|---| +| `type` | The actual value must be the same type as the expected value. | +| `regex` | The actual value must match the `regex` field. | +| `integer` | The actual value must be an integer. | +| `decimal` | The actual value must be a decimal number. | +| `number` | The actual value must be any number. | +| `min` | The array must have at least `min` items. | +| `max` | The array must have at most `max` items. | + +### V3+ format + +```json +"matchingRules": { + "body": { + "$.id": { + "matchers": [{ "match": "type" }], + "combine": "AND" + } + } +} +``` + +In V3+, rules are nested by section (`body`, `header`, `path`, `query`) and each key maps to an object with a `matchers` array. The `combine` field (`AND`/`OR`) controls how multiple matchers are applied. + +## Generators + +Generators are used during provider verification to substitute dynamic values (such as random IDs or current timestamps) into requests and responses. + +```json +"generators": { + "body": { + "$.id": { "type": "RandomInt", "min": 1, "max": 1000 } + } +} +``` + +Common generator types: + +| `type` | Description | +|---|---| +| `RandomInt` | Generates a random integer between `min` and `max`. | +| `RandomDecimal` | Generates a random decimal number. | +| `RandomString` | Generates a random string of `size` characters. | +| `Regex` | Generates a value matching the provided `regex`. | +| `Uuid` | Generates a random UUID. | +| `Date` | Generates a date using the provided `format`. | +| `DateTime` | Generates a datetime using the provided `format`. | +| `ProviderStateGenerator` | Reads a value from the provider state `params` map. | + +## Specification versions + +| Version | Key additions | +|---|---| +| V1 | HTTP interactions. `providerState` as a string. `matchingRules` keyed by JSONPath. | +| V2 | Regular expression and type matching. `query` as an object. | +| V3 | `providerStates` as an array of objects. Message interactions. `generators`. V3 matching rule format. | +| V4 | Plugin support. Combined interaction types (`Synchronous/HTTP`, `Asynchronous/Messages`, `Synchronous/Messages`). | + +For the full specification, see [Pact Specification on GitHub](https://github.com/pact-foundation/pact-specification). diff --git a/website/docs/getting_started/provider_states.md b/website/docs/getting_started/provider_states.md index 1d81fc355..716925c5d 100644 --- a/website/docs/getting_started/provider_states.md +++ b/website/docs/getting_started/provider_states.md @@ -15,7 +15,7 @@ So how do you test a request that requires data to already exist on the provider Provider states also allow the consumer to make the same request with different expected responses \(e.g. different response codes, or the same resource with a different subset of data\). -Keep in mind that a provider state is all about the state of the _provider_ \(eg. what data is there, how it is going to handle a given response\), not about the state of the consumer, or about what is in the request. +A provider state is all about the state of the _provider_ \(eg. what data is there, how it is going to handle a given response\), not about the state of the consumer, or about what is in the request. Refer to the the [Ruby example](../implementation_guides/ruby/provider_states.md) to see how this plays out in code. diff --git a/website/docs/getting_started/sharing_pacts.md b/website/docs/getting_started/sharing_pacts.md index 99b780169..b0eae8ffa 100644 --- a/website/docs/getting_started/sharing_pacts.md +++ b/website/docs/getting_started/sharing_pacts.md @@ -6,9 +6,9 @@ The consumer CI generates pact files during the execution of its isolated tests. > **Quick Start** > -> The Pact Broker is an open source tool that requires you to deploy, administer and host yourself. If you would prefer a plug-and-play option, we've created [PactFlow](https://pactflow.io/?utm_source=ossdocs&utm_campaign=sharing_pacts), a fully managed Pact Broker with additional features to simplify teams getting started and scaling with Pact. +> The Pact Broker is an open source tool that requires you to deploy, administer and host yourself. If you would prefer a plug-and-play option, [PactFlow](https://pactflow.io/?utm_source=ossdocs&utm_campaign=sharing_pacts) is a fully managed Pact Broker with additional features to simplify teams getting started and scaling with Pact. > -> Get started for free on our Developer Plan [here](https://pactflow.io/pricing/?utm_source=&utm_campaign=sharing_pacts_dev_plan). +> Get started for free on the [Developer Plan](https://pactflow.io/pricing/?utm_source=&utm_campaign=sharing_pacts_dev_plan). ## Pact Broker diff --git a/website/docs/getting_started/testing-scope.md b/website/docs/getting_started/testing-scope.md index 11576ea6c..698786a63 100644 --- a/website/docs/getting_started/testing-scope.md +++ b/website/docs/getting_started/testing-scope.md @@ -10,13 +10,13 @@ Or, watch the [full series on contract testing](https://www.youtube.com/embed/vi ## Scope of a Consumer Pact Test -Ideally, the Pact tests should be "unit tests" for your client class, and they should just focus on ensuring that the request creation and response handling are correct. If you use pact for your UI tests, you'll end up with an explosion of redundant interactions that will make the verification process tedious. Remember that pact is for testing the contract used for communication, and not for testing particular UI behaviour or business logic. +Ideally, the Pact tests should be "unit tests" for your client class, and they should just focus on ensuring that the request creation and response handling are correct. If you use pact for your UI tests, you'll end up with an explosion of redundant interactions that will make the verification process tedious. Pact is for testing the contract used for communication, not for testing particular UI behaviour or business logic. Usually, your application will be broken down into a number of sub-components, depending on what type of application your consumer is \(e.g. a Web application or another API\). This is how you might visualise the coverage of a consumer Pact test: ![Scope of a consumer Pact test](/img/consumer-test-coverage.png) -Here, a _Collaborator_ is a component whose job is to communicate with another system. In our case, this is the `OrderApiClient`communicating with the external `Order Api` system. This is what we want our consumer test to inspect. +Here, a _Collaborator_ is a component whose job is to communicate with another system. In this example, the `OrderApiClient` communicates with the external `Order Api` system — this is what the consumer test inspects. ## Scope of a Provider Pact Test diff --git a/website/docs/getting_started/verifying_pacts.md b/website/docs/getting_started/verifying_pacts.md index 2aa0a1e98..ae01637a0 100644 --- a/website/docs/getting_started/verifying_pacts.md +++ b/website/docs/getting_started/verifying_pacts.md @@ -2,9 +2,9 @@ title: Verifying Pacts --- -"Verifying a pact" is the second step of the Pact testing process. Each request in the pact file is replayed against the provider, and the response is compared with the expected response in the pact file. If the two match, then we know the consumer and provider are compatible. +"Verifying a pact" is the second step of the Pact testing process. Each request in the pact file is replayed against the provider, and the response is compared with the expected response in the pact file. If the two match, the consumer and provider are compatible. -To verify a pact, we must: +To verify a pact, you need to: 1. Configure the location of the pact to be verified. This can be a HTTP URL, or a local file system path. 2. Set up the data for the [provider states](provider_states.md). diff --git a/website/docs/getting_started/what_is_pact_good_for.md b/website/docs/getting_started/what_is_pact_good_for.md index 94b7ecde9..6982a1eea 100644 --- a/website/docs/getting_started/what_is_pact_good_for.md +++ b/website/docs/getting_started/what_is_pact_good_for.md @@ -40,7 +40,7 @@ Pact is fantastic tool for developing and testing intra-organisation microservic If you use Pact to test a public API, the only way to set up the right provider state is to use the very API that you're actually testing, which will make the tests slower and more brittle compared to the "normal" pact verification tests. Resetting the state between interactions may be quite slow, as you may need to iterate over many APIs to clear previously created data. An alternative might be to modify requests so that they work on a new context each time. -If this is still a better situation for you than integration testing, or using another tool, then you can give it a go! We strongly recommend you do not try to rely on state set up by previous interactions, as this will inevitably lead to flakey tests with dependencies that are hard to reason about. Reliably ordering the interactions within the pact or during replay is not supported in all implementations. +If this is still a better situation for you than integration testing, or using another tool, you can give it a go. Do not rely on state set up by previous interactions — this will inevitably lead to flaky tests with dependencies that are hard to reason about. Reliably ordering interactions within the pact or during replay is not supported in all implementations. ### Why Pact may not be the best tool for testing pass through APIs like BFFs diff --git a/website/docs/getting_started/why-pact.md b/website/docs/getting_started/why-pact.md new file mode 100644 index 000000000..4b93063f1 --- /dev/null +++ b/website/docs/getting_started/why-pact.md @@ -0,0 +1,87 @@ +--- +title: Why Pact? +--- + +Contract testing exists because distributed systems are hard to test safely. This page explains the design decisions behind Pact: why the contract is generated by the consumer, why the pact file is a machine-readable artefact rather than a hand-written document, and why the Pact Broker model exists. These aren't arbitrary choices — each one is a deliberate answer to a specific, well-understood failure mode. + +## The core problem: testing distributed systems + +When two services need to communicate, you need confidence that: + +1. The consumer sends a request the provider will understand. +2. The provider returns a response the consumer will handle correctly. + +The traditional answer is an integration test: deploy both services into a shared environment and exercise them together. This works, but it has well-documented costs: + +- **You can only test after deployment.** Bugs are found late, in environments that are expensive to provision and maintain. +- **Failures are ambiguous.** When an integration test fails, the cause could be in either service, or in the environment itself. +- **Tests are not owned.** No single team is responsible for the health of the full integration test suite. +- **Environments drift.** The version combination you test is rarely the exact combination that ends up in production. + +Pact trades the integration environment for a shared contract. Each side tests against a local mock — consumer against a mock provider, provider against the recorded consumer expectations — and the contract file guarantees that the mocks stay in sync with the real implementations. + +## Why the consumer generates the contract + +In Pact, the pact file is not written by hand. It is generated automatically as a side effect of the consumer running its own tests. + +This is the "consumer-driven" part of consumer-driven contract testing. It has two important consequences: + +**The contract reflects actual consumer behaviour.** A hand-written specification can drift from what the consumer code actually sends. A generated contract cannot — if the consumer code changes, the next test run produces a new contract that reflects the change. + +**The contract captures only what the consumer needs.** Hand-written API specifications tend to describe the full API surface. A generated consumer contract only records the fields and response shapes that the consumer actually uses. This means the provider knows exactly which parts of its API are load-bearing for each consumer, and can safely evolve the rest without breaking anything. + +This is the mechanism that makes it safe for the provider team to remove fields, refactor responses, and evolve their API — as long as the verification step passes, no known consumer will break. + +## Why the contract is stored as a file + +The pact file is a JSON document that both sides can access independently. It has no runtime dependency on either service being available. This is what allows the two sides of the test to run in isolation — each team in their own CI pipeline, on their own schedule. + +The file format is also versioned and language-neutral. Any Pact implementation, regardless of language, can read and write a pact file. This is why a consumer written in JavaScript can be verified against a provider written in Go without either team sharing code. + +## Why consumer-driven, not provider-driven + +An alternative model is for the provider team to publish their API schema (in OpenAPI, for example) and for consumers to validate against it. This is provider-driven. + +The practical problem with provider-driven contracts is that publishing a schema tells consumers what the API _can_ do, not what they _should_ rely on. Consumers have to read the schema and manually work out whether a provider change breaks their usage. That analysis is manual, error-prone, and often doesn't happen until deployment. + +Consumer-driven contracts invert this. The consumer declares what it needs. The provider runs the consumer's expectations as its own tests. If the provider breaks a consumer's expectation, the provider's test suite fails — before deployment. The feedback loop is tight and automatic. + +This also addresses a common team-dynamics problem. Provider teams often don't know which of their consumers use which parts of the API. Consumer-driven contracts make this explicit: you can look at the pacts for a provider and see exactly which consumer calls which endpoint with which fields. + +## Why provider states exist + +Pact tests are designed to be independent of each other. Each interaction is tested in isolation — which means there is no database of shared state that builds up over the course of a test run. + +The challenge is that many API responses depend on prior data. "Return the order with ID 123" only works if an order with ID 123 exists. Provider states are the mechanism for expressing this dependency without making tests order-dependent. + +When the consumer writes a test, it declares: "given [some named state], when [this request] is made, expect [this response]." The provider registers a handler for each named state that creates whatever data is needed before the request is replayed. This keeps each interaction independent while allowing realistic response scenarios. + +Provider states also serve as documentation. A list of provider states for a given provider is, in effect, a list of all the scenarios that consumers depend on — which is useful for understanding which test data setups are actually required in production. + +## Why the Pact Broker exists + +Without the broker, sharing pact files requires teams to figure out their own mechanism — typically committing pact files to a shared repository, or passing them through build artefacts. This works at small scale, and is a perfectly valid way to start. + +The Pact Broker exists to solve the problems that arise as the number of services and teams grows: + +- **Which version of the pact should the provider verify against?** The broker knows which consumer versions are deployed to which environment, and can answer "verify the pact version currently in production" automatically. +- **Is it safe to deploy this version?** The `can-i-deploy` command queries the broker's verification matrix and tells you, before you deploy, whether the version you're releasing is compatible with what's already in each environment. +- **Who uses what?** The broker provides network graphs and dashboards that show the relationships between all registered services, making it easier to reason about the impact of changes. + +The broker is not required to use Pact. It is the piece that converts Pact from a testing tool into a deployment safety mechanism. + +## Trade-offs and limits + +Consumer-driven contract testing is not a replacement for all other forms of integration testing. It is best suited to: + +- Teams that control both the consumer and the provider. +- Services that are in active development on both sides. +- Organisations where consumer teams can manage a direct relationship with provider teams. + +It is less suited to: + +- Public APIs with unknown or uncontrolled consumers. +- APIs where the provider team cannot set up test data directly (without calling the API under test). +- Pass-through services that validate nothing before forwarding a request. + +For a detailed breakdown of when Pact is and isn't the right tool, see [When to use Pact](/getting_started/what_is_pact_good_for). diff --git a/website/docs/pact_broker/can_i_deploy.md b/website/docs/pact_broker/can_i_deploy.md index db168f265..3e77e332e 100644 --- a/website/docs/pact_broker/can_i_deploy.md +++ b/website/docs/pact_broker/can_i_deploy.md @@ -4,26 +4,9 @@ description: How to use Pact + the can-i-deploy tool to ensure that you are safe toc_max_heading_level: 4 --- -Before you deploy a new version of an application to a production environment, you need to know whether or not the version you're about to deploy is compatible with the versions of the other apps that already exist in that environment. The old-fashioned way of managing these dependencies involved deploying sets of pre-tested applications together, creating a bottleneck, and meaning that speedy development and testing on one application may be negated by slow development and testing on another. +Before you deploy, you need to confirm that the version you're releasing is compatible with what's already running in the target environment. The `can-i-deploy` tool queries the Pact Broker's verification matrix and tells you whether it's safe to proceed. -The Pact way of managing these dependencies is to use the Pact "Matrix" and the `can-i-deploy` tool. The "Matrix" is the grid created when you create a table of all the consumer and provider versions that have been tested against each other using Pact. \(When a pact is published, the version of the consumer that generated the pact is recorded. When a pact is verified against a provider, the verification results are published to the Pact Broker, along with the version of the provider that verified the pact. When you put all of the consumer versions and provider versions that have been tested against each other into a table, you end up with the "Pact Matrix".\) - -You can view the Pact Matrix for any pair of applications by clicking on the little grid icon for your pact in the Pact Broker index page. - -Imagine the pact verification matrix for consumer Foo and provider Bar. It shows the consumer version, the provider version, and whether or not the verification passed. - -| Foo version \(consumer\) | Bar version \(provider\) | Verification success? | -| :--- | :--- | :--- | -| 22 | 56 | true | -| 23 | 56 | true | -| 23 | 57 | false | -| 23 | 58 | true | -| 24 | 58 | true | -| 25 | 58 | false | - -So how does this help us? Well, if we know that version 56 of Bar is already in our prod environment, looking at the matrix tells us that we are safe to deploy version 22 or 23 to prod, but not any of the versions after. Conversely, if we know that version of 24 of Foo is in prod, then we know we're safe to deploy version 58 of Bar, but not any of the versions before. - -Let's see how the Pact Matrix helps us deploy safely in practice. +For an explanation of how the Pact Matrix works and why it exists, see the [Pact Broker overview](/pact_broker/overview). In the deployment script for each application that uses Pact, we need to add a step after the successful deployment that notifies the Pact Broker of the event. Depending on which version of the Pact Broker you are using, you will notify the Broker in a different way. The latest versions of the Pact Broker support the [`record-deployment`](/pact_broker/recording_deployments_and_releases/#recording-deployments) and [`record-release`](/pact_broker/recording_deployments_and_releases/#recording-releases) commands. Older versions of the Pact Broker use "tags" to keep track of deployments. See the [section below](#using-can-i-deploy-with-tags) if you are using an version of the Broker that does not support recording deployments. diff --git a/website/docs/pact_nirvana.md b/website/docs/pact_nirvana.md index d8baa81fb..b02681007 100644 --- a/website/docs/pact_nirvana.md +++ b/website/docs/pact_nirvana.md @@ -5,9 +5,9 @@ sidebar_label: Introduction Or _"The steps for reaching Pact Nirvana"_ -This is a technical guide for developers and testers who want to use Pact to implement consumer driven contract testing as part of your ongoing CI/CD workflow. +This is a **how-to guide** for developers and testers who want to integrate Pact into their CI/CD workflow. Each step addresses a specific task — getting a test running, connecting the Pact Broker, wiring up pipelines, and enabling deployment safety gates. -By the end of the guide, you will understand how to create a release pipeline that allows you to independently deploy any application with the confidence that it will work correctly with the other applications in its environment - without having to run a suite of end to end tests. +By the end, you will have a release pipeline that allows you to independently deploy any application with the confidence that it will work correctly with the other applications in its environment — without running a suite of end-to-end tests. This guide focuses on the scenario where the consumer and provider are both being deployed to an environment rather than released to customers (e.g. a mobile app). That workflow is slightly different, and that guidance will be coming soon. diff --git a/website/docs/pact_nirvana/step_1.md b/website/docs/pact_nirvana/step_1.md index 1da5ca43a..81ece604a 100644 --- a/website/docs/pact_nirvana/step_1.md +++ b/website/docs/pact_nirvana/step_1.md @@ -3,15 +3,18 @@ title: 1. Get prepared - learn about Pact sidebar_label: 1. Get prepared - learn about Pact --- -Before you read this document, you should: +## Recommended reading -* have a basic understanding of the concepts of both [consumer driven contracts](https://martinfowler.com/articles/consumerDrivenContracts.html) and Pact, -* have read the [Pact Broker Overview](pact_broker/overview.md) -* have read the section on [versioning in the Pact Broker](getting_started/versioning_in_the_pact_broker.md) -* Do the CI/CD workshop +Before working through this guide, make sure you have: -Before setting up Pact in your own environment, it can help to have an idea of what a working pipeline might look like. +* A basic understanding of [consumer-driven contracts](https://martinfowler.com/articles/consumerDrivenContracts.html) and how Pact works — see [How Pact works](/getting_started/how_pact_works) if you need a refresher. +* Read the [Pact Broker Overview](/pact_broker/overview) — this guide assumes you understand what the broker is and why it exists. +* Read [Versioning in the Pact Broker](/getting_started/versioning_in_the_pact_broker) — branch and environment concepts are used throughout. -Completing the [Pact CI/CD workshop](https://docs.pactflow.io/docs/workshops/ci-cd/) will give you a good understanding of how the Pact client libraries interact with the Pact Broker and where these interactions fit into your release pipeline. +## Work through the CI/CD workshop first (optional but recommended) -> Note that if you are using your own instance of the open source Pact Broker, it does not support secrets, and it does not have a user interface for managing [webhooks](https://docs.pact.io/pact_broker/webhooks). You'll need to use the API or HAL Browser to create the webhook, and your CI token will have to be stored in plain text in the webhook. See the Webhooks API reference docs [here](https://docs.pact.io/pact_broker/advanced_topics/api_docs/webhooks) +Before configuring Pact in your own environment, it helps to see what a working pipeline looks like. The [Pact CI/CD workshop](https://docs.pactflow.io/docs/workshops/ci-cd/) gives you a hands-on run through of the full pipeline — from writing a consumer test to deploying safely with `can-i-deploy`. + +:::note +If you are using your own instance of the open source Pact Broker, it does not support secrets and does not have a UI for managing [webhooks](/pact_broker/webhooks). You will need to use the API or HAL Browser to create webhooks, and your CI token will be stored in plain text in the webhook. See the [Webhooks documentation](/pact_broker/webhooks) for details. +::: diff --git a/website/docs/pact_nirvana/step_3.md b/website/docs/pact_nirvana/step_3.md index 5c29c7924..ebeb6aa3d 100644 --- a/website/docs/pact_nirvana/step_3.md +++ b/website/docs/pact_nirvana/step_3.md @@ -22,7 +22,7 @@ The simplest first steps you can take ## Tasks 1. Write a Pact test in your consumer project \(consult [the documentation](/implementation_guides/cli) for your chosen language\) and generate the pact. -2. Ship the pact file to the provider team via some manual mechanism (we will introduce the concept of a pact broker in a later step) +2. Ship the pact file to the provider team via some manual mechanism (the Pact Broker is introduced in a later step) 3. Verify the pact against the provider \(again, consult the documentation for your chosen language\) 4. Talk about it. 5. Decide whether or not Pact is the right tool for you, taking into account that there are situations for which Pact is [a good solution](/faq/#what-is-pact-good-for), and situations for which Pact is [not a good solution](/faq/#what-is-pact-not-good-for). @@ -72,9 +72,9 @@ Usually it won't at the first attempt unless you're pretty lucky. So now you ca ### Additional Notes -We also have features such as: +Pact also includes features such as: - provider states - matchers -However before diving in, follow this guide to make sure you understand how the pact generation and verification steps work, these will form part of our Honours course. +Before diving in, follow this guide to make sure you understand how the pact generation and verification steps work — these form part of the higher-level steps in this guide. diff --git a/website/docs/pact_nirvana/step_4.md b/website/docs/pact_nirvana/step_4.md index 278e6c354..4413a2ec9 100644 --- a/website/docs/pact_nirvana/step_4.md +++ b/website/docs/pact_nirvana/step_4.md @@ -3,7 +3,7 @@ title: 4. Silver - manually integrate with Pact Broker sidebar_label: 4. Silver - manually integrate with Pact Broker --- -Now you have two different sets of tests in two different codebases. The artifacts that tie these tests together are the Pact file, and the verification results. The next step is to automate the exchange of these artifacts. At this level, you still run on your machine, but you have the consumer test publish its pact to the Pact Broker, and manually run provider verification where we will verify the pact from the broker via URL, rather than pointing to our local file system. We will then create a new verification task, which will retrieve the same pact but via consumer version selectors. +Now you have two different sets of tests in two different codebases. The artifacts that tie these tests together are the Pact file, and the verification results. The next step is to automate the exchange of these artifacts. At this level, you still run on your machine, but you have the consumer test publish its pact to the Pact Broker, and manually run provider verification, pulling the pact from the broker via URL rather than from the local file system. You will then create a new verification task that retrieves the same pact via consumer version selectors.
Silver diagram @@ -35,7 +35,7 @@ graph LR; > Sharing is caring Now that you have created and run your consumer tests, producing a contract \(the pact file\) as an artefact. -You've shared it with the team responsible for managing the provider manually, and they've confirmed they meet all of the expectations set in it. We need a mechanism to ensure that we can eliminate the manual step of sharing our contracts which will be invaluable as our code changes and evolves. +You've shared it with the team responsible for managing the provider manually, and they've confirmed they meet all of the expectations set in it. A mechanism is needed to eliminate the manual sharing step, which becomes invaluable as the code changes and evolves. There are multiple ways to [share pacts](getting_started/sharing_pacts.md), but the recommended approach is to use a Pact Broker which is a service that allows your projects to exchange pacts and verification results in an automated way enabling powerful automation workflows. @@ -69,7 +69,7 @@ There are two reasons that a verification task will need to be run: 2. When the pact changes -* We recommend that a separate provider verification pipeline is set up to verify just the changed pact, this will utilise our verification step created in step 3. +* A separate provider verification pipeline is recommended to verify just the changed pact, utilising the verification step created in step 3. ### Consumer pipeline @@ -141,7 +141,7 @@ The rest of this guide assumes you are using a Pact Broker or PactFlow Broker. #### Pact Broker ->You'll find the quickest way to run the Pact-Broker locally is via our [example docker-compose setup](https://docs.pact.io/pact_broker/docker_images/pactfoundation#running-with-docker-compose) +>The quickest way to run the Pact-Broker locally is via the [example docker-compose setup](https://docs.pact.io/pact_broker/docker_images/pactfoundation#running-with-docker-compose) >There are alternative mechanisms listed on the page, such as [OpenShift](https://docs.pact.io/pact_broker/docker_images/pactfoundation#running-with-openshift), or a [Helm Chart](https://docs.pact.io/pact_broker/kubernetes/readme) for Kubernetes #### PactFlow Broker @@ -149,8 +149,7 @@ The rest of this guide assumes you are using a Pact Broker or PactFlow Broker. > You can sign up for PactFlow's free Starter Plan [here](https://pactflow.io/pricing/?utm_source=ossdocs&utm_campaign=five_minute_guide_dev_plan) which will set you up with your own SaaS PactFlow Broker. :::info -For the purposes of this guide to Pact Nirvana, we will not use any PactFlow specific features, so you can safely trial a POC and revert -back to an Open Source Pact Broker at the end. +This guide does not use any PactFlow-specific features, so you can safely trial a POC and revert to an open source Pact Broker at the end. ::: #### Next Steps @@ -168,7 +167,7 @@ Now that you have your Pact Broker available, you can start taking advantage of communication between your provider and consumer. * Ensure you follow the recommended configuration for publishing -* Use one of our [Pact CLI tools](https://docs.pact.io/pact_broker/client_cli) +* Use one of the [Pact CLI tools](https://docs.pact.io/pact_broker/client_cli) 1. [Docker](https://hub.docker.com/r/pactfoundation/pact-cli) 2. [Pact Standalone CLI](https://github.com/pact-foundation/pact-ruby-standalone/releases) 3. [Pact Broker Client (Ruby)](https://github.com/pact-foundation/pact_broker-client) @@ -180,11 +179,11 @@ Debug this until it works and looks good:- - Your consumer pact is published - It has the agreed consumer and provider team names displayed -- It has the correct consumer version (ideally relating to a git sha. We recommend [`absolute-version`](https://www.npmjs.com/package/absolute-version)) +- It has the correct consumer version (ideally relating to a git sha — [`absolute-version`](https://www.npmjs.com/package/absolute-version) is recommended) - It has a branch associated with it :::tip -Although some Pact implementations allow for publishing pacts as a wrapper around the Pact cli tools, these may be outdated, inconsistent, or absent in some implementations. It is our recommendation that you use one of the tools described above in the list +Although some Pact implementations allow for publishing pacts as a wrapper around the Pact CLI tools, these may be outdated, inconsistent, or absent in some implementations. The tools described in the list above are recommended. ::: ### C. Manually verify the pact by URL using the Pact Broker @@ -192,7 +191,7 @@ Although some Pact implementations allow for publishing pacts as a wrapper aroun :::tip This task should be run when the pact changes -* We recommend that a separate provider verification pipeline is set up to verify just the changed pact. This will utilise our verification step created in step 3 +* A separate provider verification pipeline is recommended to verify just the changed pact. This uses the verification step created in step 3. ::: Now you can see if you can run your provider tests, this time pulling the pact file not from your local filesystem, @@ -203,7 +202,7 @@ but from the broker. 3. Run your tests and they should retrieve the pact file from your Broker, and successfully verify it. 4. This verification task by url, will be used by webhooks, will be triggered, whenever a consumer contract involving our provider, that requires verification is published. Our webhook will be configured in a later step. -See our recommendations for this task [here](https://docs.pact.io/provider/recommended_configuration#verification-triggered-by-a-contract-requiring-verification-published): +See the recommendations for this task [here](https://docs.pact.io/provider/recommended_configuration#verification-triggered-by-a-contract-requiring-verification-published): - Set the URL as a configurable property such as `PACT_URL` environment variable. - Configure publication of verification results by a `CI` flag so that verification results are only published from `CI` systems @@ -223,12 +222,12 @@ This task should be run when the provider code changes * The verification task will fetch and verify all the relevant pacts from all consumers from the Pact Broker to ensure no regressions have occurred. ::: -Pact verification should run as part of your providers regular unit test run. We use consumer version selectors to determine which pacts to select. +Pact verification should run as part of your provider's regular unit test run. Consumer version selectors are used to determine which pacts to verify. 1. Configure a __new__ provider task which instead of using the Pact URL, uses `consumer-version-selectors`. 2. In the provider verification configuration, setup the consumer version selectors, so the pact that is being verified is the latest for targeted branches and later environments. This will help keep your provider builds green. - 1. Our [recommended configuration](https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors#recommended) is here. + 1. See the [recommended configuration](https://docs.pact.io/pact_broker/advanced_topics/consumer_version_selectors#recommended). 3. Configure publication of verification results by a `CI` flag so that verification results are only published from `CI` systems @@ -236,7 +235,7 @@ Pact verification should run as part of your providers regular unit test run. We they relate to a known build. 5. Run the provider task, it should fetch pacts matching your consumer version selectors and verify them as before - 1. `{ "mainBranch": true }` assuming our consumer was published from `main` / `master` / `master` - see [docs](https://docs.pact.io/pact_broker/branches#automatic-main-branch-detection) for setup in your pact-broker. + 1. `{ "mainBranch": true }` assuming the consumer was published from `main` / `master` - see [docs](https://docs.pact.io/pact_broker/branches#automatic-main-branch-detection) for setup in your pact-broker. ### E. Enable WIP and Pending Pacts @@ -266,9 +265,9 @@ Refer to the [WIP](/pact_broker/advanced_topics/wip_pacts) and [Pending](/pact_b ### Notes -In these examples, we will publish pacts and verify from our local machine. +In these examples, pacts are published and verified from a local machine. -In our next step, we will show Pact integration in your CI/CD system. +The next step covers Pact integration in your CI/CD system. Traditionally users would not publish from their local machines, and when running local verification tasks would not publish results to the Pact Broker. Developers should utilise read only based authentication mechanisms to enforce this. diff --git a/website/docs/pact_nirvana/step_5.md b/website/docs/pact_nirvana/step_5.md index f0af0b6d4..65b5216ed 100644 --- a/website/docs/pact_nirvana/step_5.md +++ b/website/docs/pact_nirvana/step_5.md @@ -37,7 +37,7 @@ By using branches alongside application versions in the broker allows you to sep To achieve this, when a pact is published, the associated pacticipant version should be published with a branch identifier that will be used by the provider to differentiate between the stable, safe pacts \(eg. branch "master"\) and the potentially breaking pacts \(eg. branch "feat-new-foobar"\). -To maintain a green build in your provider’s CI, we will use consumer version selectors to verify the main branch rather than the latest overall pact. +To maintain a green build in your provider’s CI, use consumer version selectors to verify the main branch rather than the latest overall pact. If you use feature branches for your consumer development, it is recommended to publish the pact with the [pacticipant](https://docs.pact.io/getting_started/terminology#pacticipant) version and the name of the branch. If you use feature toggles, the branch could be the name of the feature toggle. Your Pact client library will allow you to configure the name of the branch to be applied when you publish your pacts. @@ -66,11 +66,11 @@ Pact verification by consumer version selectors should be part of your regular u 1. As per step 4, configure your provider build to fetch the pact(s) from the broker with consumer version selectors and publish the verification results as part of its main build \(consult the documentation for your chosen language\). This would typically happen after the unit tests and before deploying to a test environment. You can find the recommended configuration [here](/provider/recommended_configuration#verification-triggered-by-provider-change). 2. In the provider verification configuration, ensure the pact that is being verified from the latest pact for the main branch (see the relevant documentation for your library). This will help keep your provider builds green. - 1. `{ "mainBranch": true }` assuming our consumer was published from `main` / `master` - see [docs](https://docs.pact.io/pact_broker/branches#automatic-main-branch-detection) for setup in your pact-broker. + 1. `{ "mainBranch": true }` assuming the consumer was published from `main` / `master` - see [docs](https://docs.pact.io/pact_broker/branches#automatic-main-branch-detection) for setup in your pact-broker. ### Notes -Although some language-specific Pact tools \(eg Gradle\), provide methods for publishing, we recommend utilsing one of our [Pact CLI tools](https://docs.pact.io/pact_broker/client_cli) +Although some language-specific Pact tools \(eg Gradle\) provide methods for publishing, using one of the [Pact CLI tools](https://docs.pact.io/pact_broker/client_cli) is recommended: 1. [Docker](https://hub.docker.com/r/pactfoundation/pact-cli) 2. [Pact Standalone CLI](https://github.com/pact-foundation/pact-ruby-standalone/releases) 3. [Pact Broker Client (Ruby)](https://github.com/pact-foundation/pact_broker-client) diff --git a/website/docs/pact_nirvana/step_6.md b/website/docs/pact_nirvana/step_6.md index 83c1d376c..aecd5d0a5 100644 --- a/website/docs/pact_nirvana/step_6.md +++ b/website/docs/pact_nirvana/step_6.md @@ -3,9 +3,9 @@ title: 6. Platinum - Add can-i-deploy with branch to PR pipelines sidebar_label: 6. Platinum - Add can-i-deploy with branch to PR pipelines --- -Before we merge a PR, it would be good to know if this change is compatible with the consumers and/or providers the application talks to. +Before merging a PR, it is valuable to know whether the change is compatible with the consumers and/or providers the application talks to. -We can answer this question using the **can-i-deploy** feature of Pact. +The **can-i-deploy** feature of Pact answers this question. ## The Matrix The Pact way of managing these dependencies is to use the Pact Matrix - this is the matrix generated when you create a table of all the consumer and provider versions that have been tested against each other using Pact. You can view the Pact Matrix for any pair of applications by opening `/matrix/provider/PROVIDER/consumer/CONSUMER` in your Pact Broker. @@ -32,13 +32,13 @@ Another reason that an extra "safe to deploy?" check is required (as well as the ## Can I Deploy? The `can-i-deploy` tool is a CLI that has been written to query the Matrix to ensure that you are safe to deploy. You can read a detailed description of how it works [here](/pact_broker/can_i_deploy/). -Ultimately, we will want to ask if you can deploy to a particular environment. But a good first step in that direction is to just make sure your change is compatible with what your consumers and providers have currently checked into their main branch. +Ultimately, the goal is to ask whether you can deploy to a particular environment. A good first step is to make sure your change is compatible with what your consumers and providers have currently checked into their main branch. -This is a great step, but it's not fully what we want. It's quite possible that the pact version used by the application checked into the main branch is not the same as the pact version running in production or even staging. The more frequently you deploy, and the faster you deploy, the less likely this is, but it's definitely a possibility. +This is a valuable step, but it is not the full picture. It's quite possible that the pact version used by the application checked into the main branch is not the same as the pact version running in production or even staging. The more frequently you deploy, and the faster you deploy, the less likely this is — but it is definitely a possibility. This is much more the case for native apps, where you will likely have many old pact versions running on user's mobile devices. Any providers those apps talk to need to be compatible with all those versions. -In these cases, checking to see if your provider is compatible with the consumer version checked into main is not sufficient. We'll talk about how to check against what is actually deployed in a particular environment in the next section. +In these cases, checking to see if your provider is compatible with the consumer version checked into main is not sufficient. The next section covers how to check against what is actually deployed in a particular environment. ### Add `can-i-deploy` to consumer PR pipeline diff --git a/website/docs/pact_nirvana/step_7.md b/website/docs/pact_nirvana/step_7.md index 63d9e8f5e..2ccd88c47 100644 --- a/website/docs/pact_nirvana/step_7.md +++ b/website/docs/pact_nirvana/step_7.md @@ -9,9 +9,9 @@ Environments allow you to ensure backwards compatibility between production and Before you deploy to a production environment, you need to know whether or not your app is compatible with the versions of the other apps that already exist in that environment. The old-fashioned way of managing these dependencies involved deploying sets of pre-tested applications together, creating a bottleneck and meaning that speedy development and testing on one application may be negated by slow development and testing on another. -## Let's record our deployments +## Record deployments -Now we're ready to tell the pact-broker when we deploy a version of our application, and switch `can-i-deploy` to use environment names rather than branch names. +At this step, you configure the pact-broker to be notified when a version of your application is deployed, and switch `can-i-deploy` to use environment names rather than branch names.
Diamond level diagram @@ -36,17 +36,11 @@ sequenceDiagram ```
-Note how in the PR pipeline we are checking to see if we can deploy to all the environments that we deploy to from -the main branch. The rationale for this is that we don't want to merge something into our main branch which we know -will break once we deploy - we want our main branch to stay clean and not get blocked by broken builds. If you -deploy to other environments such as preprod, you would want to check that environment as well. +Note how in the PR pipeline, the check is against all environments that are deployed to from the main branch. The rationale is that you don't want to merge something into the main branch that will break on deploy — the main branch should stay clean and not get blocked by broken builds. If you deploy to other environments such as preprod, check those environments as well. -Then, when we're actually ready to deploy to an environment, we check again. We do this because the version of the -other side of the contract could have changed since the PR validation ran. Doing the check in the PR pipeline -reduces the chance of breakage, but it's still a possibility, and we want to check for that before we deploy. +Then, when you are actually ready to deploy to an environment, check again. The version on the other side of the contract could have changed since the PR validation ran. Checking in the PR pipeline reduces the chance of breakage, but it remains a possibility — so check again just before deploying. -Note also that the first step in our commit pipeline is to let the broker know that a particular version of the -provider or consumer is now in the main branch. +Note also that the first step in the commit pipeline is to let the broker know that a particular version of the provider or consumer is now in the main branch. ### Add `record-deployment` to your commit pipelines @@ -68,12 +62,12 @@ deploy to an environment, to make sure it's safe to deploy. The final step is to make sure before you merge a PR that you can successfully deploy to all the environments that are targeted from the main branch (staging, prod, preprod, etc.) -### What if we use a release branch? +### What if you use a release branch? -If you use a release branch, then you do not deploy to prod and preprod from your main branch, so you can eliminate +If you use a release branch, you do not deploy to prod and preprod from your main branch, so you can eliminate that check on the PR pipeline. -However, before you cut a release branch, you want to check if you can deploy to preprod and prod. +However, before cutting a release branch, check whether you can deploy to preprod and prod. The general idea is: diff --git a/website/docs/plugins/concepts.md b/website/docs/plugins/concepts.md new file mode 100644 index 000000000..1c45086a9 --- /dev/null +++ b/website/docs/plugins/concepts.md @@ -0,0 +1,130 @@ +--- +title: How plugins work +--- + +The Pact Plugin Framework extends Pact beyond HTTP and asynchronous messages to support any transport or content type your architecture requires. This page explains the conceptual model behind plugins — what they are, how they integrate with the Pact core, and when it makes sense to build or use one. + +## Why plugins exist + +Pact was originally designed around RESTful HTTP interactions. Over time, three forces pushed against that constraint: + +1. **New transports** — gRPC, WebSockets, MQTT, and event streaming platforms require different interaction models than HTTP. +2. **New content types** — Protobuf, Avro, and GraphQL cannot be described using the standard JSON-based matching rules. +3. **The maintainability problem** — Adding native support for each new protocol in every Pact client library (JavaScript, Java, Go, Python, etc.) is slow, and puts a heavy burden on core maintainers. + +The plugin framework solves all three. Rather than building each capability into the core, it defines a standard interface that external plugins can implement. Any new protocol, transport, or content type can be supported without waiting for core maintainers to build it, and without each language library needing its own implementation. + +## The three interaction types + +Pact has three interaction models. Plugins can extend any of them: + +| Interaction type | Description | Examples | +|---|---|---| +| **Synchronous/HTTP** | The consumer sends a request; the provider returns a response. | REST, GraphQL over HTTP | +| **Asynchronous/Messages** | One-way events. Producer publishes; consumer reads. | SNS, SQS, Kafka, AMQP | +| **Synchronous/Messages** | Bidirectional or streaming. | gRPC (streaming), WebSockets | + +A plugin can add support for a new _transport_ (how messages are sent) or a new _content type_ (how message bodies are serialised and matched). + +## System actors and how they fit together + +When a Pact test runs with a plugin, the following actors interact: + +``` +Your test code + └─> Test framework (Jest, JUnit, Go test, ...) + └─> Pact client library (pact-js, pact-jvm, pact-go, ...) + └─> Pact Core (Rust FFI) or Java Plugin Driver + └─> Plugin Driver + └─> Plugin process + └─> System under test +``` + +1. **Your test code** defines the expected interaction using the Pact DSL. +2. **The Pact client library** passes the interaction definition to the Pact Core. +3. **The Pact Core** (a shared Rust library, accessed via FFI) manages the test lifecycle and delegates to the Plugin Driver for plugin-capable interactions. +4. **The Plugin Driver** discovers, starts, and manages the plugin process. +5. **The Plugin process** is an external binary that handles the protocol-specific work: + - For transport plugins: serves as the mock server during consumer tests and sends verification requests to the provider. + - For content plugins: serialises, deserialises, and applies matching rules to the message body. +6. **The pact file** is updated to record the plugin name and version, the interaction type, and any protocol-specific metadata the plugin needs. + +Plugins run as separate processes. They communicate with the Pact Core over gRPC using the Pact Plugin Protocol. This isolation means a plugin crash cannot crash the test runner. + +## Two types of plugins + +### Transport plugins + +A transport plugin provides support for a new network protocol. It takes over the roles normally handled by Pact's built-in HTTP engine: + +- During **consumer testing**: acts as the mock server, serving responses the consumer expects. +- During **provider verification**: sends the recorded interactions to the real provider and compares the responses. + +Example: the Protobuf/gRPC plugin provides a mock gRPC server for consumer tests and a gRPC verification client for provider tests. + +### Content matcher plugins + +A content matcher plugin provides support for a new data serialisation format. It handles: + +- **Serialisation and deserialisation** of the body into a comparable form. +- **Matching rules** specific to the format (e.g. Protobuf field paths, Avro schema types). +- **Generators** for producing dynamic test values in the format. + +Content matcher plugins are typically combined with a standard transport (e.g. Protobuf over HTTP, Avro over Kafka). + +## How the pact file records plugin interactions + +When a plugin is used, the pact file includes additional metadata that the provider verification step needs: + +```json +{ + "interactions": [ + { + "description": "a product event", + "type": "Asynchronous/Messages", + "contents": { ... }, + "pluginConfiguration": { + "contentType": "application/protobuf", + "descriptorKey": "ProductEvent" + } + } + ], + "metadata": { + "pactSpecification": { "version": "4.0" }, + "plugins": [ + { "name": "protobuf", "version": "0.3.15" } + ] + } +} +``` + +The `plugins` metadata entry tells the verifier which plugin to load before replaying the interaction. If the required plugin is not installed, verification will fail with a clear error. + +## When to use a plugin vs when to build one + +**Use an existing plugin** if one already exists for your protocol. Check the [Plugin Directory](/plugins/directory) first. Installing a plugin takes a few minutes; building one takes significantly longer. + +**Build a plugin** if: +- No plugin exists for your protocol or content type. +- You need private or proprietary protocol support. +- You want to add a new Pact capability without waiting for a core maintainer to implement it. + +**You do not need a plugin** for most common use cases: +- Standard REST/HTTP → use core Pact with HTTP interactions. +- JSON over Kafka/SNS/SQS → use core Pact with async message interactions. +- GraphQL over HTTP → use a content matcher plugin or handle as a standard HTTP POST. + +## What plugins cannot do + +Plugins operate within the Pact interaction model. They cannot: + +- Change how Pact tracks versions, publishes pacts, or records verification results. +- Replace the Pact Broker or the can-i-deploy workflow. +- Test business logic on the provider — verification still only checks the response matches the consumer's recorded expectation. + +## Next steps + +- See available plugins in the [Plugin Directory](/plugins/directory). +- Try an existing plugin with the [Plugin Quick Start](/plugins/quick_start). +- Build your own with the [Create a Plugin workshop](/plugins/workshops/create-a-plugin/intro). +- Read the plugin design documents in [Pact Docs](/implementation_guides/pact_plugins) for the full technical specification. diff --git a/website/docs/plugins/directory.md b/website/docs/plugins/directory.md index 4522a6f80..3319ae02d 100644 --- a/website/docs/plugins/directory.md +++ b/website/docs/plugins/directory.md @@ -30,7 +30,7 @@ These templates come with all the boilerplate out the way, so you can concentrat ## Demos -We have built out some demos based on a few scenarios that you can run on your machine or in browser via our interactive tutorials. +The following demos cover a few scenarios you can run on your machine or in the browser via interactive tutorials. 1. [Area Calculator](https://github.com/pact-foundation/pact-plugins/tree/main/examples/gRPC/area_calculator) 1. [Proto file](https://github.com/pact-foundation/pact-plugins/blob/main/examples/gRPC/area_calculator/proto/area_calculator.proto) diff --git a/website/docs/plugins/quick_start.md b/website/docs/plugins/quick_start.md index 122d5810d..b81f80169 100644 --- a/website/docs/plugins/quick_start.md +++ b/website/docs/plugins/quick_start.md @@ -12,7 +12,7 @@ The Pact Plugin Framework was created in order to: * Expand support to the widest range of use cases * Improve time-to-market for new features -* Grow our community (users + contributors) +* Grow the community (users + contributors) The framework allows users to extend Pact by creating new types of: @@ -33,7 +33,7 @@ The combination of these interaction types with new transports and protocols, en ## Conceptual Overview -Diving a bit deeper, let's look at the various system actors and how they work together with Plugins. +The following diagram shows the various system actors and how they work together with plugins. ![Conceptual Overview](./workshops/create-a-plugin/assets/conceptual-overview.png) @@ -56,12 +56,12 @@ Diving a bit deeper, let's look at the various system actors and how they work t - Try out a pre-made plugin - Download the [Pact CLI tool](/plugins/directory#plugin-tooling) for managing plugins - Visit the [Pact Plugin Directory](/plugins/directory) and check if one already exists - - Check out our [demo applications](/plugins/directory#demos) that you can clone and run on your machine. + - Check out the [demo applications](/plugins/directory#demos) that you can clone and run on your machine. - Build your own plugin for any use case you and your team require. - - Check out our Pact University [workshop](/plugins/workshops/create-a-plugin/intro), where you will create and publish your first distributable Pact Plugin. + - Check out the Pact University [workshop](/plugins/workshops/create-a-plugin/intro), where you will create and publish your first distributable Pact Plugin. - You can [run it in your browser](https://killercoda.com/pactflow/scenario/create-a-plugin) too! - Check out some of the [early protoype plugins](/plugins/directory#plugin-prototypes). - - Grab one of our [Pact Plugin Starter kits](/plugins/directory#template-plugins) + - Grab one of the [Pact Plugin Starter kits](/plugins/directory#template-plugins) - Check out the docs for authors - Writing a plugin [guide](https://docs.pact.io/implementation_guides/pact_plugins/docs/writing-plugin-guide) - Plugins that provide protocol transport implementations - See [Protocol design docs](https://docs.pact.io/implementation_guides/pact_plugins/docs/protocol-plugin-design). @@ -78,13 +78,13 @@ Diving a bit deeper, let's look at the various system actors and how they work t ## FAQ -### Why did we build the Pact Plugin Framework? +### Why was the Pact Plugin Framework built? Loved by thousands of development teams globally, Pact was originally created to support the rise of RESTful microservices and has since expanded to support asynchronous messaging, becoming the defacto API contract testing solution. As architectures have evolved, organisations find that the existing Pact contract testing framework may not support all of their use cases. -The industry has continued to innovate since Pact was created in 2013, and RESTful microservices are now only one of the key use cases today. According to SmartBear’s 2021 State of Quality [report](https://smartbear.com/state-of-software-quality/api/tools/#api-protocols), we are seeing the growth of: +The industry has continued to innovate since Pact was created in 2013, and RESTful microservices are now only one of the key use cases today. According to SmartBear’s 2021 State of Quality [report](https://smartbear.com/state-of-software-quality/api/tools/#api-protocols), the industry has seen the growth of: - Protocols such as Protobufs and GraphQL (80% of organisations run multi-protocol and more than 60% of organisations manage three or more) - Transports such as gRPC, Websockets and MQTT @@ -96,7 +96,7 @@ Read the full use case for the Plugin Framework is explained in detail on the [G ### When did it go live? -It is live today as of December 1st. It was in developer preview since 2021. We [blogged](https://pactflow.io/blog/extending-pact-with-plugins/) about the idea for the Pact Plugin Framework in 2021 and have been actively chipping award at the [roadmap](https://github.com/pactflow/roadmap/issues/33) item to enable developers everywhere to use contract testing where they previously couldn’t. +It went live on December 1st, after being in developer preview since 2021. The idea for the Pact Plugin Framework was [announced in a blog post](https://pactflow.io/blog/extending-pact-with-plugins/) in 2021, and the [roadmap item](https://github.com/pactflow/roadmap/issues/33) to enable developers everywhere to use contract testing where they previously couldn’t has since been delivered. Hats off to Ronald Holshausen who undertook the mammoth task of standing up the Framework. With this, development teams can now harness the power of contract testing where they previously couldn’t, applying it to unique and emerging use cases and technologies - no matter the scale or the language, transport, protocol or content type. diff --git a/website/docs/provider/best_practices.md b/website/docs/provider/best_practices.md new file mode 100644 index 000000000..0fa295181 --- /dev/null +++ b/website/docs/provider/best_practices.md @@ -0,0 +1,140 @@ +--- +title: Provider best practices +--- + +:::info The golden rule +When writing provider verification tests, your goal is to confirm that your provider correctly honours the contracts that real consumers have published. The tests should give you confidence that a deploy will not break a consumer — nothing more, nothing less. +::: + +Good provider verification is mostly about knowing what _not_ to do. It is easy to write provider tests that produce false positives, pass despite real integration bugs, or become brittle and expensive to maintain. This guide covers the principles and common pitfalls. + +## Verify against real consumers, not hand-crafted pacts + +Only verify pacts that have been generated by a real consumer's Pact tests. Pacts that are written by hand, or copy-pasted as examples, are not contracts — they are documentation. They will not be updated when the consumer changes, and verification against them gives you no useful guarantee. + +If you do not yet have any consumers running Pact, the right starting point is to get at least one consumer running consumer tests and publishing pacts to a broker before setting up provider verification. + +## Understand the scope of provider verification + +Provider verification is not functional testing. It does not check whether your business logic is correct, whether you handle all error conditions, or whether your service behaves correctly in edge cases. Those tests belong in the provider's own unit and integration test suite. + +What provider verification _does_ check: + +- The provider returns a response body that matches the structure the consumer expected. +- The provider honours the matching rules the consumer specified (type matching, regex, etc.). +- The provider returns the correct HTTP status code. +- The provider handles the request format the consumer sends. + +What it does _not_ check: + +- Business rules that are invisible from the outside. +- Error responses the consumer never described. +- Behaviour under load. + +If you find yourself trying to write provider states that test complex business logic, reconsider. That logic should be tested in the provider's own test suite. + +## Use the "pacts for verification" API + +Rather than manually specifying which pact URLs to verify, use the [Pacts for Verification API](/pact_broker/advanced_topics/provider_verification_results#pacts-for-verification) provided by the Pact Broker. This API returns exactly the set of pacts your provider should verify, based on consumer version selectors. + +At a minimum, verify the latest pact from each consumer's main branch: + +```js +consumerVersionSelectors: [ + { mainBranch: true } +] +``` + +Add selectors for deployed and released versions to catch regressions in production consumers: + +```js +consumerVersionSelectors: [ + { mainBranch: true }, + { deployedOrReleased: true } +] +``` + +See [Recommended configuration for verifying pacts](/provider/recommended_configuration) for full configuration examples. + +## Use the matching branch pattern for parallel development + +When a consumer team is developing a new feature on a branch, and their changes affect the contract, the provider needs to verify the new pact before the consumer merges. The `matchingBranch` selector makes this automatic: + +```js +consumerVersionSelectors: [ + { mainBranch: true }, + { deployedOrReleased: true }, + { matchingBranch: true } +] +``` + +When the provider is on a branch with the same name as the consumer's feature branch, `matchingBranch` pulls in the corresponding consumer pact. When branches don't match, the selector is simply ignored — no configuration changes needed. + +## Enable pending pacts on main; disable on feature branches + +[Pending pacts](/pact_broker/advanced_topics/pending_pacts) allow new consumer pacts — pacts that have never been successfully verified — to be included in the verification run without breaking the build. This prevents a consumer's in-progress work from blocking the provider's main pipeline. + +Enable pending pacts on your main branch only: + +```js +enablePending: true, +includeWipPactsSince: process.env.GIT_BRANCH === "main" ? "2020-01-01" : undefined +``` + +On a feature branch, you typically want a new consumer pact to fail the provider build (so you know when you have fulfilled it). Disabling pending pacts on feature branches preserves this signal. + +## Design provider states to be minimal and isolated + +Each provider state should create only the data required for its interactions. Provider states that share or accumulate state across interactions create brittle tests that fail for reasons unrelated to the contract. + +**Avoid:** +- Shared database fixtures that are created once and reused across all interactions. +- Provider states that depend on other provider states having already run. +- Provider states that perform real external API calls to set up data. + +**Prefer:** +- Provider states that create data in-process (directly to the database or via service layer) without going through the API under test. +- Provider states that clean up after themselves, or are wrapped in transactions that are rolled back. +- One provider state per named scenario, even if multiple interactions share similar data. + +See [Using provider states effectively](/provider/using_provider_states_effectively) for detailed examples of provider state design. + +## Publish verification results from CI only + +Verification results should only be published from your CI environment. Publishing locally risks polluting the broker with results that don't reflect the state of your deployable artefact. + +```js +publishVerificationResult: process.env.CI === "true" +``` + +Always publish the git commit SHA as the provider version, and include the branch name: + +```js +providerVersion: process.env.GIT_COMMIT, +providerVersionBranch: process.env.GIT_BRANCH +``` + +## Run verification in two scenarios + +There are two distinct triggers for provider verification: + +1. **The provider changes** — run a full verification against all current consumer pacts. This is your regression check. +2. **A pact changes** — a consumer has published a new version of a pact. Run verification for just that pact against all relevant provider versions. + +Configure webhooks in the Pact Broker so that a new pact publication automatically triggers a provider build. This closes the loop between consumer and provider development cycles. See [Recommended configuration](/provider/recommended_configuration#verification-triggered-by-a-contract-requiring-verification-published) for webhook setup. + +## Do not stub the provider's own dependencies too aggressively + +When running provider verification, the service under test should behave as close to production as possible. Stubbing downstream services is generally fine — you should not be testing their behaviour in your Pact tests. However, be careful about what you stub within the provider itself. + +Stubbing the data layer in a way that always returns a success, regardless of input, can cause false positives where verification passes even when the provider's business logic is broken. If you stub your data layer, make sure it returns data that accurately reflects what the provider state promises. + +## Understand what a failing verification means + +A failing provider verification means one of: + +1. The consumer has an expectation the provider does not fulfil — a real bug that would cause integration failure in production. +2. The consumer has changed their contract, and the provider needs to update to support the new requirement. +3. The provider has made a breaking change that would affect a consumer currently in production. + +Each of these is actionable and specific. Investigate the failure before deciding whether to fix the provider, update the consumer, or discuss the change with the consumer team. Do not mark failures as expected or suppress them — they exist to surface real compatibility problems. diff --git a/website/docs/provider/how_to_fix_failing_verification_tests.md b/website/docs/provider/how_to_fix_failing_verification_tests.md index e7bc4dda9..3d1173756 100644 --- a/website/docs/provider/how_to_fix_failing_verification_tests.md +++ b/website/docs/provider/how_to_fix_failing_verification_tests.md @@ -43,8 +43,8 @@ title: Help! My pact verification tests are failing. What do I do? ## For JVM -What to make a contribution? If you know how to do the above flow in Pact JVM, please click the `Edit this page` link at the bottom and submit a PR. You can find more info on contributing to the docs [here](https://docs.pact.io/contributing/docs). +Want to make a contribution? If you know how to do the above flow in Pact JVM, please click the `Edit this page` link at the bottom and submit a PR. You can find more info on contributing to the docs [here](https://docs.pact.io/contributing/docs). ## For Rust -What to make a contribution? If you know how to do the above flow in Pact Rust, please click the `Edit this page` link at the bottom and submit a PR. You can find more info on contributing to the docs [here](https://docs.pact.io/contributing/docs). +Want to make a contribution? If you know how to do the above flow in Pact Rust, please click the `Edit this page` link at the bottom and submit a PR. You can find more info on contributing to the docs [here](https://docs.pact.io/contributing/docs). diff --git a/website/docs/provider/recommended_configuration.md b/website/docs/provider/recommended_configuration.md index c18828d30..cca6330e6 100644 --- a/website/docs/provider/recommended_configuration.md +++ b/website/docs/provider/recommended_configuration.md @@ -215,7 +215,7 @@ When the pact URL is known, the `pactBrokerUrl`, `providerName`, `consumerVersio ### Examples -You can find samples in our [webhook template library](/pact_broker/webhooks/template_library) for common build systems and see below for how to configure your provider build to use the pact URL. +You can find samples in the [webhook template library](/pact_broker/webhooks/template_library) for common build systems and see below for how to configure your provider build to use the pact URL. ` containing the Pact-managed message. Convert the message to a byte array and pass it to your production deserialisation code: + +```java @Test @PactTestFor(pactMethod = "simpleJsonPact", providerType = ProviderType.ASYNCH) void simpleMessage(List messages) { @@ -69,14 +87,15 @@ private Deserializer getProductionKafkaDeserializer() { } ``` -Our test method gets a List of Messages passed in - these are Pact's representation of the message(s) that we defined in the previous step. Each message has space for a byte array and also metadata. We're not going to look much at metadata at this point. +:::tip +The `getProductionKafkaDeserializer` method should return your application's actual deserializer. In production, your `KafkaConsumer` is configured with a deserializer class name — reach into that configuration and instantiate it rather than constructing a new instance directly. The example above is a simplified stand-in. +::: -The `useProductionCodeToDeserializeKafkaBytesToDomain` method tries to indicate that we're meant to be using some real production code in our tests. After all, there would be little point if our tests just test themselves. With Kafka, however, this is quite tricky. We need a production object that will take our byte array and process it. With Kafka, we're not so close to the low level action - we typically use "out of the box" classes to do this work for us. A `KafkaConsumer` will typically return `ConsumerRecords` and you can't pass it a byte array! However, it would be configured with the appropriate Deserializers. For normal JSON, we'd configure it with a `KafkaJsonDeserializer`. Now, those take byte arrays as input. So, here I would suggest that you read your app's configuration, and work our what the class of the deserialiser is, instantiating an instance and returning it. A little tricky, but that's about as close to production code as we're going to get. The `useProductionCodeToDeserializeKafkaBytesToDomain` is a quicker way of doing it for the sake of this demo. +The `assertDoesNotThrow` assertion is intentional. Contract tests verify that the consumer can *handle* the message — the detailed behaviour under that message is covered by your functional tests. -Having turned our byte array into a custom domain object, what do we do? Well, what would your production application do with it? Call that code and make sure it can process it successfully. We don't need to do all the assertions that "service" tests might be interested in, we just need to be sure that our application understood the input. A simple `assertDoesNotThrow` is likely sufficient. +Running this test generates a pact file. The published pact looks like this: -Assuming you are using the Pact broker (well, why wouldn't you? Its awesome!), you might want to run and publish the consumer test. This is what it looks like on the Pact broker: -``` +```json { "consumer": { "name": "jsonKafkaConsumerApp" @@ -92,11 +111,7 @@ Assuming you are using the Pact broker (well, why wouldn't you? Its awesome!), y "body": { "$.name": { "combine": "AND", - "matchers": [ - { - "match": "type" - } - ] + "matchers": [{ "match": "type" }] } } }, @@ -106,12 +121,8 @@ Assuming you are using the Pact broker (well, why wouldn't you? Its awesome!), y } ], "metadata": { - "pact-jvm": { - "version": "4.3.4" - }, - "pactSpecification": { - "version": "3.0.0" - } + "pact-jvm": { "version": "4.3.4" }, + "pactSpecification": { "version": "3.0.0" } }, "provider": { "name": "jsonKafkaProviderApp" @@ -119,9 +130,13 @@ Assuming you are using the Pact broker (well, why wouldn't you? Its awesome!), y } ``` -## Kafka JSON Provider -Lets start off by getting our test class in place with the required annotations - its pretty self explanatory, but note the names of the consumer and provider must match what we put in the corresponding consumer test. We'll also add a `@BeforeEach` which tells Pact we're running Message related tests (rather than HTTP), and implement the `@TestTemplate` method. Finally, we have a couple of constants that we'll need in the next step: -``` +## Write the provider test + +### 1. Set up the test class + +Configure the provider and consumer names to match the consumer test. Set the test target to `MessageTestTarget`: + +```java @Provider("jsonKafkaProviderApp") @Consumer("jsonKafkaConsumerApp") @PactBroker(url = "http://localhost:9292") @@ -142,8 +157,12 @@ class JsonKafkaProviderTest { } } ``` -All that is left, is to write the test: -``` + +### 2. Implement the message producer method + +Annotate a method with `@PactVerifyProvider` matching the description in the consumer pact. The method must produce a `MessageAndMetadata` containing a serialised byte array of the message: + +```java @PactVerifyProvider("A simple message") MessageAndMetadata verifySimpleMessageEvent() { Map metadata = Map.of( @@ -156,8 +175,7 @@ MessageAndMetadata verifySimpleMessageEvent() { } private KafkaJsonSerializer createProductionKafkaSerializer() { - Map config = Map.of( - ); + Map config = Map.of(); KafkaJsonSerializer jsonSerializer = new KafkaJsonSerializer<>(); jsonSerializer.configure(config, false); return jsonSerializer; @@ -167,27 +185,36 @@ private MessageAndMetadata createPactRepresentationFor(Map metad return new MessageAndMetadata(bytes, metadata); } ``` -In a similar fashion to the consumer test, we will need to get hold of a Kafka Serializer, which we should obtain through looking up the configuration of our production code (and again, in this demo, we'll just create an instance for use - make sure you do a better job!). -The `KafkaJsonSerializer` will take our domain object and turn it into a byte array. we pass this back to Pact which will compare it to what is held in the Pact broker. If its a match, we're green... +Pact serialises your domain object to a byte array, then compares it against the consumer's expectation in the broker. If it matches, verification passes. + +--- # Schema Registry Compliant JSON -The code examples here use Java, and are based on writing JSON messages to Kafka. If you are using AVRO or Protobuf, then the Kafka Schema-Registry examples may not yet be implemented (the PR for the JSON implementation can be found [here](https://github.com/pact-foundation/pact-jvm/pull/1503)). -## Magic Bytes -When a schema registry compliant message is written to a Kafka topic, the message (whether it is JSON, AVRO or Protobuf) is preceded by [5 "magic" bytes](https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#wire-format). If you were to use a compliant producer with a non-compliant consumer, things wouldn't work. We want our contract test to cover us here! Let's take a look at how this can be achieved (reminder - at the time of writing, this only works for JSON). +Use this approach when your Kafka topic uses the Confluent Schema Registry wire format. Schema Registry compliant messages are preceded by [5 "magic" bytes](https://docs.confluent.io/platform/current/schema-registry/serdes-develop/index.html#wire-format) that identify the schema version. Your consumer code must be able to process these bytes correctly. -## Schema Registry JSON Consumer -We'll start with the bones of a class and we will use Junit5 annotations: -``` +:::note +At the time of writing, this approach is implemented for JSON only. The PR for the JSON implementation is [here](https://github.com/pact-foundation/pact-jvm/pull/1503). AVRO and Protobuf schema registry support may not yet be available. +::: + +## Write the consumer test + +### 1. Set up the test class + +```java @ExtendWith(PactConsumerTestExt.class) @PactTestFor(providerName = "jsonSchemaKafkaProviderApp", providerType = ProviderType.ASYNCH, pactVersion = PactSpecVersion.V3) class JsonSchemaKafkaConsumerTest { ... } ``` -Next we define what we *expect* the contract to look like: -``` + +### 2. Define the expected message contract + +The only difference from the standard JSON case is the `contentType` metadata. Setting it to `application/vnd.schemaregistry.v1+json` tells Pact to prepend the 5 magic bytes to the message data: + +```java @Pact(consumer = "jsonSchemaKafkaConsumerApp") MessagePact schemaJsonPact(MessagePactBuilder builder) { PactDslJsonBody body = new PactDslJsonBody(); @@ -199,8 +226,10 @@ MessagePact schemaJsonPact(MessagePactBuilder builder) { .toPact(); } ``` -And there was our change: `.withMetadata(Map.of("contentType", "application/vnd.schemaregistry.v1+json"))` The metadata has been changed to match that of a schema registry compliant JSON message. This is significant - Pact will now be aware of the need for those 5 magic bytes mentioned earlier. We can now write our test. First, lets add a few constants and a field: -``` + +### 3. Add the required constants and schema registry client + +```java private static final String TOPIC_NAME = "myKafkaTopic"; private static final boolean FAIL_UNKNOWN_PROPERTIES = true; private static final boolean USE_ONEOF_FOR_NULLABLES = true; @@ -209,8 +238,12 @@ private static final String IGNORED_TOPIC = ""; private MockSchemaRegistryClient schemaRegistryClient = new MockSchemaRegistryClient(); ``` -And then the test itself: -``` + +### 4. Write the test method + +The test method is identical in structure to the standard JSON case. Because the `contentType` metadata signals the magic bytes, Pact will include them in the byte array passed to your test. Your deserialiser must handle them: + +```java @Test @PactTestFor(pactMethod = "schemaJsonPact", providerType = ProviderType.ASYNCH) void jsonSchemaMessage(List messages) { @@ -220,23 +253,12 @@ void jsonSchemaMessage(List messages) { }); } ``` -Now, as we have set the contentType to application/vnd.schemaregistry.v1+json then Pact will augment each of the Message data structures passed in to the method with 5 magic bytes. The values of the bytes themselves are not important (they would be used to communicate the schema id to the registry were we running kafka), but it is now important for your consumer test to be able to process the messages correctly - they will need to effectively ignore those bytes before the JSON is parsed. If it doesn't, then the JSON will be invalid and the test will fail - that's what we're after! -We've referenced some private methods in there, so lets look at the implementations for those: -``` -private byte[] convertToKafkaBytes(Message message) { - return message.contentsAsBytes(); -} +### 5. Implement the deserialiser helper methods -private void expectApplicationToConsumeKafkaBytesSuccessfully(byte[] kafkaBytes) { - ConsumerDomainRecord consumerDomainRecord = useProductionCodeToDeserializeKafkaBytesToDomain(kafkaBytes); - ProductionCode productionCode = new ProductionCode(); - productionCode.handle(consumerDomainRecord); -} -``` -A Kafka consumer application will deserialize messages from a topic into a custom data structure, which the application will then process. That's what we see here - the deserialization happening (more of that in a moment), followed by a call to the application (ProductionCode.handle()) to process the message. Our assertion checks that no Exceptions are thrown - in other words, the application can successfully consume the message (we don't care what it does - that's the job of the functional tests!). Also note that the byte[] being passed in will contain those 5 magic bytes at the start, preceding the actual JSON. -Moving on: -``` +Use a `MockSchemaRegistryClient` to avoid communicating with a real Schema Registry during tests. Configure the URL as `mock://anything`: + +```java private ConsumerDomainRecord useProductionCodeToDeserializeKafkaBytesToDomain(byte[] kafkaBytes) { KafkaJsonSchemaDeserializer deserializer = getProductionKafkaDeserializer(); return deserializer.deserialize(IGNORED_TOPIC, kafkaBytes); @@ -252,13 +274,7 @@ private KafkaJsonSchemaDeserializer getProductionKafkaDese tryRegisterSchemaForValue(); return deserializer; } -``` -Those last two methods are worthy of note. The getProductionKafkaDeserializer method is intended to reach into your production code (otherwise we're just testing the test code, and not a lot of point in that!) and get hold of the deserializer used. Most Kafka consumer applications would use out-of-the-box deserializers, which are simply a configuration parameter for the KafkaConsumer. You may want to reach into the config and pull out the class name defined for the Deserializer, and then instantiate it perhaps using reflection - I'm going to leave that as an exercise for you as I don't want to play with your production code here! The point is that the implementation given here needs to be replaced with some production code for your test to have much meaning. - -Also keep an eye out for the config for the URL to the Schema Registry. As this won't be running for our tests, we configure the value of the URL with "mock://" - this tells the deserializer not to try and communicate to the registry. -Moving on, let's complete the last few private methods: -``` private void tryRegisterSchemaForValue() { try { registerSchemaForValue(); @@ -282,57 +298,37 @@ private JsonSchema getConsumerDomainRecordSchema() throws IOException { ); } ``` -We're using a MockSchemaRegistryClient which is provided out-of-the-box. Given that we're not actually running Kafka, we're also not running the Schema Registry, so we don't want anything to try and communicate with it, and thats what this class does for us. We need to register the subject (usually the Kafka topic + either "-key" or "-value") along with the JsonSchema itself into the MockSchemaRegistryClient. -## The Pact -As ever, running the consumer test will generate the Pact file (JSON), and push it out to the Pact Broker (well, you are using one, right?!) if running in CI, or publishPacts flag is set: -``` +Running this test generates a pact file. Note the `contentType` in the metadata: + +```json { - "consumer": { - "name": "jsonSchemaKafkaConsumerApp" - }, + "consumer": { "name": "jsonSchemaKafkaConsumerApp" }, "messages": [ { - "_id": "b99581bf53e0c0cdca0479b93aa6539b142394f6", - "contents": { - "name": "almost-anything2" - }, + "contents": { "name": "almost-anything2" }, "description": "A json schema message", "matchingRules": { "body": { - "$.name": { - "combine": "AND", - "matchers": [ - { - "match": "type" - } - ] - } + "$.name": { "combine": "AND", "matchers": [{ "match": "type" }] } } }, - "metaData": { - "contentType": "application/vnd.schemaregistry.v1+json" - } + "metaData": { "contentType": "application/vnd.schemaregistry.v1+json" } } ], "metadata": { - "pact-jvm": { - "version": "4.3.4" - }, - "pactSpecification": { - "version": "3.0.0" - } + "pact-jvm": { "version": "4.3.4" }, + "pactSpecification": { "version": "3.0.0" } }, - "provider": { - "name": "jsonSchemaKafkaProviderApp" - } + "provider": { "name": "jsonSchemaKafkaProviderApp" } } ``` -Hopefully, that's pretty much what you expected to see. Again, note the contentType shows up. -## Schema Registry JSON Provider -Lets take a look at the other side now - the provider test. We'll get our JUnit5 test class into being, along with a handful of constants: -``` +## Write the provider test + +### 1. Set up the test class + +```java @Provider("jsonSchemaKafkaProviderApp") @Consumer("jsonSchemaKafkaConsumerApp") @PactBroker(url = "http://localhost:9292") @@ -343,7 +339,7 @@ class JsonSchemaKafkaProviderTest { private static final String DO_NOT_USE_SCHEMA_REGISTRY = "mock://anything"; private static final boolean IS_KEY = false; -@BeforeEach + @BeforeEach void before(PactVerificationContext context) { context.setTarget(new MessageTestTarget()); } @@ -355,8 +351,12 @@ class JsonSchemaKafkaProviderTest { } } ``` -Lets add the test: -``` + +### 2. Implement the message producer method + +Use a `KafkaJsonSchemaSerializer` backed by a `MockSchemaRegistryClient`. Because the `contentType` is set to the schema registry type, Pact will skip the first 5 bytes when comparing the JSON output: + +```java @PactVerifyProvider("A json schema message") MessageAndMetadata verifyJsonSchemaMessageEvent() { Map metadata = Map.of( @@ -383,4 +383,3 @@ private MessageAndMetadata createPactRepresentationFor(Map metad return new MessageAndMetadata(bytes, metadata); } ``` -We get hold of a serializer from the production code (again you probably want to get hold of the kafka config and instantiate the serializer used via reflection), and give it our domain object to turn into a byte[]. We wrap that in Pact's MessageAndMetadata type and that's what our test returns. Pact will check that it matches what the broker holds, and as we're using the SCHEMA_REGISTRY_JSON_CONTENT_TYPE, it will start parsing the JSON from the 6th byte - missing off those magic bytes. diff --git a/website/docs/recipes/munit.md b/website/docs/recipes/munit.md index 9dde25a45..5025a1459 100644 --- a/website/docs/recipes/munit.md +++ b/website/docs/recipes/munit.md @@ -8,21 +8,21 @@ description: How to use Pact for testing MuleSoft (Mule 4) APIs with MUnit As of Mule 4, it is no longer possible to extend the MUnit runner through Java code. Previously, the solution of Pact testing with MuleSoft was to extend “FunctionalMunitSuite” in your test class. An example can be found here: https://docs.mulesoft.com/munit/1.3/munit-tests-with-Java#creating-your-suite-class -However, MuleSoft now wants alignment with the Mule language to be able to provide a better service. Which leaves the outstanding problem; How can we use Pact to test Mule 4 applications? +However, MuleSoft now wants alignment with the Mule language to be able to provide a better service. This leaves the outstanding problem: how can Pact be used to test Mule 4 applications? -In a nutshell, the solution is to have MUnit be the test runner and execute the standalone Pact-JVM server by calling Java code within the MUnit console. This will allow for both the benefit of MUnit tests along with pact files being created for contract testing. As a result, we will be able to leverage the MUnit unit test coverage while also generating our pact files and publishing them to a Pact Broker or sharing them into your own file storage system. +In a nutshell, the solution is to have MUnit be the test runner and execute the standalone Pact-JVM server by calling Java code within the MUnit console. This provides the benefit of MUnit tests along with pact files being created for contract testing — leveraging MUnit unit test coverage while also generating pact files and publishing them to a Pact Broker or sharing them to your own file storage system. ## Consumer -### So how do we do it? +### How it works -First, we need to understand the MUnit test layout. It has a very similar test structure to other unit testing tools, such as Jest, jUnit, nUnit, etc., with the usual before suite, before test, after test, and after suite. The test logic is formed into the test blocks where “flows” are executed in the test scripts. +The first step is to understand the MUnit test layout. It has a very similar test structure to other unit testing tools, such as Jest, jUnit, nUnit, etc., with the usual before suite, before test, after test, and after suite. The test logic is formed into the test blocks where “flows” are executed in the test scripts. An empty MUnit test would look like the following: ![Anypoint Studio MUnit Test Layout](../../static/img/docs/recipes/munit/anypoint-test-struct.png) -In the test block, we can see it is divided into three sections; execution, behavior, and validation. The assertions for tests come from MUnit tools providing a low code UI block that we configure for each type of unit test assertion we wish to perform on the flow. +The test block is divided into three sections: execution, behavior, and validation. Assertions come from MUnit tools providing a low-code UI block configured for each type of unit test assertion on the flow. Looking through the Mule Palette, in addition to these low code blocks there are also Java blocks for writing Java classes with the ability to call functions. @@ -35,11 +35,11 @@ The functionality of these blocks is as follows: ![Mule Java Palette](../../static/img/docs/recipes/munit/mule-palette-java.png) -For this recipe, we will be utilizing all but the ‘Validate type’ Java block, but you are welcome to use it in your own solution. +This recipe uses all but the ‘Validate type’ Java block — you are welcome to include it in your own solution. ### Starting the Pact JVM from Java -To use the standalone JVM of Pact we will need to create a static class called PactService. This will handle the command to run and close the Pact Standalone Server. The class will have the following functions: +To use the standalone JVM of Pact, create a static class called `PactService`. This handles the commands to run and close the Pact Standalone Server. The class needs the following functions: - `startPactService()`: will start the service through the CLI command - `stopPactService()`: will stop the Java process @@ -66,13 +66,13 @@ public static void stopPactService() throws ClientProtocolException, IOException } ``` -By setting the service to a variable, we will be able to then stop the running process later with the `.destroy()` command. +Setting the service to a variable allows you to stop the running process later with the `.destroy()` command. More information on the Pact JVM can be found here: https://docs.pact.io/implementation_guides/jvm/pact-jvm-server #### From Anypoint Studio -From Anypoint Studio, we will use the "Invoke Static" Java block to call the `startPactService()` method within the PactService class. +From Anypoint Studio, use the "Invoke Static" Java block to call the `startPactService()` method within the PactService class. This will start running the Pact standalone server as a separate Java process on the machine. @@ -80,7 +80,7 @@ This will start running the Pact standalone server as a separate Java process on ### Creating a Mock Service -Now that we have the service running, all we need to do is make some API calls to localhost with the port. We can use the Java "Invoke static" mule widget for this as we only need to make API calls to set up the Mock API Provider. +With the service running, make API calls to localhost on the configured port. Use the Java "Invoke static" Mule widget for this, as only API calls to localhost are needed to set up the Mock API Provider. Here is an example of using a CloseableHttpClient but you can use any preferred method for making the API calls from Java. @@ -103,7 +103,7 @@ The variable pactServicePort is the port that the Pact service is running on. Th This would be very similar to the example logic provided here: https://docs.pact.io/implementation_guides/jvm/consumer -When setting up the `pactBody` variable, I found the need to use the "New" and "Invoke" Mule Java blocks in order to avoid errors. However, there might be other solutions where static methods can be used like the rest of the solution. +When setting up the `pactBody` variable, use the "New" and "Invoke" Mule Java blocks to avoid errors. Other solutions using static methods may also work. #### From Anypoint Studio diff --git a/website/docs/tutorials.md b/website/docs/tutorials.md new file mode 100644 index 000000000..94399e595 --- /dev/null +++ b/website/docs/tutorials.md @@ -0,0 +1,68 @@ +--- +id: tutorials +title: Tutorials +description: Hands-on tutorials to learn Pact by doing — from your first contract test to CI/CD integration. +--- + +These tutorials take you from zero to working Pact tests through step-by-step, hands-on exercises. Each tutorial is designed so you can follow along in your own environment. + +## Where to start + +If this is your first time with Pact, begin with the **5-minute quick start** to see a consumer and provider test end-to-end. Then continue with the **Introduction to Pact** workshop for a thorough, multi-step treatment of the core workflow. + +## Available tutorials + +### 5-minute quick start + +The fastest path to your first running Pact test. Covers consumer testing, provider verification, and sharing pacts via a broker — all in JavaScript. + +[Start the 5-minute quick start →](/5-minute-getting-started-guide) + +--- + +### Introduction to Pact (13 steps) + +A comprehensive, language-agnostic introduction to consumer-driven contract testing with Pact. Covers the full workflow from writing consumer tests through to CI/CD integration. + +Topics covered: +- Writing consumer tests +- Provider verification +- Publishing pacts to a broker +- Using `can-i-deploy` +- Setting up CI/CD pipelines + +[Start Introduction to Pact →](/university/introduction/00_1_Intro) + +--- + +### Introduction to Message Pact — Async (4 steps) + +Learn how to use Pact for async messaging systems such as Kafka, SNS, and SQS. Covers consumer and provider (producer) testing for message-based integrations. + +[Start async messaging tutorial →](/university/message-pact-async/00_1_Intro) + +--- + +### Create a Pact Plugin (7 steps) + +A hands-on guide to building your own Pact plugin to support custom protocols and content types not natively supported by Pact. + +[Start the plugin workshop →](/plugins/workshops/create-a-plugin/intro) + +--- + +### External workshops + +Community-maintained workshops covering Pact in specific languages and frameworks. + +[Browse workshops →](/implementation_guides/workshops) + +--- + +## After the tutorials + +Once you've completed a tutorial: + +- **Best practices**: See [How-to Guides](/consumer) for consumer and provider best practices, recipes for specific scenarios (Kafka, GraphQL, AWS), and the [CI/CD setup guide](/pact_nirvana). +- **Deeper understanding**: Visit [Concepts](/concepts) to understand the theory behind what you practiced. +- **Language-specific docs**: See [Pact Docs](/implementation_guides/overview) for documentation on your specific language client. diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js index 50688349c..e04545798 100644 --- a/website/docusaurus.config.js +++ b/website/docusaurus.config.js @@ -104,15 +104,16 @@ const siteConfig = { src: 'img/favicon.ico' }, items: [ - { to: '/', label: 'Getting Started', position: 'left', activeBaseRegex: '^/$|getting_started|faq|5-minute' }, - { to: 'consumer', label: 'Guides', position: 'left', activeBaseRegex: '^/(consumer|provider|pact_nirvana|recipes)' }, + { to: '/', label: 'Getting Started', position: 'left', activeBaseRegex: '^/$|^/getting_started/(how_pact_works|what_is_pact_good_for|comparisons)|^/faq' }, + { to: 'concepts', label: 'Concepts', position: 'left', activeBaseRegex: '^/concepts|^/getting_started/(conceptual_overview|terminology|testing-scope|matching|verifying_pacts|provider_states|sharing_pacts|specification|further_reading)' }, + { to: 'tutorials', label: 'Tutorials', position: 'left', activeBaseRegex: '^/(tutorials|university|5-minute)' }, + { to: 'consumer', label: 'How-to Guides', position: 'left', activeBaseRegex: '^/(consumer|provider|pact_nirvana|recipes)' }, { to: 'implementation_guides/overview', label: 'Pact Docs', position: 'left', activeBaseRegex: '^/(implementation_guides|roadmap|wrapper_implementations|telemetry)/' }, { to: 'pact_broker', label: 'Pact Broker Docs', position: 'left' }, { to: 'history', label: 'Resources', position: 'left' }, { to: 'contributing', label: 'Contributing', position: 'left', activeBaseRegex: '^/contributing|diagrams/ecosystem' }, { to: 'blog', label: 'Blog', position: 'left' }, - { to: 'help', label: 'Help', position: 'left' }, - { to: 'university', label: 'University', position: 'left' } + { to: 'help', label: 'Help', position: 'left' } ] }, @@ -132,7 +133,15 @@ const siteConfig = { to: '/' }, { - label: 'Guides', + label: 'Concepts', + to: 'concepts' + }, + { + label: 'Tutorials', + to: 'tutorials' + }, + { + label: 'How-to Guides', to: 'consumer' }, { diff --git a/website/sidebars.json b/website/sidebars.json index 1e7db161a..b9623f408 100644 --- a/website/sidebars.json +++ b/website/sidebars.json @@ -190,6 +190,7 @@ }, { "Pact Plugins": [ + "plugins/concepts", "plugins/quick_start", "implementation_guides/pact_plugins/readme", { @@ -304,8 +305,7 @@ "label": "Getting started", "items": [ "getting_started", - "getting_started/how_pact_works", - "5-minute-getting-started-guide" + "getting_started/how_pact_works" ] }, { @@ -315,8 +315,15 @@ "faq/convinceme" ] }, + "faq" + ], + "concepts": [ + "concepts", { - "Pact concepts": [ + "type": "category", + "label": "Pact Concepts", + "items": [ + "getting_started/why-pact", "getting_started/conceptual_overview", "getting_started/terminology", "getting_started/testing-scope", @@ -330,53 +337,76 @@ }, "getting_started/verifying_pacts", "getting_started/provider_states", - "getting_started/sharing_pacts", + "getting_started/sharing_pacts" + ] + }, + { + "type": "category", + "label": "Reference", + "items": [ "getting_started/specification", - "getting_started/further_reading" + "getting_started/pact-file-format" + ] + } + ], + "guides": [ + { + "type": "category", + "label": "Consumer", + "items": [ + "consumer", + "consumer/best_practices", + "consumer/contract_tests_not_functional_tests", + "consumer/how_to_test_async_messages", + "consumer/using_pact_to_support_ui_testing", + "consumer/recommended_configuration" ] }, - "implementation_guides/workshops", - "faq" + { + "type": "category", + "label": "Provider", + "items": [ + "provider", + "provider/best_practices", + "provider/using_provider_states_effectively", + "provider/handling_auth", + "provider/recommended_configuration", + "provider/how_to_fix_failing_verification_tests" + ] + }, + { + "type": "category", + "label": "CI/CD Setup Guide", + "items": [ + "pact_nirvana", + "pact_nirvana/step_1", + "pact_nirvana/step_2", + "pact_nirvana/step_3", + "pact_nirvana/step_4", + "pact_nirvana/step_5", + "pact_nirvana/step_6", + "pact_nirvana/step_7", + "pact_nirvana/notes_1" + ] + }, + { + "type": "category", + "label": "Recipes", + "items": [ + "recipes", + "recipes/optional", + "recipes/kafka", + "recipes/graphql", + "recipes/apigateway", + "recipes/awssignedrequests", + "recipes/lambdaasync", + "recipes/lambdahttp", + "recipes/cypress", + "recipes/munit", + "implementation_guides/feature_toggles" + ] + } ], - "guides": { - "Consumer": [ - "consumer", - "consumer/contract_tests_not_functional_tests", - "consumer/using_pact_to_support_ui_testing", - "consumer/recommended_configuration" - ], - "Provider": [ - "provider", - "provider/using_provider_states_effectively", - "provider/handling_auth", - "provider/recommended_configuration", - "provider/how_to_fix_failing_verification_tests" - ], - "CI/CD Setup Guide": [ - "pact_nirvana", - "pact_nirvana/step_1", - "pact_nirvana/step_2", - "pact_nirvana/step_3", - "pact_nirvana/step_4", - "pact_nirvana/step_5", - "pact_nirvana/step_6", - "pact_nirvana/step_7", - "pact_nirvana/notes_1" - ], - "Recipes": [ - "recipes", - "recipes/optional", - "recipes/kafka", - "recipes/graphql", - "recipes/apigateway", - "recipes/awssignedrequests", - "recipes/lambdaasync", - "recipes/lambdahttp", - "recipes/cypress", - "recipes/munit", - "implementation_guides/feature_toggles" - ] - }, "pact_broker": { "Pact Broker": [ "pact_broker", @@ -466,6 +496,7 @@ }, "resources": [ "history", + "getting_started/further_reading", "blogs_videos_and_articles", "community_repos", "books", @@ -508,12 +539,15 @@ "contributing/code-of-conduct", "diagrams/ecosystem" ], - "university": [ + "tutorials": [ { "type": "category", - "label": "Pact University", + "label": "Tutorials", "items": [ + "tutorials", "university", + "5-minute-getting-started-guide", + "implementation_guides/workshops", { "type": "category", "label": "Introduction to Pact", diff --git a/website/yarn.lock b/website/yarn.lock index e298b5e3f..17dff7de0 100644 --- a/website/yarn.lock +++ b/website/yarn.lock @@ -144,7 +144,7 @@ resolved "https://registry.npmjs.org/@babel/compat-data/-/compat-data-7.28.5.tgz" integrity sha512-6uFXyCayocRbqhZOB+6XcuZbkMNimwfVGFji8CTZnCzOHVGvDqzvitu1re2AU5LROliz7eQPhB8CpAMvnx9EjA== -"@babel/core@^7.0.0", "@babel/core@^7.0.0-0", "@babel/core@^7.12.0", "@babel/core@^7.13.0", "@babel/core@^7.18.6", "@babel/core@^7.19.6", "@babel/core@^7.4.0-0": +"@babel/core@^7.0.0", "@babel/core@^7.0.0-0", "@babel/core@^7.11.6", "@babel/core@^7.12.0", "@babel/core@^7.13.0", "@babel/core@^7.18.6", "@babel/core@^7.19.6", "@babel/core@^7.4.0-0": version "7.28.5" resolved "https://registry.npmjs.org/@babel/core/-/core-7.28.5.tgz" integrity sha512-e7jT4DxYvIDLk1ZHmU/m/mB19rex9sv0c2ftBtjSBv+kVM/902eh0fINUzD7UwLLNR+jU585GxUJ8/EBfAM5fw== @@ -165,7 +165,7 @@ json5 "^2.2.3" semver "^6.3.1" -"@babel/core@^7.11.6", "@babel/core@7.12.9": +"@babel/core@7.12.9": version "7.12.9" resolved "https://registry.npmjs.org/@babel/core/-/core-7.12.9.tgz" integrity sha512-gTXYh3M5wb7FRXQy+FErKFAv90BnlOuNn1QkCK2lREoPAjrQCO49+HVSrFoe5uakFAF5eenS75KbO2vQiLrTMQ== @@ -259,7 +259,11 @@ semver "^6.1.2" "@babel/helper-environment-visitor@^7.18.9": - version "7.18.9" + version "7.24.7" + resolved "https://registry.npmjs.org/@babel/helper-environment-visitor/-/helper-environment-visitor-7.24.7.tgz" + integrity sha512-DoiN84+4Gnd0ncbBOM9AZENV4a5ZiL39HYMyZJGZ/AZEykHYdJw0wW3kdcsh9/Kn+BRXHLkkklZ51ecPKmI1CQ== + dependencies: + "@babel/types" "^7.24.7" "@babel/helper-explode-assignable-expression@^7.18.6": version "7.18.6" @@ -269,10 +273,12 @@ "@babel/types" "^7.18.6" "@babel/helper-function-name@^7.18.9", "@babel/helper-function-name@^7.19.0", "@babel/helper-function-name@^7.21.0": - version "7.21.0" + version "7.24.7" + resolved "https://registry.npmjs.org/@babel/helper-function-name/-/helper-function-name-7.24.7.tgz" + integrity sha512-FyoJTsj/PEUWu1/TYRiXTIHc8lbw+TDYkZuoE43opPS5TrI7MyONBE1oNvfguEXAD9yhQRrVBnXdXzSLQl9XnA== dependencies: - "@babel/template" "^7.20.7" - "@babel/types" "^7.21.0" + "@babel/template" "^7.24.7" + "@babel/types" "^7.24.7" "@babel/helper-globals@^7.28.0": version "7.28.0" @@ -280,9 +286,11 @@ integrity sha512-+W6cISkXFa1jXsDEdYA8HeevQT/FULhxzR99pxphltZcVaugps53THCeiWA8SguxxpSp3gKPiuYfSWopkLQ4hw== "@babel/helper-hoist-variables@^7.18.6": - version "7.18.6" + version "7.24.7" + resolved "https://registry.npmjs.org/@babel/helper-hoist-variables/-/helper-hoist-variables-7.24.7.tgz" + integrity sha512-MJJwhkoGy5c4ehfoRyrJ/owKeMl19U54h27YYftT0o2teQ3FJ3nQUf/I3LlJsX4l3qlw7WRXUmiyajvHXoTubQ== dependencies: - "@babel/types" "^7.18.6" + "@babel/types" "^7.24.7" "@babel/helper-member-expression-to-functions@^7.20.7", "@babel/helper-member-expression-to-functions@^7.21.0": version "7.21.0" @@ -362,9 +370,11 @@ "@babel/types" "^7.20.0" "@babel/helper-split-export-declaration@^7.18.6": - version "7.18.6" + version "7.24.7" + resolved "https://registry.npmjs.org/@babel/helper-split-export-declaration/-/helper-split-export-declaration-7.24.7.tgz" + integrity sha512-oy5V7pD+UvfkEATUKvIjvIAH/xCzfsFVw7ygW2SI6NClZzquT+mwdTfgfdbUiceh6iQO0CHtCPsyze/MZ2YbAA== dependencies: - "@babel/types" "^7.18.6" + "@babel/types" "^7.24.7" "@babel/helper-string-parser@^7.27.1": version "7.27.1" @@ -609,14 +619,7 @@ dependencies: "@babel/helper-plugin-utils" "^7.8.0" -"@babel/plugin-syntax-jsx@^7.18.6": - version "7.21.4" - resolved "https://registry.npmjs.org/@babel/plugin-syntax-jsx/-/plugin-syntax-jsx-7.21.4.tgz" - integrity sha512-5hewiLct5OKyh6PLKEYaFclcqtIgCb6bmELouxjF6up5q3Sov7rOayW4RwhbaBL0dit8rA80GNfY+UuDp2mBbQ== - dependencies: - "@babel/helper-plugin-utils" "^7.20.2" - -"@babel/plugin-syntax-jsx@^7.21.4": +"@babel/plugin-syntax-jsx@^7.18.6", "@babel/plugin-syntax-jsx@^7.21.4": version "7.21.4" resolved "https://registry.npmjs.org/@babel/plugin-syntax-jsx/-/plugin-syntax-jsx-7.21.4.tgz" integrity sha512-5hewiLct5OKyh6PLKEYaFclcqtIgCb6bmELouxjF6up5q3Sov7rOayW4RwhbaBL0dit8rA80GNfY+UuDp2mBbQ== @@ -1141,7 +1144,7 @@ dependencies: regenerator-runtime "^0.13.11" -"@babel/template@^7.12.7", "@babel/template@^7.18.10", "@babel/template@^7.20.7", "@babel/template@^7.27.2": +"@babel/template@^7.12.7", "@babel/template@^7.18.10", "@babel/template@^7.20.7", "@babel/template@^7.24.7", "@babel/template@^7.27.2": version "7.27.2" resolved "https://registry.npmjs.org/@babel/template/-/template-7.27.2.tgz" integrity sha512-LPDZ85aEJyYSd18/DkjNh4/y1ntkE5KwUHWTiqgRxruuZL2F1yuHligVHLvcHY2vMHXttKFpJn6LwfI7cw7ODw== @@ -1163,7 +1166,7 @@ "@babel/types" "^7.28.5" debug "^4.3.1" -"@babel/types@^7.12.7", "@babel/types@^7.18.6", "@babel/types@^7.18.9", "@babel/types@^7.20.0", "@babel/types@^7.20.2", "@babel/types@^7.20.5", "@babel/types@^7.20.7", "@babel/types@^7.21.0", "@babel/types@^7.21.4", "@babel/types@^7.27.1", "@babel/types@^7.28.4", "@babel/types@^7.28.5", "@babel/types@^7.4.4": +"@babel/types@^7.12.7", "@babel/types@^7.18.6", "@babel/types@^7.18.9", "@babel/types@^7.20.0", "@babel/types@^7.20.2", "@babel/types@^7.20.5", "@babel/types@^7.20.7", "@babel/types@^7.21.0", "@babel/types@^7.21.4", "@babel/types@^7.24.7", "@babel/types@^7.27.1", "@babel/types@^7.28.4", "@babel/types@^7.28.5", "@babel/types@^7.4.4": version "7.28.5" resolved "https://registry.npmjs.org/@babel/types/-/types-7.28.5.tgz" integrity sha512-qQ5m48eI/MFLQ5PxQj4PFaprjyCTLI37ElWMmNs0K8Lk3dVeOdNpB3ks8jc7yM5CDmVC73eMVk/trk3fgmrUpA== @@ -2000,10 +2003,7 @@ "@types/estree" "*" "@types/json-schema" "*" -"@types/estree@*": - version "0.0.51" - -"@types/estree@^1.0.0": +"@types/estree@*", "@types/estree@^1.0.0": version "1.0.1" resolved "https://registry.npmjs.org/@types/estree/-/estree-1.0.1.tgz" integrity sha512-LG4opVs2ANWZ1TJoKc937iMmNstM/d0ae1vNbnBvBhqCSezgVUOzcLCqbI5elV8Vy6WKwKjaqR+zO9VKirBBCA== @@ -2365,10 +2365,16 @@ acorn-import-assertions@^1.9.0: integrity sha512-cmMwop9x+8KFhxvKrKfPYmN6/pKTYYHBqLa0DfvVZcKMJWNyWLnaqND7dx/qn66R7ewM1UX5XMaDVP5wlVTaVA== acorn-walk@^8.0.0, acorn-walk@^8.1.1: - version "8.2.0" + version "8.3.5" + resolved "https://registry.npmjs.org/acorn-walk/-/acorn-walk-8.3.5.tgz" + integrity sha512-HEHNfbars9v4pgpW6SO1KSPkfoS0xVOM/9UzkJltjlsHZmJasxg8aXkuZa7SMf8vKGIBhpUsPluQSqhJFCqebw== + dependencies: + acorn "^8.11.0" -acorn@^8, acorn@^8.0.4, acorn@^8.4.1, acorn@^8.5.0, acorn@^8.7.1: - version "8.8.2" +acorn@^8, acorn@^8.0.4, acorn@^8.11.0, acorn@^8.4.1, acorn@^8.5.0, acorn@^8.7.1: + version "8.16.0" + resolved "https://registry.npmjs.org/acorn/-/acorn-8.16.0.tgz" + integrity sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw== address@^1.0.1, address@^1.1.2: version "1.2.2" @@ -2412,7 +2418,17 @@ ajv@^6.12.2, ajv@^6.12.4, ajv@^6.12.5, ajv@^6.9.1: json-schema-traverse "^0.4.1" uri-js "^4.2.2" -ajv@^8.0.0, ajv@^8.8.0, ajv@^8.8.2: +ajv@^8.0.0: + version "8.12.0" + resolved "https://registry.npmjs.org/ajv/-/ajv-8.12.0.tgz" + integrity sha512-sRu1kpcO9yLtYxBKvqfTeh9KzZEwO3STyX1HT+4CaDzC6HpTGYhIhPIzj9XuKU7KYDwnaeh5hcOwjy1QuJzBPA== + dependencies: + fast-deep-equal "^3.1.1" + json-schema-traverse "^1.0.0" + require-from-string "^2.0.2" + uri-js "^4.2.2" + +ajv@^8.8.0, ajv@^8.8.2: version "8.12.0" resolved "https://registry.npmjs.org/ajv/-/ajv-8.12.0.tgz" integrity sha512-sRu1kpcO9yLtYxBKvqfTeh9KzZEwO3STyX1HT+4CaDzC6HpTGYhIhPIzj9XuKU7KYDwnaeh5hcOwjy1QuJzBPA== @@ -2501,7 +2517,14 @@ arg@^5.0.0: resolved "https://registry.npmjs.org/arg/-/arg-5.0.2.tgz" integrity sha512-PYjyFOLKQ9y57JvQ6QLo8dAgNqswh8M1RMJYdQduT6xbWSgK36P/Z/v+p888pM69jMMfS8Xd8F6I1kQ/I9HUGg== -argparse@^1.0.10, argparse@^1.0.7: +argparse@^1.0.10: + version "1.0.10" + resolved "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz" + integrity sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg== + dependencies: + sprintf-js "~1.0.2" + +argparse@^1.0.7: version "1.0.10" resolved "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz" integrity sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg== @@ -3003,7 +3026,7 @@ commander@^5.1.0: resolved "https://registry.npmjs.org/commander/-/commander-5.1.0.tgz" integrity sha512-P0CysNDQ7rtVw4QIQtm+MRxV66vKFSvlsQvGYXZWR3qFU0jlMKHZZZgw8e+8DSah4UDKMqnknRDQz+xuQXQ/Zg== -commander@^7.2.0, commander@7: +commander@^7.2.0: version "7.2.0" resolved "https://registry.npmjs.org/commander/-/commander-7.2.0.tgz" integrity sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw== @@ -3013,6 +3036,11 @@ commander@^8.3.0: resolved "https://registry.npmjs.org/commander/-/commander-8.3.0.tgz" integrity sha512-OkTL9umf+He2DZkUq8f8J9of7yL6RJKI24dVITBmNfZBmri9zYZQrKkuXiKhyfPSu8tUhnVBB1iKXevvnlR4Ww== +commander@7: + version "7.2.0" + resolved "https://registry.npmjs.org/commander/-/commander-7.2.0.tgz" + integrity sha512-QrWXB+ZQSVPmIWIhtEO9H+gwHaMGYiF5ChvoJ+K9ZGHG/sVsa6yiesAD1GC/x46sET00Xlwo1u49RVVVzvcSkw== + commondir@^1.0.1: version "1.0.1" resolved "https://registry.npmjs.org/commondir/-/commondir-1.0.1.tgz" @@ -3171,7 +3199,7 @@ cosmiconfig@^6.0.0: path-type "^4.0.0" yaml "^1.7.2" -cosmiconfig@^7.0.1: +cosmiconfig@^7.0.1, cosmiconfig@>=7: version "7.1.0" resolved "https://registry.npmjs.org/cosmiconfig/-/cosmiconfig-7.1.0.tgz" integrity sha512-AdmX6xUzdNASswsFtmwSt7Vj8po9IuqXm0UXz7QKPuEUmPB4XyjGfaAr2PSuELMwkRMVH1EpIkX5bTZGRB3eCA== @@ -3182,7 +3210,7 @@ cosmiconfig@^7.0.1: path-type "^4.0.0" yaml "^1.10.0" -cosmiconfig@^8.1.3, cosmiconfig@>=7: +cosmiconfig@^8.1.3: version "8.1.3" resolved "https://registry.npmjs.org/cosmiconfig/-/cosmiconfig-8.1.3.tgz" integrity sha512-/UkO2JKI18b5jVMJUp0lvKFMpa/Gye+ZgZjKD+DGEN9y7NRcf/nK1A0sp67ONmKtnDCNMS44E6jrk0Yc3bDuUw== @@ -3641,40 +3669,26 @@ dayjs@^1.11.7: resolved "https://registry.npmjs.org/dayjs/-/dayjs-1.11.7.tgz" integrity sha512-+Yw9U6YO5TQohxLcIkrXBeY73WP3ejHWVvx8XCk3gxvQDCTEmS48ZrSZCKciI7Bhl/uCMyxYtE9UqRILmFphkQ== -debug@^2.6.0, debug@2.6.9: +debug@^2.6.0: version "2.6.9" resolved "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz" integrity sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA== dependencies: ms "2.0.0" -debug@^4.1.0: +debug@^4.1.0, debug@^4.1.1, debug@^4.3.1, debug@4: version "4.3.4" resolved "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz" integrity sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ== dependencies: ms "2.1.2" -debug@^4.1.1: - version "4.3.4" - resolved "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz" - integrity sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ== - dependencies: - ms "2.1.2" - -debug@^4.3.1: - version "4.3.4" - resolved "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz" - integrity sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ== - dependencies: - ms "2.1.2" - -debug@4: - version "4.3.4" - resolved "https://registry.npmjs.org/debug/-/debug-4.3.4.tgz" - integrity sha512-PRWFHuSU3eDtQJPvnNY7Jcket1j0t5OuOsFzPPzsekD52Zl8qUfFIPEiswXqIvHWGVHOgX+7G/vCNNhehwxfkQ== +debug@2.6.9: + version "2.6.9" + resolved "https://registry.npmjs.org/debug/-/debug-2.6.9.tgz" + integrity sha512-bC7ElrdJaJnPbAP+1EotYvqZsb3ecl5wi6Bfi6BJTUcNowp6cvspg0jXznRTKDjm/E7AdgFBVeAPVMNcKGsHMA== dependencies: - ms "2.1.2" + ms "2.0.0" decompress-response@^3.3.0: version "3.3.0" @@ -3855,16 +3869,7 @@ dompurify@2.4.3: resolved "https://registry.npmjs.org/dompurify/-/dompurify-2.4.3.tgz" integrity sha512-q6QaLcakcRjebxjg8/+NP+h0rPfatOgOzc46Fst9VAA3jF2ApfKBNKMzdP4DYTqtUMXSCd5pRS/8Po/OmoCHZQ== -domutils@^2.5.2: - version "2.8.0" - resolved "https://registry.npmjs.org/domutils/-/domutils-2.8.0.tgz" - integrity sha512-w96Cjofp72M5IIhpjgobBimYEfoPjx1Vx0BSX9P30WBdZW2WIKU0T1Bd0kz2eNZ9ikjKgHbEyKx8BB6H1L3h3A== - dependencies: - dom-serializer "^1.0.1" - domelementtype "^2.2.0" - domhandler "^4.2.0" - -domutils@^2.8.0: +domutils@^2.5.2, domutils@^2.8.0: version "2.8.0" resolved "https://registry.npmjs.org/domutils/-/domutils-2.8.0.tgz" integrity sha512-w96Cjofp72M5IIhpjgobBimYEfoPjx1Vx0BSX9P30WBdZW2WIKU0T1Bd0kz2eNZ9ikjKgHbEyKx8BB6H1L3h3A== @@ -4351,11 +4356,6 @@ fs.realpath@^1.0.0: resolved "https://registry.npmjs.org/fs.realpath/-/fs.realpath-1.0.0.tgz" integrity sha512-OO0pH2lK6a0hZnAdau5ItzHPI6pUlvI7jMVnxUQRtw4owF2wk8lOSabtGDCTP4Ggrg2MbGnWO9X8K1t4+fGMDw== -fsevents@~2.3.2: - version "2.3.2" - resolved "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz" - integrity sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA== - function-bind@^1.1.1: version "1.1.1" resolved "https://registry.npmjs.org/function-bind/-/function-bind-1.1.1.tgz" @@ -5517,7 +5517,7 @@ micromatch@^4.0.2, micromatch@^4.0.4, micromatch@^4.0.5: braces "^3.0.2" picomatch "^2.3.1" -"mime-db@>= 1.43.0 < 2", mime-db@1.52.0: +"mime-db@>= 1.43.0 < 2": version "1.52.0" resolved "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz" integrity sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg== @@ -5527,27 +5527,46 @@ mime-db@~1.33.0: resolved "https://registry.npmjs.org/mime-db/-/mime-db-1.33.0.tgz" integrity sha512-BHJ/EKruNIqJf/QahvxwQZXKygOQ256myeN/Ew+THcAa5q+PjyTTMMeNQC4DZw5AwfvelsUrA6B67NKMqXDbzQ== -mime-types@^2.1.27, mime-types@^2.1.31, mime-types@~2.1.24, mime-types@~2.1.34: +mime-db@1.52.0: + version "1.52.0" + resolved "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz" + integrity sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg== + +mime-types@^2.1.27: version "2.1.35" resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz" integrity sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw== dependencies: mime-db "1.52.0" -mime-types@~2.1.17: - version "2.1.18" - resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.18.tgz" - integrity sha512-lc/aahn+t4/SWV/qcmumYjymLsWfN3ELhpmVuUFjgsORruuZPVSwAQryq+HHGvO/SI2KVX26bx+En+zhM8g8hQ== +mime-types@^2.1.31: + version "2.1.35" + resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz" + integrity sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw== dependencies: - mime-db "~1.33.0" + mime-db "1.52.0" -mime-types@2.1.18: +mime-types@~2.1.17, mime-types@2.1.18: version "2.1.18" resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.18.tgz" integrity sha512-lc/aahn+t4/SWV/qcmumYjymLsWfN3ELhpmVuUFjgsORruuZPVSwAQryq+HHGvO/SI2KVX26bx+En+zhM8g8hQ== dependencies: mime-db "~1.33.0" +mime-types@~2.1.24: + version "2.1.35" + resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz" + integrity sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw== + dependencies: + mime-db "1.52.0" + +mime-types@~2.1.34: + version "2.1.35" + resolved "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz" + integrity sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw== + dependencies: + mime-db "1.52.0" + mime@1.6.0: version "1.6.0" resolved "https://registry.npmjs.org/mime/-/mime-1.6.0.tgz" @@ -6416,7 +6435,12 @@ randombytes@^2.1.0: dependencies: safe-buffer "^5.1.0" -range-parser@^1.2.1, range-parser@~1.2.1: +range-parser@^1.2.1: + version "1.2.1" + resolved "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz" + integrity sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg== + +range-parser@~1.2.1: version "1.2.1" resolved "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz" integrity sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg== @@ -7011,33 +7035,37 @@ semver@^5.4.1: resolved "https://registry.npmjs.org/semver/-/semver-5.7.2.tgz" integrity sha512-cBznnQ9KjJqU67B52RMC65CMarK2600WFnbkcaiwWq3xy/5haFJlshgnpjovMVJ+Hff49d8GEn0b87C5pDQ10g== -semver@^6.0.0, semver@^6.1.1, semver@^6.1.2, semver@^6.2.0, semver@^6.3.0, semver@^6.3.1: +semver@^6.0.0: version "6.3.1" resolved "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz" integrity sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA== -semver@^7.3.2: - version "7.5.4" - resolved "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz" - integrity sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA== - dependencies: - lru-cache "^6.0.0" +semver@^6.1.1: + version "6.3.1" + resolved "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz" + integrity sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA== -semver@^7.3.4: - version "7.5.4" - resolved "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz" - integrity sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA== - dependencies: - lru-cache "^6.0.0" +semver@^6.1.2: + version "6.3.1" + resolved "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz" + integrity sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA== -semver@^7.3.7: - version "7.5.4" - resolved "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz" - integrity sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA== - dependencies: - lru-cache "^6.0.0" +semver@^6.2.0: + version "6.3.1" + resolved "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz" + integrity sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA== -semver@^7.3.8: +semver@^6.3.0: + version "6.3.1" + resolved "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz" + integrity sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA== + +semver@^6.3.1: + version "6.3.1" + resolved "https://registry.npmjs.org/semver/-/semver-6.3.1.tgz" + integrity sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA== + +semver@^7.3.2, semver@^7.3.4, semver@^7.3.7, semver@^7.3.8: version "7.5.4" resolved "https://registry.npmjs.org/semver/-/semver-7.5.4.tgz" integrity sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA== @@ -7317,7 +7345,16 @@ string_decoder@~1.1.1: dependencies: safe-buffer "~5.1.0" -string-width@^4.0.0, string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.2: +string-width@^4.0.0, string-width@^4.1.0, string-width@^4.2.2: + version "4.2.3" + resolved "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz" + integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g== + dependencies: + emoji-regex "^8.0.0" + is-fullwidth-code-point "^3.0.0" + strip-ansi "^6.0.1" + +string-width@^4.2.0: version "4.2.3" resolved "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz" integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==