# How to detect breaking changes between two OpenAPI versions in CI > Fail a pull request whose OpenAPI change would break existing clients: bundle both sides with one tool, run oasdiff, and post its verdict as a comment. Source: https://voxgig.com/howto/detect-breaking-changes-between-openapi-versions-in-ci - Audience: platform-team - Level: intermediate - Verified: 2026-10-15 - Published: 2026-10-15 ## Short answer Check out the base branch beside the pull request, bundle both OpenAPI documents with the same Redocly version, and run oasdiff breaking base.yaml head.yaml --fail-on ERR. Post the markdown report as a pull request comment, and treat any exit status other than 0 or 1 as a failed check, never a clean one. pb33f openapi-changes and OpenAPITools openapi-diff do the same job with different rules. --- ## You will need Node 22 or later, an OpenAPI 3 document tracked in git, and Docker on the runner, which GitHub's `ubuntu-latest` image provides. Verified against Node 22.22.0. The page on making additive API changes without a new version covers what to do once this job blocks a change. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [oasdiff](https://www.oasdiff.com/docs/breaking-changes) | You want a fixed, published rule set and a verdict per endpoint | A Go binary or Docker image in CI, and rule changes limited to ignore files and severity overrides | Your API's idea of breaking differs from the published list | | [pb33f openapi-changes](https://pb33f.io/openapi-changes/summary/) | You want to choose what counts as breaking, rule by rule, in `changes-rules.yaml` | A rules file you now maintain, and a binary the npm wrapper downloads from GitHub at install | You want a rule set someone else maintains and documents | | [OpenAPITools openapi-diff](https://github.com/OpenAPITools/openapi-diff) | A Java shop, or a Docker image already in the pipeline | Its README lists OpenAPI 3.0 support, so a 3.1 document is outside what it promises | Your spec uses 3.1 features | oasdiff publishes its rules:[^1] its documentation lists 339 breaking kinds of change, so a verdict can be checked against a page. openapi-changes [hands that decision to you](https://github.com/pb33f/openapi-changes#custom-breaking-rules-configuration) and costs you a file of rules to keep current. OpenAPITools openapi-diff runs from a Docker image with `--fail-on-incompatible`, at the price of a JVM and a 3.0-only claim. If Optic's `optic diff` is already in your pipeline, plan to replace it: the CLI is archived.[^2] ## Bundle both sides with one tool Diff what the clients are built from, and build both sides the same way. A spec split across files reaches the differ as a different shape from the same spec bundled, and the [oasdiff action](https://github.com/oasdiff/oasdiff-action) refuses external `$ref`s by default. This script runs one [Redocly bundle](https://redocly.com/docs/cli/commands/bundle) over each side. ```js title="bundle-both.mjs" export function bundleBoth(baseDir, headDir, outDir, entry = 'openapi.yaml') { // A stale bundle from an earlier run must never stand in for this one. for (const side of ['base', 'head']) rmSync(join(outDir, `${side}.yaml`), { force: true }) const head = join(headDir, entry) if (!existsSync(head)) { throw new Error(`${head} does not exist: a deleted or moved spec breaks every client`) } const base = join(baseDir, entry) if (!existsSync(base)) return { skipped: `${base} does not exist, so no client depends on it yet` } mkdirSync(outDir, { recursive: true }) const sides = {} for (const [side, file] of [['base', base], ['head', head]]) { const doc = canonical(bundle(file)) const refs = externalRefs(doc) if (refs.length) throw new Error(`${file} still has external refs after bundling: ${refs.join(', ')}`) const text = YAML.stringify(doc) writeFileSync(join(outDir, `${side}.yaml`), text) ``` A missing head fails, because deleting the spec breaks every client at once. A missing base is skipped, because nothing can depend on a spec that does not exist yet. `canonical` sorts keys, so moving a path within the file does not show up in the text a reviewer reads. ```bash node bundle-both.mjs base head .diff ``` ```text output base base/openapi.yaml -> .diff/base.yaml paths 2, schemas 1 head head/openapi.yaml -> .diff/head.yaml paths 2, schemas 1 bundled with @redocly/cli 2.54.3 on both sides the bundles differ: run oasdiff ``` The two bundles differ only where the pull request changed something. ```bash node bundle-both.mjs base head .diff > /dev/null && diff -u --label base --label head .diff/base.yaml .diff/head.yaml ``` ```text output --- base +++ head @@ -15,17 +15,10 @@ type: - string - "null" - status: - enum: - - pending - - paid - - shipped - type: string total: type: number required: - id - - status - total type: object ``` ## Run oasdiff and comment on the pull request Put the oasdiff settings in the repository, where the CLI and the action both find them. ```yaml title=".oasdiff.yaml" fail-on: ERR format: markdown allow-external-refs: false ``` The job checks the base branch out with `git worktree`, bundles both sides, and runs oasdiff 1.33.0 from its Docker image. The [`github.base_ref` context](https://docs.github.com/en/actions/reference/workflows-and-actions/contexts) names the target branch. ```yaml title="workflow/breaking-changes.yml" - name: Bundle both sides with the same Redocly run: node bundle-both.mjs .base/api api .diff - name: Compare with oasdiff id: oasdiff if: hashFiles('.diff/base.yaml') != '' run: | status=0 docker run --rm -v "$PWD:/w" -w /w tufin/oasdiff:v1.33.0 \ breaking .diff/base.yaml .diff/head.yaml > .diff/report.md || status=$? echo "status=$status" >> "$GITHUB_OUTPUT" - name: Comment, then pass or fail if: steps.oasdiff.outcome == 'success' env: GH_TOKEN: ${{ github.token }} STATUS: ${{ steps.oasdiff.outputs.status }} PR: ${{ github.event.pull_request.number }} run: | verdict=0 node comment.mjs "$STATUS" .diff/report.md > .diff/comment.md || verdict=1 gh pr comment "$PR" --edit-last --create-if-none --body-file .diff/comment.md \ || echo "::warning::could not comment; a fork's token is read-only" exit "$verdict" ``` The workflow itself was not run for this page. `bundle-both.mjs` and `comment.mjs` ran as shown on this page, and oasdiff 1.33.0 ran locally from a `go install` build rather than the image. oasdiff exits 0 for no breaking change and 1 when `--fail-on` matches, as its [breaking-changes guide](https://github.com/oasdiff/oasdiff/blob/main/docs/BREAKING-CHANGES.md) says. Run locally on a spec it could not load, it exited 102 with an empty report. `comment.mjs` turns an empty report into "no breaking changes" only when the status is 0. It refuses a status that is not a number, because a misspelled step output expands to an empty string, which JavaScript's `Number` reads as 0. The report below is a recording, not a live run. oasdiff 1.33.0 produced it locally from the two bundles built earlier, and it is committed as `fixtures/oasdiff-breaking.md`. The captured command runs `comment.mjs` over that file, so CI never runs oasdiff. ```bash node comment.mjs 1 fixtures/oasdiff-breaking.md ``` ```text output ### oasdiff: breaking changes # API Changelog 2.4.0 vs. 2.4.0 ## API Changes ### GET /orders - :warning: removed the required property `items/status` from the response with the `200` status ### GET /orders/{orderId} - :warning: removed the required property `status` from the response with the `200` status ``` oasdiff reports a change once for each endpoint it reaches, so one schema change produced two findings. The header compares 2.4.0 with 2.4.0, because the pull request left `info.version` alone.[^3] ## Check it worked Open a pull request that removes a required property from a response schema, as `head/` does here. Run locally against the two bundles, oasdiff printed two `response-required-property-removed` errors and exited 1. The job comments the report and goes red. Then confirm the job cannot go green by crashing. ```bash node comment.mjs 102 .diff/report.md ``` ```text output ### oasdiff did not run It exited 102 before comparing anything, so this pull request is unchecked. See the job log. ``` ```bash node --test ci.test.mjs ``` ```text output 1..9 # tests 9 # suites 0 # pass 9 # fail 0 ``` The tests cover a reordered copy of the base bundling to the same bytes, a `$ref` that does not resolve, a deleted spec, the crash just shown, and an empty status. ## When it goes wrong The action fails on every pull request with `encountered disallowed external reference`. Its `allow-external-refs` input defaults to `false`, which blocks a split spec. Bundle first, as `bundle-both.mjs` does, and keep the default, which guards against SSRF on untrusted pull requests. openapi-changes reports a breaking change on a pull request that changed nothing. Run locally on a bundled base and the same spec unbundled, it reported "Removed schemas" as breaking, and oasdiff's `changelog` reported `api-schema-removed`. The documents were equivalent but not the same shape. Bundle both sides with the same tool and version. The job overwrites another workflow's comment. `gh pr comment --edit-last` edits the last comment by the current user, per the [GitHub CLI manual](https://cli.github.com/manual/gh_pr_comment), and every workflow using `GITHUB_TOKEN` comments as the same bot. Start the body with a marker line and find the comment by it with `gh api` instead. An encrypted copy of your spec leaves CI. The action's [`review` input](https://www.oasdiff.com/docs/github-action) defaults to `true` and uploads the comparison to oasdiff.com when it finds changes. Set `review: false` to keep the spec in CI. ## When not to do this Do not gate a spec that no external client builds from. An internal API whose callers deploy with it gets more from contract tests than from a rule list written for strangers. Do not let the job become the versioning policy. oasdiff says a change breaks clients. A person decides whether that means a new major version, a deprecation window, or a refused merge, and records it where a reviewer can find it. Do not silence findings with `err-ignore` to get a release out. Each ignored line is a break you shipped on purpose, so write down who agreed to it. Do not compare against the branch tip when clients are built from releases. If consumers use tagged versions, diff the pull request against the last release tag instead of `main`. ## Related how-tos - [Split and bundle a large OpenAPI document](/howto/split-and-bundle-an-openapi-document) - [Detect a reference site drifting from its OpenAPI spec](/howto/check-a-reference-site-in-ci) - [Evolve an error contract without breaking clients](/howto/evolve-an-error-contract-without-breaking-clients) - [Publish an OpenAPI 3.0 copy beside your 3.1 document](/howto/publish-openapi-3-0-alongside-3-1) ## Last verified Verified 2026-10-15 against Node 22.22.0, `@redocly/cli` 2.54.3 and `yaml` 2.9.1. Every output block is what the preceding command printed in the page's code directory after `npm ci`. oasdiff 1.33.0 and openapi-changes 0.2.13 were built with `go install` and run locally, outside the captured runs. The workflow was not run on GitHub. [^1]: The [breaking changes reference](https://www.oasdiff.com/docs/breaking-changes) counts 755 kinds of change across ten areas of an OpenAPI document: 339 breaking, 17 warnings and 399 informational. The three add up to the total, and each kind has an identifier such as `response-required-property-removed`, so a verdict can be traced to a line in a list. [^2]: The fullest account of Optic's retirement is a [migration guide](https://www.oasdiff.com/docs/migrate-from-optic) published by oasdiff. It notes that Optic's team joined Atlassian, that the CLI is archived, and that the archived CLI still installs from npm. Some teams, it adds, keep Optic for capturing traffic and use oasdiff for the gate. [^3]: The [OpenAPI Specification](https://spec.openapis.org/oas/v3.1.0.html#info-object) defines `info.version` as the version of the OpenAPI document and adds, in parentheses, that it is distinct from the specification's version and from the implementation's. It says nothing about when the number must move. A pull request that removes a required field is free to leave it alone, and this one did.