Managing Schema Changes in Data Product APIs

Illustrative API compatibility matrix: adding a field needs strict-reader and permission checks; rename, removal and type changes need consumer migration or a compatible representation.

Manage a data product API schema change as a consumer migration, not just a database edit. Compare the old and proposed response, identify the applications that depend on it, preserve a compatible path where possible, and retire the old contract only after the affected owners have evidence that their consumers can move.

A field addition can still break a strict client. A rename can break a parser even when the underlying value stays the same. A response can remain valid JSON while changing business meaning. The practical question is therefore: will existing consumers still receive data they can interpret and are authorized to use?

This guide provides a compatibility matrix, consumer inventory and release worksheet. The examples and proposed checks are synthetic architecture recommendations; they are not executed test results or a description of automatic Elementrix versioning or impact analysis. Start with the data product specification template when the initial contract has not yet been agreed.

Classify additive and breaking changes

Compare the consumer-facing contract, not only the source table. Include field names, types, nullability, allowed values, meaning, units, nesting, ordering, filters and error behavior. A source column can be renamed behind a stable API mapping without changing the published field; a source change becomes a consumer change when its effects reach that interface.

Google’s API compatibility guidance distinguishes source, wire and semantic compatibility. It treats removal and renaming as incompatible changes and cautions against changing an existing field’s type. Apply those principles to your actual clients and transport rather than assuming one rule covers every API.

Illustrative API compatibility matrix: an added response field needs tolerant-reader and permission checks; a rename, removal or type change requires a consumer migration or a compatible representation. Source schema changes do not automatically authorize new response fields.
Illustrative change assessment. A release decision depends on the published contract, actual clients and approved scope.
Compatibility worksheet for a JSON response API
Proposed changeRisk to an existing consumerRecommended release decision
Add response field service_regionA tolerant reader may ignore it; a strict decoder or closed schema may reject it. It may also reveal data outside an app’s grant.Check the contract, client behavior and field authorization. Keep unchanged consumers on their agreed representation when necessary.
Rename region to service_regionA consumer looking up region loses its input.Preserve the old representation during migration or introduce an explicitly selected new version. Do not silently replace the key.
Remove statusRouting or eligibility logic may no longer have a required value.Identify dependencies and migrate them before removal. A field being unused by one app says nothing about others.
Change customer_id from string to numberParsing, joins and stored identifiers can change; leading zeros may be lost.Keep identifiers stable. Design a separate representation if a different type is necessary and test both paths.

Additive does not mean safe by default

A response validator using additionalProperties: false can reject an unfamiliar field; the JSON Schema object reference explains this control. Test the real deserializer and validation settings, including generated clients. For requests, adding a new required input without compatible behavior for old callers is a separate breaking risk.

Also inspect changes that preserve the shape: converting currency units, expanding a response enum, making previously populated data null, or changing the default sort. A decoder may accept a new value while application logic makes the wrong decision. Document the expected behavior for unknown values rather than assuming every client tolerates them.

Find the consumers affected by a field change

Create a small dependency inventory before choosing the release date. Combine registered consuming applications, access grants, supported API versions, available request telemetry and owner interviews. Logs can show calls to an endpoint without showing which response fields an application actually reads. Treat missing evidence as an investigation task, not proof that a field is unused.

Synthetic inventory for a proposed region-field rename
Consumer and owner roleDependencyMigration evidence to collect
Routing app / application ownerReads region on each customer lookup.Named client version, updated mapping and representative routing acceptance cases.
Nightly export / integration ownerWrites the field to a downstream file header.Updated export and confirmation from the receiving job owner.
Quarterly reconciliation / finance data ownerUses stored exports and runs infrequently.Replay a representative historical batch; agree handling of old files.

For each consumer, record its approved fields and record boundary, contact, deployment lead time, last confirmed use and fallback. Include scheduled jobs, partner integrations, reports, SDKs, caches and retained files. An observation window that misses a quarterly job cannot establish that the old contract is unused.

The existing data product guide introduces ownership and consumption. This inventory makes ownership operational for a specific change. A lineage tool can contribute evidence where available, but it should not be presented as complete coverage of every external application or copied file.

Version the schema and communicate a migration window

Keep the contract artifact, implementation release and consumer-selected API version distinct in the change record. A schema file can have a new revision without forcing every client to change its endpoint. Conversely, changing a version label does not itself create a stable implementation or a migration path.

Worked example: preserve the old field while introducing the new contract

In this synthetic example, the team decides to replace the ambiguous name region with service_region. The business meaning remains the service-routing area, and the app’s approved record scope stays unchanged. Two illustrative read-only routes make version selection explicit:

GET /example/v1/customer-status/C-1042
{"customer_id":"C-1042","status":"active","region":"Cairo"}

GET /example/v2/customer-status/C-1042
{"customer_id":"C-1042","status":"active","service_region":"Cairo"}

These are example routes, not live Elementrix endpoints. A compatible serving mapping could keep the v1 key stable while sourcing its value from a renamed column. That adapter must preserve meaning, null handling and authorized scope. If both names are temporarily exposed in one representation, define their relationship and verify clients tolerate the addition; do not assume duplicate keys solve compatibility.

Select a versioning approach that the existing API can support, such as a path or negotiated representation. Record routing and cache behavior so clients do not accidentally receive a different shape. Keep authorization independent of version selection: requesting v2 must not grant additional fields or records.

Copyable migration notice and release record
FieldWhat to write
Change and reasonProduct, old/new schema references, exact field differences and business rationale.
Affected consumersNamed application owners, dependencies, approved scope and unconfirmed consumers.
Migration windowAvailability of the replacement, acknowledgement deadline and proposed retirement date, with timezone.
Required consumer actionMapping or SDK changes, example payloads, validation steps and support contact.
Release gatesRequired compatibility and permission checks, owner acceptance and unresolved blockers.
Recovery planRollback decision owner, trigger, compatible source/mapping and limits of recovery.

Choose the window from actual deployment cycles, partner commitments and usage frequency. There is no universal safe number of days. Send the notice through the channels used by the affected owners, track acknowledgements and resolve nonresponses. Documentation alone does not prove a consumer is ready.

Test old and new consumers

Use synthetic fixtures and representative client implementations to check both the shape and the task. Keep the old client unchanged when testing it against the proposed release; otherwise the test no longer answers whether the old contract survived. Record expected behavior before running checks, then retain actual results separately.

  • Old client, old representation: identifiers, field names, nulls, meaning and authorized records remain consistent.
  • New client, new representation: the renamed field supports the intended routing task and documented missing-value handling.
  • Strict and tolerant readers: extra fields, new enum values and serialization changes produce the agreed behavior.
  • Permission boundaries: excluded fields, nested data and out-of-scope records stay unavailable through both versions and alternate routes.
  • Operational behavior: paging, filters, sorting, errors and freshness still meet the agreed consumer requirements.
  • Recovery: rehearse the rollback or forward-fix plan against the source representation that will actually exist.

Schema comparison tools can identify structural differences, but they cannot establish that “active” still means the same thing to a routing application. Add business acceptance cases and owner review. None of the checks listed here has been executed for a customer deployment.

Deprecation, rollback and owner sign-off

Deprecation communicates that a contract should be replaced; retirement removes the old access path. Keep those decisions separate. RFC 9745 defines the HTTP Deprecation response header and a relation for linking to more information. RFC 8594 defines Sunset for a URI expected to become unresponsive at a future time. Use these signals where your implementation supports them, alongside direct migration communication. A header is not a substitute for owner acknowledgement.

Before retiring v1, confirm that the inventory is resolved, consumers have accepted the replacement, agreed checks pass and observation covers relevant job cycles. Document exceptions with an owner and a review date. Restrict creation of new dependencies on the deprecated version according to the agreed policy.

A rollback needs a viable data path

Reverting application code may not restore a deleted source column, old enum meaning or transformed historical data. Retain the compatible mapping and source representation for the agreed recovery period, or explicitly choose a forward fix when reversal is infeasible. Define recovery triggers such as decoding failures or incorrect business decisions, who can make the call and how consumers will be informed. Do not roll back to insecure access rules to recover an old response.

Have the product owner accept the consumer impact, the API owner accept serving and recovery behavior, the data owner accept meaning and the relevant access reviewer accept permission boundaries. Keep evidence with the schema change record. After release, monitor version use and support incidents, update the inventory and only then close the migration.

For the broader delivery model, see how governed data products standardize enterprise delivery. To discuss your own change, walk through a schema change and consumer impact in an Elementrix demo.