Skip to content

Data hazards

Some changes are perfectly valid against the schema, but can fail against the data when the migration runs. The classic example is adding a NOT NULL column without a default: on an empty table it succeeds, but if the table has rows, the database refuses mid-deployment. NSchema calls these data hazards and flags them at plan time, with a suggested fix.

On tables that already exist (a freshly created table is empty, so nothing in it can fail on data):

Change Why it can fail Typical fix
Adding a NOT NULL column with no default Existing rows have no value for it Declare a DEFAULT — on PostgreSQL 11+ this fills existing rows without rewriting the table
Changing a column to NOT NULL Existing rows may hold NULLs Backfill the NULLs first
Changing a column’s type where the cast can fail (e.g. textint, varchar(100)varchar(50), bigintint) Stored values may not fit or parse into the new type Verify the data fits, or migrate through a backfill
Adding a primary key or unique constraint/index over existing columns Existing rows may hold duplicates (or NULLs, for a key) De-duplicate first, or scope uniqueness to the new columns

Identity and generated columns are exempt (they compute their own values), as is uniqueness confined to columns added in the same change (a new column starts empty). Type changes involving a custom type the engine has no knowledge of are not flagged.

The reaction is controlled by the --data-hazards option:

Value Behaviour
error A hazardous change fails the run. Nothing is applied.
warn (default) Each hazard is reported as a warning, but the run continues.
allow Hazards are reported as informational notes only.
ignore Hazards are not reported at all.
Terminal window
# environment variable
export NSCHEMA_DATA_HAZARD_POLICY=error
Terminal window
# command-line flag (highest precedence)
nschema apply --data-hazards error

The option applies to both plan and apply.

Resolving a hazard with a migration script

Section titled “Resolving a hazard with a migration script”

Declaring a matching change-event SCRIPT silences the corresponding hazard: the script states how the data gets into shape (a backfill before SET NOT NULL, a de-duplication before a unique constraint), so there is nothing left to warn about. The plan output shows the script against its change instead.

Like every diff policy, data-hazard detection applies to forward migrations. A teardown (destroy, plan --destroy) bypasses the diff and its policies entirely.