Upgrading to v5

NSchema v5 is a rearchitecture of the engine, with the goal of better consistency, safety, and transparency. The headline changes for a project are that plugins are as follows:

• Plugin versions now support NuGet ranges, but specific versions are pinned by a lockfile for CI/CD.
• Planning is always offline, so a backend state store is now required.
• The language separates declarations from directives. RENAMED FROM, PARTIAL, DEPLOYMENT, MIGRATION and DROP are gone in favor of RENAME and SCRIPT directives and better state management.

This page lists what changed and the concrete edits you need to make.

Lockstep majors

The CLI, Core, and the first-party providers and state stores all move to 5.x together: a 5.x CLI expects 5.x plugins.

The short version

Most projects need these edits:

1. Replace the PROVIDER / BACKEND blocks with DATABASE / STATE statements and add PLUGIN declarations.
2. Run nschema init to install the plugins and write the nschema.lock lockfile. You should add this to version control.
3. Configure a state store if you didn’t have one, and nschema refresh to seed it.
4. Replace RENAMED FROM clauses with RENAME directives, and delete DROP statements and PARTIAL markers.
5. Replace any remaining PRE|POST DEPLOYMENT / MIGRATION blocks with SCRIPT statements.
6. Re-record your run-once script ledger (see State).

Configuration

PROVIDER and BACKEND become DATABASE and STATE

PROVIDER and BACKEND came from “database provider” and “state backend”, but without context, both descriptors are meaningless. DATABASE and STATE more accurately describe the thing it configures. Neither carries a version any more, but instead plugin dependencies are declared separately, by a PLUGIN statement, and referenced by label:

-- v4
PROVIDER postgres (
  version = '4.0.0',
  connection_string = ''
);

BACKEND s3 (
  version = '4.0.0',
  bucket = 'my-state',
  key = 'prod/state.json'
);

-- v5
PLUGIN postgres (
  source  = 'NSchema.Postgres',
  version = '[5.0,6.0)'
);

PLUGIN s3 (
  source  = 'NSchema.Aws',
  version = '5.0.0'
);

DATABASE postgres (
  connection_string = ''
);

STATE s3 (
  bucket = 'my-state',
  key = 'prod/state.json'
);

Two things to note:

• The built-in label-to-package map is gone. postgres used to resolve to NSchema.Postgres automatically. Every plugin declares its package explicitly with a source attribute.
• STATE file is the one exception. The local-file store is built into the engine, so it needs no PLUGIN declaration.

Add an ENGINE assertion

Version 5 now supports an optional ENGINE block for engine configuration similar to plugins. Use the version attribute to ensure that an unwanted version of NSchema can’t ever run against your project by accident.

ENGINE (
  version = '[5.0,6.0)',
  host_version = '[5.0,6.0)'
);

Lock the plugins

A declared plugin range must be resolved to a concrete version before it can be used. Run:

nschema init

This writes nschema.lock beside your configuration, pinning each plugin to the version it resolved to. Commit it. It’s what makes plans reproducible across machines and CI. You can use plugin outdated and plugin update to keep plugins updated, or just edit your PLUGIN versions by hand.

Configuration

A configuration statement is read wherever you write it, so it needs no marker; the convention new writes is config.sql. Only the environment overlays are marked: an *.env.<name>.sql file is read only when that environment is selected.

Overlay files mutate the base config rather than replacing it, so setting connection_timeout in the base config now applies in your environment overlay too, unless it’s overridden.

The --backend/--provider options on new were renamed

nschema new --provider postgres --backend s3 is now nschema new --database postgres --state s3.

Planning and state

A state store is now required

plan, apply, and destroy all require a STATE statement. Planning against the live database is no longer possible.

For a disposable database, like for integration tests, you can use the --ephemeral flag instead of configuring a store. It runs against an in-memory state store discarded when the command exits.

destroy reads the managed set from state

destroy used to fall back to your working-directory schema when no store was configured. It now always reads what to tear down from the recorded state, and drops only what NSchema manages. Objects that were merely observed are left alone. destroy also sets the destructive-action policy to allow for you, but still requires a confirmation prompt.

State

Two changes to the recorded payload:

• State carries a managed set. NSchema now independently records which objects are managed by your project, so removing an object from your project causes it to be dropped without a DROP statement.
• The run-once script ledger’s field was renamed. A pre-5.0 payload reads as an empty ledger, so every RUN ONCE script would run again.

Under the state compatibility policy this is a major-version change. After upgrading, re-record the ledger:

nschema refresh
nschema script untaint seed-users      # one per run-once script that has already run

script hash lists the scripts you may need to untaint.

The language

Renames are directives

The RENAMED FROM clause on a declaration no longer parses. A rename is now a statement of its own, so a declaration describes only the shape you:

-- v4
CREATE TABLE app.accounts RENAMED FROM users ( … );

-- v5
RENAME TABLE app.users TO accounts;

CREATE TABLE app.accounts ( … );

The source is fully qualified and the target is a bare name. Remove the directive once the rename has been applied everywhere.

DROP statements and PARTIAL SCHEMA are gone

DROP <type> and CREATE PARTIAL SCHEMA no longer parse.

• To drop an object, just delete its declaration. NSchema will drop the object automatically.
• PARTIAL was a way of saying “don’t drop what I haven’t declared”. The managed set does that properly now: NSchema never drops an object it doesn’t manage, partial marker or not.

Extensions follow the same rule as everything else in v5: a declared extension becomes managed when it is applied, and is dropped when you delete the declaration. One that NSchema never created is still never dropped.

Scripts

The deprecated forms no longer parse:

-- v4 (removed)
PRE DEPLOYMENT 'enable_citext' AS $$ … $$;
MIGRATION 'backfill' FOR ADD COLUMN app.users.email AS $$ … $$;

-- v5
SCRIPT enable_citext RUN ON PRE DEPLOYMENT AS $$ … $$;
SCRIPT backfill RUN ON ADD COLUMN app.users.email AS $$ … $$;

The SCRIPT form (introduced in 4.4) requires an identifier, where the old MIGRATION form allowed an anonymous block or a string literal.

Scripts are also now part of the diff rather than a separate section of the plan: change-event scripts appears on the change they attach to. If you consume --json, they ride the diff object instead of the old top-level dataMigrations / deployment-script lists.

Identifiers are case-sensitive, and can be quoted

An identifier’s identity is now its exact written text: users and Users are two different tables. If you relied on case-insensitive matching, seek therapy. And also normalize your declarations before upgrading.

Names can now be quoted, so a name may carry spaces, dots, or a reserved word:

CREATE TABLE app."Order Details" ("weird ""col""" int);
CREATE TABLE app.[Order Details] ([weird ]]col]]] int);   -- brackets are an accepted alternative spelling

Please don’t ever name your columns either of these examples.

--scope takes an address

--scope used to only take schema names. It now takes an address, so a run can target a single object:

nschema plan --scope app                              # a whole schema, as before
nschema plan --scope app.orders                       # one table
nschema plan --scope '"my.schema"."Order Details"'    # quoted segments carry dots and spaces

Schema-only scopes continue to work unchanged.

CLI changes

v4	v5
fmt, db, scaffold	format, database, new (the old names are gone)
scaffold --provider / --backend	new --database / --state
—	new prompts for what the plugins need; --set k=v answers ahead
init (restore only)	init resolves, locks, and restores
—	plugin update [<label>], plugin outdated
—	--ephemeral on plan / apply / destroy
--destructive-actions error|warn|allow	adds ignore

Other behavioral changes worth knowing:

• apply --plan-file re-runs the policies. A saved plan is re-checked immediately before it executes.
• A policy-blocked plan still renders in full. Being blocked means “may not apply”, not “stopped computing”: you get the complete diff and SQL alongside the diagnostics.
• state show reports an error when nothing has been recorded yet, instead of failing on a missing source.
• script hash / taint / untaint address a template-scoped script as schema.name, as script hash lists it.
• new is interactive. With no arguments it asks which database and state store to use, and then whatever those plugins need to know, writing the answers into the configuration.

Still on v3?

Work through the v3 → v4 guide first, then this page.