Migration Examples

One recipe per use case. Structural changes (columns) are declarative [[op]] steps; content transforms that need computation or row surgery are script steps. Overview, workflow, and the startup gate: Data Migrations.

A new column is added

Cells stay empty; the workspace owner fills them (or the release declares a column default in module.toml [assets]).

# app.csv — before                # app.csv — after
name,customer                     name,customer,zone
web1,acme                         web1,acme,

# migrations/001-app-zone.toml
from = 1
to = 2
targets = ["app.csv"]

[[op]]
op = "add"
column = "zone"

A new column is added with a default value

# app.csv — before                # app.csv — after
name,customer                     name,customer,retention_days
web1,acme                         web1,acme,30

# migrations/001-app-retention.toml
from = 1
to = 2
targets = ["app.csv"]

[[op]]
op = "add"
column = "retention_days"
default = "30"

add appends at the end; after = "customer" inserts mid-header. It is a no-op when the column already exists — re-runs are safe.

A new column is added based on two others

derive renders a Tera template over the row’s columns — the same template syntax used everywhere in rescile:

# app.csv — before                # app.csv — after
name,customer                     name,customer,server_name
web1,acme                         web1,acme,acme-web1

# migrations/001-app-server-name.toml
from = 1
to = 2
targets = ["app.csv"]

[[op]]
op = "derive"
column = "server_name"
template = "{{ customer }}-{{ name }}"

derive also transforms a column in place — the template reads the column’s old value. Encoding a cell is a one-liner (the b64_encode Tera filter ships with rescile):

# utils.csv: hash column "sha256" -> "sha256-c2hhMjU2"
[[op]]
op = "derive"
column = "hash"
template = "{{ hash }}-{{ hash | b64_encode }}"

A column is renamed

Header text moves verbatim, so ~/_ prefixes move with it (and can be added or removed by naming them in to).

# app.csv — before                # app.csv — after
name,env                          name,environment
web1,prod                         web1,prod

# migrations/001-app-env-rename.toml
from = 1
to = 2
targets = ["app.csv"]

[[op]]
op = "rename"
from = "env"
to = "environment"

map sets specific values based on other columns — first matching [[op.mapping]] rule wins, unmatched rows get default (or keep their value without it):

[[op]]
op = "map"
column = "retention_days"
default = "180"

[[op.mapping]]
when = { name = "web1" }
value = "365"

[[op.mapping]]
when = { environment = "staging" }
value = "14"

A column must be split — with computation: script

network.csv replaces one cidr column (10.0.0.0/24) with ip_v4 and a dotted netmask_v4 (255.255.255.0). Splitting the cell is trivial, but the prefix length → dotted mask conversion is arithmetic — integer shifts, not a template. Computation belongs in a script step:

# network.csv — before             # network.csv — after
name,cidr                          name,ip_v4,netmask_v4
frontend,10.0.0.0/24               frontend,10.0.0.0,255.255.255.0
backend,192.168.100.0/25           backend,192.168.100.0,255.255.255.128

# migrations/001-network.toml
from = 1
to = 2
targets = ["network.csv"]
description = "cidr becomes ip_v4 + dotted netmask_v4"

// migrations/001-network.rhai
let rows = read_csv("network.csv");
if !rows[0].contains("cidr") {
    return;   // idempotency guard: after migration the column is gone
}
for i in 0..rows.len() {
    let parts = rows[i]["cidr"].split("/");
    let prefix = parse_int(parts[1]);
    let bits = 0xFFFFFFFF << (32 - prefix);
    rows[i]["ip_v4"] = parts[0];
    rows[i]["netmask_v4"] = ((bits >> 24) & 255) + "." + ((bits >> 16) & 255) + "." + ((bits >> 8) & 255) + "." + (bits & 255);
}
write_csv("network.csv", ["name", "ip_v4", "netmask_v4"], rows);

A cell list becomes one row per entry: script

instance_admins.csv kept a comma list; the new format normalizes it into one row per admin. Ops are column-only by design — changing rows is script territory:

# instance_admins.csv — before     # instance_admins.csv — after
instance,admins                    instance,admin
web1,"alice,bob,carol"             web1,alice
web2,alice                         web1,bob
                                   web1,carol
                                   web2,alice

# migrations/002-admin-rows.toml
from = 2
to = 3
targets = ["instance_admins.csv"]
description = "Expand the admins comma list into one row per admin"

// migrations/002-admin-rows.rhai
let rows = read_csv("instance_admins.csv");
if !rows[0].contains("admins") {
    return;   // the expansion destroys its own input column
}

let out = [];
for i in 0..rows.len() {
    let admins = rows[i]["admins"].split(",");
    for j in 0..admins.len() {
        let row = #{};
        row["instance"] = rows[i]["instance"];
        row["admin"] = admins[j];
        out.push(row);
    }
}
write_csv("instance_admins.csv", ["instance", "admin"], out);

When a step must be a script

Reach for a script when the change is not column-shaped:

• Row surgery — adding, filtering, or deduplicating rows (the map-key trick: keys of a #{} map iterate sorted, so dedup and deterministic order come together)
• Cross-file enrichment — read two declared targets, join values from one into the other
• Computation on cell values — arithmetic, encodings beyond the Tera filters
• JSON restructuring in input/ — column operations only support CSV

Script API reference

Scripts are Rhai, executed inside the controller, sandboxed to the declared targets. No shell, no runners, no network, no environment access.

Script paths are root-relative, like targets: CSV files live under assets/, JSON under input/ — read_csv("instance.csv") reads assets/instance.csv, and read_csv("subdir/instance.csv") reads a CSV in a subdirectory of assets/.

Rules that matter when writing scripts:

• Mutate rows by index. for row in rows iterates over copies — use for i in 0..rows.len() and assign through rows[i][...].
• Cell values are raw strings. No importer type coercion runs on migration output; ~-prefixed headers appear in row objects verbatim (row["~jwk"]).
• String methods mutate in place and return (): load into a local, mutate it, assign it back.
• Steps must be idempotent: an interrupted run re-plans from the last recorded step and may re-run the interrupted one. Guard on something the step destroys (a column it removes or renames) — both script examples above do exactly that.
• Renaming via script (when combined with row logic): copy the values into the new key and write with an explicit header that lists the new name and not the old one.

Restructuring input/*.json

// 003-config-shape.rhai — array becomes map, scalars are kept
let cfg = read_json("config.json");
let zones = cfg.zones;               // was: ["eu", "us"]
let map = #{};
for i in 0..zones.len() {
    map[zones[i]] = true;            // now: { "eu": true, "us": true }
}
cfg.zones = map;
write_json("config.json", cfg);