vendkit docs

DR-0016: The engine becomes a pinned, checksummed artefact (INV-6 revision)

Status: accepted (takes effect when the compiled engine ships — DR-0017)

Context

INV-6 ("engine-version") currently holds for free: the sync pipeline pins the publisher at TARGET, and because the engine is interpreted Python living in that same tree, the checkout that supplies the content also supplies the engine that materialises it. A compiled engine (DR-0017) breaks the free ride — running Go from source on every consumer runner would require a Go toolchain, which is heavier than the Python interpreter it replaces and reintroduces a build step into the integrity path.

Decision

  1. The engine is a released binary artefact: built per platform (linux/amd64, linux/arm64, darwin/arm64, windows/amd64) by the framework repo's release pipeline and attached to the release; each release also publishes a checksum file.
  2. Consumers pin the engine explicitly. The slice config for the machinery slice gains engine: {version, sha256: {<platform>: <hex>}}; the scaffolded pipelines' first step is fetch → verify checksum → cache (platform tool cache where available). vendkit self-verify re-asserts the running binary against the pinned expectation.
  3. INV-6 is restated: materialise runs an engine whose declared schema competence covers the target release. Concretely: the engine refuses declaration/manifest schema_versions newer than it knows (already the unknown-key/schema hard-error rule), and a declaration may state min_engine to demand a newer engine explicitly. The sync PR that advances the content pin also advances the engine pin when the target requires it — one reviewed change, no skew window.
  4. The human tier keeps its documented relaxation (cli spec): the installed engine runs against fetched trees, guarded by the same schema gating.

Alternatives considered

  • go run from the pinned checkout — preserves the original INV-6 mechanics but puts a compiler on every runner and a compile in every lane; slower and a larger supply-chain surface than a checksummed binary. Rejected.
  • Vendoring the engine binary into consumer trees — makes the gate hash its own executor (attractive!) but bloats every consumer repo by tens of MB per platform and churns it every engine release. Rejected; the checksum pin gives the same tamper-evidence without the bytes.
  • Keep the engine interpreted — see DR-0017's alternatives; the dependency-free-binary win was judged worth this DR's cost.

Consequences

  • A network fetch and a supply-chain surface enter the integrity path; both are mitigated by the consumer-held checksum pin (a swapped artefact fails verification loudly) and release-pipeline provenance.
  • Offline/air-gapped consumers can mirror artefacts and override the fetch URL; the checksum pin still decides trust.
  • INV-9 strengthens from "stdlib-only Python" to "a static binary with no runtime prerequisites at all" — the gate no longer even needs Python.
  • Until DR-0017 completes, the Python engine still rides the checkout and INV-6 applies in its original form; architecture.md carries a forward note.

Amendment (2026-08): the bootstrap is part of the artefact set

Decision §2 left a gap: the fetch-verify-cache step itself was not shipped, so every adopter wrote it — ~150 lines of bash plus a CI counterpart — and one publisher ended up re-exporting its hand-rolled launcher in its own slice, becoming an unofficial distributor of the engine's entry point. The script that decides whether a downloaded engine is authentic sits inside the trust boundary, so it must be authored, versioned, and released by the project whose checksums it verifies.

Amended decision:

  1. vendkitw is engine-owned and released: authored in-tree (scaffold/vendkitw, embedded in the binary), attached to every release beside the platform binaries, and listed in SHA256SUMS.txt like any other asset.
  2. vendkit init scaffolds it into consumers (all ci modes, byte-identical to the release asset), together with a per-pack reusable vendkit-fetch CI template for consumer-authored pipelines (onboarding spec §2–§3).
  3. The launcher resolves its own pin from committed files (#15): in a consumer, engine.version from .vendkit/consumer/<slice>.yml; in a publisher checkout — identified without guessing by .vendkit/publisher/export-declaration.yml with no .vendkit/consumer/ beside it — from .vendkit/publisher/vendkit-engine.yml, the same key paths one level up (override: VENDKIT_VERSION; escape hatch: VENDKIT_BIN). The release-asset URL likewise: VENDKIT_BASE_URL (a full per-release URL) beats engine.base_url in the same file the pin came from (the version-independent release root — written once, never advanced by the sync PR; the launcher appends /<version>), which beats derivation from publisher: {scm: github, repo} — safe as a fallback because it is correct exactly when the slice's publisher and the engine's releases share a GitHub repo. The launcher caches outside the repository keyed by version/os/arch, verifies before exec, and fails loudly — it never degrades to an unpinned or unverified fetch.
  4. Acquisition is documented and checksummable (onboarding spec §5): a one-time verified download of vendkitw (or of a full binary) from a tagged release — not a package-manager alias that cannot be checksummed in advance, and not a README recipe each adopter retypes.