Databricks delivery

Runway's job at the workspace edge is to make what you gated exactly what you deploy, and to make the deploy itself boring: a wrapped databricks bundle deploy, run by a trusted worker, recorded as an audited event. This page walks the full path from a built bundle to a running Databricks App.

The immutable artifact contract

The core idea is content addressing. A build is uploaded to a Unity Catalog Volume at a path derived from its own SHA-256 tree digest, and the digest is re-verified at the workspace before anything is deployed.

1. Stage (CLI side). fr deploy computes the Harness tree digest of the build directory and uploads it to a content-addressed path: artifactRef = <artifactVolume>/<app-slug>/<sha256>. Symlinks and non-file entries are rejected. The deploy request carries only the /Volumes/... reference and the 64-character SHA-256 digest — never the bytes.
2. Register. runway.artifact_register records the source revision, Harness version, and App/worker component manifest only after workspace verification succeeds. It returns a stable content-derived artifactId. Every required manifest entrypoint must exist inside the verified bytes; traversal and absolute paths are rejected. Production deploy actions accept that id, not caller-entered paths.
3. Materialize + verify (workspace edge). The Databricks adapter streams the Volume object into an isolated temp directory (with path-traversal protection), recomputes the digest, and throws on any mismatch before the bundle is validated. Tampering between stage and deploy is caught here.
4. Deploy. Only then does the adapter run databricks bundle validate and databricks bundle deploy --target <env> from the bundle root.

Schema enforces the shape: artifactRef must match ^/Volumes/<catalog>/<schema>/<volume>/… and artifactDigest must be a lowercase 64-hex SHA-256.

Asset Bundles

The scaffolded databricks.yml is a standard Databricks Asset Bundle with one target per environment. Runway passes the target and a per-environment secret scope variable (BUNDLE_VAR_secret_scope = runway-<appId>-<environment>) into the CLI; secrets are passed over stdin, never argv, and configured secret strings are scrubbed from all CLI stdout, stderr, and error messages.

For previews it first creates an isolated Lakebase copy-on-write branch and passes BUNDLE_VAR_lakebase_branch plus BUNDLE_VAR_lakebase_endpoint. It also passes BUNDLE_VAR_preview_suffix=-pr-N; the bundle root, App name, secret scope, Unity Catalog resources, and other non-database resources are therefore owned by that exact pull request and can be destroyed without touching another preview. See preview environments for lifecycle and failure behavior.

resources:
  apps:
    support-agent:
      name: support-agent-${bundle.target}
      source_code_path: ./
      resources:
        - name: ai-search-index
          secret:
            scope: ${var.secret_scope}
            key: ai-search-index

Databricks Apps

The template ships a long-running Databricks App. Its app.yaml declares the runtime command and the environment it reads, including secret-backed values resolved from the app's scope:

command: ["node", "dist/server.mjs"]
env:
  - name: DATABRICKS_APP_PORT
    value: "8080"
  - name: DATABRICKS_MODEL
    value: system.ai.gpt-oss-20b
  - name: DATABRICKS_AI_SEARCH_INDEX
    valueFrom: ai-search-index

After bundle deploy, Harness activates the App snapshot so the new revision serves traffic. Both app.yaml and databricks.yml are regenerated by the build step (fh build --target databricks-app) — you edit the agent, not the wiring.

Inside the installed Runway App, each governed adapter attempt obtains a fresh short-lived token from the App service principal and supplies it to Harness' CLI edge. The token is never a Platform action parameter or event field, and retries resolve a new credential rather than persisting an expiring bearer token. Harness command failures preserve stderr—or a stdout diagnostic when a CLI generation emits errors there—after Runway scrubs the short-lived token and client secret.

The same App exposes the shared Runway MCP registry at /api/v1/orgs/<org>/mcp. The route uses the App's verified user boundary and tenant membership, then delegates every tool to the existing org-scoped API. Register that URL as a schema-scoped HTTP connection and Unity Catalog MCP Service when AI Gateway or Agent Bricks should call Runway. See MCP tools for tool selection, permissions, health, expected behavior, and failure modes.

The customer build packages a pinned official Linux Databricks CLI archive after validating its published SHA-256. It splits the archive below the Databricks App per-file limit and re-verifies it before extraction at startup, so deployment does not inherit an outdated CLI from the managed App image or require runtime GitHub egress.

Durable external-effect evidence

Runway does not implement Databricks authentication or a second governance pipeline. Production composes Platform Host 0.5 with @fabric-harness/databricks/platform. Before policies run, the host persists the exact Databricks resource footprint and the App service principal delegated by the requesting actor. After each successful workspace adapter call, it records an execution attestation that identifies the operation and resource using only redacted adapter input and audit-safe output.

These records live in the same Lakebase transaction boundary and tenant/space namespace as action, policy, event, and adapter records. The bridge cannot serialize a bearer token or client secret. Runway's existing multi-approver release model remains the approval authority; the optional single-decision Host HITL evaluator stays unconfigured.

What a client workspace must provide

To run this path for real (not the local demo), a workspace needs:

• A Unity Catalog artifact Volume with WRITE_VOLUME, referenced by artifactVolume.
• The databricks CLI authenticated (or a service principal via --profile).
• A Lakebase / PostgreSQL database for the Platform Host durable store.
• CAN MANAGE on the configured Lakebase project so the trusted App principal can create and delete preview branches; branch TTL defaults to 24 hours.
• For Gateway-correlated evidence, a separate account-admin-authorized evidence reader and a SQL warehouse. The current Gateway Beta documents system.ai_gateway.usage as account-admin-only; do not elevate the ordinary Runway App principal solely to read it.
• A trusted worker service principal to execute prod (RUNWAY_TRUSTED_WORKER_IDS).
• Databricks Apps' default current-user and access-control read scopes for OBO workspace-group resolution; Runway does not request a broader user API scope.
• A Fabric Experiments API endpoint, immutable organization id, and release-to-dataset plus evaluator resolver for gates. Runway business-unit tenant ids remain independent from that organization, and governance gate labels remain independent from concrete evaluator names.
• If that endpoint is a Databricks App, CAN_USE for the Runway App service principal; Runway resolves a fresh upstream App OAuth token per Experiments request while the typed client also supplies the organization API key.

DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
RUNWAY_ARTIFACT_VOLUME=/Volumes/main/runway/artifacts
RUNWAY_EXPERIMENTS_URL=https://your-experiments-api
RUNWAY_EXPERIMENTS_ORGANIZATION_ID=org_enterprise
RUNWAY_EXPERIMENTS_ORGANIZATION_SLUG=enterprise-platform
RUNWAY_TRUSTED_WORKER_IDS=svc-runway-worker
RUNWAY_LAKEBASE_PROJECT=projects/my-runway-project
RUNWAY_LAKEBASE_SOURCE_BRANCH=production
RUNWAY_PREVIEW_BRANCH_TTL_SECONDS=86400

Credentials come from Databricks App resources, workload identity, or secret references — never from Runway action parameters or bundle variables. See production readiness for the full workspace checklist.

RUNWAY_SQL_WAREHOUSE_ID is intentionally absent from the default customer bundle. Set it only in a separately reviewed account-admin evidence-worker composition that injects createDatabricksGatewayEvidenceSource(), and set RUNWAY_GATEWAY_EVIDENCE_ACCOUNT_ADMIN_ACK=1 to acknowledge that privilege boundary. Production startup rejects the warehouse setting without that acknowledgement. Without the source, signed evidence reports the Gateway projection as not-configured. RUNWAY_REQUIRE_GATEWAY_EVIDENCE=1 is appropriate only in that worker; otherwise it would make every export fail while preserving the honest missing-source state.

Repeatable live certification

Runway includes a disposable live-workspace suite built on @fabricorg/databricks-testkit. It proves the real path rather than only checking that a bundle validates:

1. The Databricks test kit writes and removes a small Volume probe.
2. Runway provisions a disposable internal tenant through request, worker bootstrap, and completion actions, then installs a deterministic two-party certification policy.
3. Harness stages the immutable certification App into the named Volume.
4. Runway submits app_register, artifact_register, and deploy_request through Platform Host.
5. Runway creates a TTL-bound Lakebase preview branch. Harness materializes and verifies the digest during registration, then validates/deploys the preview bundle against that branch and activates the App.
6. Runway records a certification gate, collects distinct agent and human endorsements, pins the cycle policy, and finalizes through the trusted-worker action. The certification gate deliberately disables Experiments provenance; each client must separately certify its real dataset mapping.
7. Fabric Experiments requires the App to be running and its authenticated health route to pass.
8. Runway submits preview_destroy, removes the exact preview-owned App and Lakebase branch, and completes governed tenant retirement. The staged artifact directory is then removed.

The command is gated because it creates and deletes workspace resources:

RUNWAY_CERTIFY_LIVE=1 pnpm certify:databricks:live -- \
  --profile <databricks-cli-profile> \
  --catalog <scratch-catalog> \
  --lakebase-project <project-id> \
  --sql-warehouse <warehouse-id> \
  --gateway-model <system.ai.model-service>

--catalog creates a run-scoped schema and managed Volume, then deletes both. To reuse a pre-provisioned test Volume instead, pass --volume /Volumes/<catalog>/<schema>/<artifact-volume>; the caller must already have USE CATALOG, USE SCHEMA, READ VOLUME, and WRITE VOLUME. Use --lakebase-source-branch <branch-id> only when the disposable preview must fork from something other than production. The caller also needs CAN MANAGE on the named Lakebase project. The current certifier identity must be an account admin with CAN USE on the SQL warehouse. The suite sends one tagged model request, then polls the Gateway usage table for up to five minutes; override with --gateway-evidence-timeout-seconds when the workspace's system-table delivery is slower. A successful model response is not enough: if the correlated usage row is still absent at the deadline, the check fails and retains the correlation ID for later diagnosis.

Evidence is written to reports/databricks/runway-live.json and JUnit output to reports/databricks/runway-live.xml. Missing profile/Volume configuration fails before mutation. Missing Lakebase, warehouse, or Gateway model configuration also fails before mutation. The suite uses the real Databricks adapter and governed domain pipeline but an ephemeral in-process projection; the customer-installation drill additionally certifies Lakebase restart recovery and the real Fabric Experiments resolver. If governed cleanup cannot run after a partial failure, the suite performs a narrowly scoped emergency bundle destroy for the unique preview suffix and reports the certification as failed; it never sweeps unrelated Apps or Volume paths.