InstantMLDocs

Storage model

Control records, tenant records, metric tables, artifact bytes, and usage counters.

Open .md

InstantML separates low-volume operational state from high-volume metric history, so run tables and charts stay fast without turning every dashboard view into a full history scan.

Logical planes

text
Control plane (account state):
  Users, identities, workspaces (organizations), memberships, sessions,
  API keys, service accounts, billing state, invitations, tenant routes,
  dashboard preferences, and saved workspace views.

Data plane (tenant product state):
  Projects, runs, attributes, artifacts, reports, imports, usage snapshots,
  console logs, scalar metrics, rank metrics, and rich object rows.

In hosted mode, control-plane records live in a Postgres database and tenant product rows live in the workspace's routed ClickHouse database. In local single-binary mode, both planes live in one ClickHouse database.

Append-only operational records

Durable low-volume tenant records are append-only. A newer record for the same entity replaces the previous one in the serving index when records are replayed; older record versions remain in storage.

Tenant operational records include:

  • Projects.
  • Runs.
  • Attributes.
  • Artifacts (raw rows and versioned collections).
  • Reports and report shares.
  • Imports.
  • Idempotency records.
  • Usage snapshots.
  • Table preview rows.
  • Local-mode user, workspace, session, and API-key records.

Hosted control-plane records (Postgres) include:

  • Users and identities.
  • Workspaces (organizations) and memberships.
  • Sessions.
  • API keys and service accounts.
  • Billing accounts, subscriptions, and events.
  • Invitations and email deliveries.
  • Tenant routes and data-cell placement.
  • Dashboard preferences and saved workspace views.

Metric tables

Scalar metrics are stored as raw metric_points rows plus a maintained summary table. Rank-aware metrics are stored separately so distributed training dashboards can compute per-rank reducers, coverage, heatmaps, and outliers.

Metric row shape:

json
{
  "org_id": "uuid",
  "run_id": "uuid",
  "key": "eval/accuracy",
  "step": 10,
  "value": 0.91,
  "logged_at": "2026-05-16T00:00:00Z",
  "created_at": "2026-05-16T00:00:01Z"
}

Summary reads derive count, min, max, latest, best, mean, and variance from aggregate state that is maintained on insert. Raw series reads are always bounded by run ID, key, step range, time range, and limit.

Console logs

Console logs are stored as bounded line records per run and stream, and the dashboard fetches them only when you open a run's Logs section.

Streams:

  • stdout
  • stderr

Artifacts

Artifact metadata lives in ClickHouse. Artifact bytes live in a separate byte backend:

BackendUse
Local diskSingle-binary and self-hosted deployments store bytes under the configured artifact root.
Private object storage (per-workspace buckets)Hosted artifact uploads.
External URI referenceMetadata-only pointer to your own storage; no bytes are uploaded.

Stored bytes use opaque instantml://artifacts/<id> URIs in public artifact rows. Raw bucket names, storage keys, and signed URLs are never returned in public metadata; downloads stream through the API.

Usage counters

Usage combines retained-resource counts with current-period ingest counters:

  • Metric points and API requests reset by UTC calendar month.
  • Projects, runs, seats, API keys, artifacts, metric series, and storage are retained-resource counts.
  • Hosted storage can include exact artifact bytes and exact tenant warehouse table bytes when the workspace has a dedicated database.
  • Customer-owned ClickHouse workspaces count only InstantML-managed artifact bytes, not tables in your own deployment.

Next steps