# Storage model InstantML separates low-volume operational state from high-volume metric history, so run tables and charts stay fast without turning every dashboard view into a full history scan. ## Logical planes ```text Control plane (account state): Users, identities, workspaces (organizations), memberships, sessions, API keys, service accounts, billing state, invitations, tenant routes, dashboard preferences, and saved workspace views. Data plane (tenant product state): Projects, runs, attributes, artifacts, reports, imports, usage snapshots, console logs, scalar metrics, rank metrics, and rich object rows. ``` In hosted mode, control-plane records live in a Postgres database and tenant product rows live in the workspace's routed ClickHouse database. In local single-binary mode, both planes live in one ClickHouse database. ## Append-only operational records Durable low-volume tenant records are append-only. A newer record for the same entity replaces the previous one in the serving index when records are replayed; older record versions remain in storage. Tenant operational records include: - Projects. - Runs. - Attributes. - Artifacts (raw rows and versioned collections). - Reports and report shares. - Imports. - Idempotency records. - Usage snapshots. - Table preview rows. - Local-mode user, workspace, session, and API-key records. Hosted control-plane records (Postgres) include: - Users and identities. - Workspaces (organizations) and memberships. - Sessions. - API keys and service accounts. - Billing accounts, subscriptions, and events. - Invitations and email deliveries. - Tenant routes and data-cell placement. - Dashboard preferences and saved workspace views. ## Metric tables Scalar metrics are stored as raw `metric_points` rows plus a maintained summary table. Rank-aware metrics are stored separately so distributed training dashboards can compute per-rank reducers, coverage, heatmaps, and outliers. Metric row shape: ```json { "org_id": "uuid", "run_id": "uuid", "key": "eval/accuracy", "step": 10, "value": 0.91, "logged_at": "2026-05-16T00:00:00Z", "created_at": "2026-05-16T00:00:01Z" } ``` Summary reads derive count, min, max, latest, best, mean, and variance from aggregate state that is maintained on insert. Raw series reads are always bounded by run ID, key, step range, time range, and limit. ## Console logs Console logs are stored as bounded line records per run and stream, and the dashboard fetches them only when you open a run's Logs section. Streams: - `stdout` - `stderr` ## Artifacts Artifact metadata lives in ClickHouse. Artifact bytes live in a separate byte backend: | Backend | Use | | --- | --- | | Local disk | Single-binary and self-hosted deployments store bytes under the configured artifact root. | | Private object storage (per-workspace buckets) | Hosted artifact uploads. | | External URI reference | Metadata-only pointer to your own storage; no bytes are uploaded. | Stored bytes use opaque `instantml://artifacts/` URIs in public artifact rows. Raw bucket names, storage keys, and signed URLs are never returned in public metadata; downloads stream through the API. ## Usage counters Usage combines retained-resource counts with current-period ingest counters: - Metric points and API requests reset by UTC calendar month. - Projects, runs, seats, API keys, artifacts, metric series, and storage are retained-resource counts. - Hosted storage can include exact artifact bytes and exact tenant warehouse table bytes when the workspace has a dedicated database. - Customer-owned ClickHouse workspaces count only InstantML-managed artifact bytes, not tables in your own deployment. > **Warning:** Writes fail with a plan-limit or payment-required error when a new write would cross an enforced limit. See [Pricing, limits, and billing](/docs/guides/pricing-limits-billing.md). ## Next steps - [Schema reference](/docs/architecture/schema-reference.md) - [Google Cloud ClickHouse](/docs/architecture/google-clickhouse.md) - [Artifacts API](/docs/api/artifacts.md) - [Import, export, and usage API](/docs/api/import-export-usage.md) - [Export, usage, and limits guide](/docs/guides/export-usage-limits.md) ## Agent navigation - [Docs index](/llms.txt) - [Full docs bundle](/llms-full.txt) ### Get Started - [Overview](/docs/index.md) - [Quickstart](/docs/quickstart.md) - [Core Concepts](/docs/concepts/core-concepts.md) - [Pricing](/docs/pricing.md) - [Benchmarks](/docs/benchmarks.md) - [Examples](/docs/guides/examples.md) - [Troubleshooting](/docs/troubleshooting.md) ### SDK - [Installation Auth](/docs/sdk/installation-auth.md) - [Logging](/docs/sdk/logging.md) - [Tracing](/docs/sdk/tracing.md) - [Metrics Steps](/docs/sdk/metrics-steps.md) - [Config Tags Notes](/docs/sdk/config-tags-notes.md) - [Artifacts Checkpoints](/docs/sdk/artifacts-checkpoints.md) - [Rich Objects](/docs/sdk/rich-objects.md) - [Distributed Training](/docs/sdk/distributed-training.md) - [Console System Integrations](/docs/sdk/console-system-integrations.md) - [Reliability](/docs/sdk/reliability.md) - [Cli Login](/docs/sdk/cli-login.md) - [Querying Data](/docs/sdk/querying-data.md) - [Agent Mcp](/docs/sdk/agent-mcp.md) - [Examples Patterns](/docs/sdk/examples-patterns.md) ### Integrations - [Overview](/docs/integrations/overview.md) - [Pytorch Lightning](/docs/integrations/pytorch-lightning.md) - [Huggingface Transformers](/docs/integrations/huggingface-transformers.md) - [Keras](/docs/integrations/keras.md) - [Wandb](/docs/integrations/wandb.md) ### Dashboard - [Tour](/docs/dashboard/tour.md) - [Organizations Workspaces](/docs/dashboard/organizations-workspaces.md) - [Runs Workspace](/docs/dashboard/runs-workspace.md) - [Metrics Charts](/docs/dashboard/metrics-charts.md) - [Run Detail](/docs/dashboard/run-detail.md) - [Traces](/docs/dashboard/traces.md) - [Compare Runs](/docs/dashboard/compare-runs.md) - [Research Dashboards](/docs/dashboard/research-dashboards.md) - [Artifacts Files](/docs/dashboard/artifacts-files.md) - [Run Health](/docs/dashboard/alerts.md) - [Datasets](/docs/dashboard/datasets.md) - [Checkpoints](/docs/dashboard/checkpoints.md) - [Reports](/docs/dashboard/reports.md) - [Settings Api Keys](/docs/dashboard/settings-api-keys.md) - [Api Tab](/docs/dashboard/api-tab.md) - [Onboarding Team Billing](/docs/dashboard/onboarding-team-billing.md) ### Data - [Imports](/docs/guides/imports.md) - [W&B alternative](/docs/guides/wandb-alternative.md) - [InstantML vs MLflow](/docs/guides/instantml-vs-mlflow.md) - [W&B import guide](/docs/guides/wandb-import-guide.md) - [W&B and Neptune imports](/docs/guides/wandb-neptune-imports.md) - [Export Usage Limits](/docs/guides/export-usage-limits.md) - [Pricing Limits Billing](/docs/guides/pricing-limits-billing.md) - [Auth Billing Storage](/docs/guides/auth-billing-storage.md) - [Customer Owned Clickhouse](/docs/guides/customer-owned-clickhouse.md) - [Observability](/docs/guides/observability.md) ### API **Practical API guides** - [Authentication](/docs/api/authentication.md) - [Errors And Limits](/docs/api/errors-and-limits.md) - [Health Observability](/docs/api/health-observability.md) - [Projects Runs](/docs/api/projects-runs.md) - [Metrics Series](/docs/api/metrics-series.md) - [Attributes Objects](/docs/api/attributes-objects.md) - [Artifacts](/docs/api/artifacts.md) - [Reports](/docs/api/reports.md) - [Iframe Embeds](/docs/api/iframe-embeds.md) - [Import Export Usage](/docs/api/import-export-usage.md) - [Dashboard Control State](/docs/api/dashboard-control-state.md) **Architecture** - [System Overview](/docs/architecture/system-overview.md) - [Service Planes](/docs/architecture/service-planes.md) - [Storage Model](/docs/architecture/storage-model.md) (current page) - [Google Clickhouse](/docs/architecture/google-clickhouse.md) - [Auth Tenancy](/docs/architecture/auth-tenancy.md) - [Schema Reference](/docs/architecture/schema-reference.md) ### API Reference - [API Reference](/docs/api-reference.md)