# Customer-owned GCP ClickHouse

Set up Premium customer-owned ClickHouse — also called BYOC ("bring your own cloud") — so tenant product data lives in your self-hosted ClickHouse database on Google Cloud.

With BYOC, InstantML still owns identity, sessions, billing, API-key metadata, route metadata, and artifact bytes in its hosted control plane.

## Know what goes into your ClickHouse

- Projects.
- Runs.
- Scalar metrics.
- Rank-aware metrics.
- Console logs.
- Attributes.
- Rich objects and table preview rows.
- Artifact metadata.
- Import records.
- Tenant operational records.

## Know what stays with InstantML

- Users and identities.
- Workspaces (organizations) and memberships.
- Browser sessions.
- API keys and service-account metadata.
- Billing and plan state.
- Tenant route metadata.
- Artifact byte storage in the private Cloudflare R2 artifact backend when InstantML stores uploaded bytes.

> **Note:** Usage storage guardrails for BYOC count only artifact bytes stored by InstantML in Cloudflare R2. They do not count customer-owned ClickHouse database bytes.

## Set up ClickHouse on GCP

1. Create or choose a read-write self-hosted ClickHouse deployment on Google Cloud near the InstantML data-plane region. A dedicated Compute Engine VM or managed instance group is the simplest first setup; use GKE only if your team already operates Kubernetes for stateful services.
2. Expose the ClickHouse HTTP interface over public HTTPS, usually through a TLS-terminating proxy or load balancer in front of the ClickHouse host.
3. Add every InstantML egress CIDR shown in onboarding to the GCP firewall, load-balancer allowlist, or reverse-proxy allowlist. This is the API/data-plane egress, not the browser IP.
4. Create one dedicated database for the workspace, such as `instantml_acme`.
5. Create one dedicated SQL user, such as `instantml_writer`.
6. Grant schema migration, insert, and read permissions for initial setup.
7. Paste the HTTPS endpoint origin, database, username, and password into InstantML onboarding.
8. Validate and save.
9. Optionally revoke DDL privileges after validation. Future schema migrations may require temporarily granting them again.

## Pick infrastructure settings

- Region: choose the same Google Cloud region as the InstantML data plane shown in onboarding or by your InstantML operator contact. Use the closest available region if your training jobs are elsewhere, because writes flow SDK -> InstantML API -> ClickHouse.
- Compute and disk: start with a dedicated ClickHouse host, an SSD persistent data disk, and enough free disk headroom for merges and backups. Keep data on a persistent disk separate from the boot disk.
- TLS: use a certificate from a public trusted CA and keep hostname verification valid for the endpoint you paste into InstantML. Self-signed certificates are only for local/operator tests.
- Backups and monitoring: enable disk snapshots or ClickHouse backups, and alert on disk usage, failed backups, CPU, memory, and ClickHouse query errors.
- Install reference: ClickHouse's current install guide is [clickhouse.com/docs/install](https://clickhouse.com/docs/install).

## Create the database and user

```sql
CREATE DATABASE IF NOT EXISTS instantml_acme;

CREATE USER IF NOT EXISTS instantml_writer
IDENTIFIED WITH sha256_password BY '<generated-password>';

GRANT SHOW, SELECT, INSERT, CREATE TABLE, CREATE VIEW, ALTER TABLE
ON instantml_acme.*
TO instantml_writer;

-- Optional after InstantML validates and saves the connection:
REVOKE CREATE TABLE, CREATE VIEW, ALTER TABLE
ON instantml_acme.*
FROM instantml_writer;
```

Use this endpoint format:

```text
https://clickhouse.acme.example.com:8443
```

Do not include username, password, path, query string, or fragment in the URL.

## Open the firewall to InstantML egress

```bash
gcloud compute firewall-rules create instantml-clickhouse-https \
  --network=<vpc-name> \
  --target-tags=<clickhouse-instance-tag> \
  --allow=tcp:8443 \
  --source-ranges=<instantml-egress-cidr-1>,<instantml-egress-cidr-2>
```

> **Warning:** Use the exact CIDRs from onboarding. Do not substitute your browser IP.

## Validate the connection

Validation runs from the data-plane service because that is the same service path that later serves SDK and dashboard product traffic. The workspace remains in a storage setup state until validation and save succeed.

Product writes and SDK-key creation are blocked while storage is unconfigured. The dashboard keeps users in onboarding and shows the required egress CIDRs and connection form.

## Plan around current limitations

- BYOC is Premium-only.
- The workspace must be empty before switching to BYOC.
- Hosted mode requires a public HTTPS ClickHouse endpoint.
- The database must already exist.
- Initial validation applies the InstantML schema and needs DDL privileges. Future schema upgrades may require temporarily re-granting DDL and revalidating.
- Runtime password storage should use the configured secret backend. Local user-data password storage is for disposable smoke tests only.

## Next steps

- [Auth, billing, and storage](/docs/guides/auth-billing-storage.md)
- [Google Cloud ClickHouse architecture](/docs/architecture/google-clickhouse.md)
- [Storage model](/docs/architecture/storage-model.md)
- [Pricing, limits, and billing](/docs/guides/pricing-limits-billing.md)
- [Onboarding, team, and billing](/docs/dashboard/onboarding-team-billing.md)

## Agent navigation

- [Docs index](/llms.txt)
- [Full docs bundle](/llms-full.txt)

### Get Started

- [Overview](/docs/index.md)
- [Quickstart](/docs/quickstart.md)
- [Core Concepts](/docs/concepts/core-concepts.md)
- [Pricing](/docs/pricing.md)
- [Benchmarks](/docs/benchmarks.md)
- [Examples](/docs/guides/examples.md)
- [Troubleshooting](/docs/troubleshooting.md)

### SDK

- [Installation Auth](/docs/sdk/installation-auth.md)
- [Logging](/docs/sdk/logging.md)
- [Tracing](/docs/sdk/tracing.md)
- [Metrics Steps](/docs/sdk/metrics-steps.md)
- [Config Tags Notes](/docs/sdk/config-tags-notes.md)
- [Artifacts Checkpoints](/docs/sdk/artifacts-checkpoints.md)
- [Rich Objects](/docs/sdk/rich-objects.md)
- [Distributed Training](/docs/sdk/distributed-training.md)
- [Console System Integrations](/docs/sdk/console-system-integrations.md)
- [Reliability](/docs/sdk/reliability.md)
- [Cli Login](/docs/sdk/cli-login.md)
- [Querying Data](/docs/sdk/querying-data.md)
- [Agent Mcp](/docs/sdk/agent-mcp.md)
- [Examples Patterns](/docs/sdk/examples-patterns.md)

### Integrations

- [Overview](/docs/integrations/overview.md)
- [Pytorch Lightning](/docs/integrations/pytorch-lightning.md)
- [Huggingface Transformers](/docs/integrations/huggingface-transformers.md)
- [Keras](/docs/integrations/keras.md)
- [Wandb](/docs/integrations/wandb.md)

### Dashboard

- [Tour](/docs/dashboard/tour.md)
- [Organizations Workspaces](/docs/dashboard/organizations-workspaces.md)
- [Runs Workspace](/docs/dashboard/runs-workspace.md)
- [Metrics Charts](/docs/dashboard/metrics-charts.md)
- [Run Detail](/docs/dashboard/run-detail.md)
- [Traces](/docs/dashboard/traces.md)
- [Compare Runs](/docs/dashboard/compare-runs.md)
- [Research Dashboards](/docs/dashboard/research-dashboards.md)
- [Artifacts Files](/docs/dashboard/artifacts-files.md)
- [Run Health](/docs/dashboard/alerts.md)
- [Datasets](/docs/dashboard/datasets.md)
- [Checkpoints](/docs/dashboard/checkpoints.md)
- [Reports](/docs/dashboard/reports.md)
- [Settings Api Keys](/docs/dashboard/settings-api-keys.md)
- [Api Tab](/docs/dashboard/api-tab.md)
- [Onboarding Team Billing](/docs/dashboard/onboarding-team-billing.md)

### Data

- [Imports](/docs/guides/imports.md)
- [W&B alternative](/docs/guides/wandb-alternative.md)
- [InstantML vs MLflow](/docs/guides/instantml-vs-mlflow.md)
- [W&B import guide](/docs/guides/wandb-import-guide.md)
- [W&B and Neptune imports](/docs/guides/wandb-neptune-imports.md)
- [Export Usage Limits](/docs/guides/export-usage-limits.md)
- [Pricing Limits Billing](/docs/guides/pricing-limits-billing.md)
- [Auth Billing Storage](/docs/guides/auth-billing-storage.md)
- [Customer Owned Clickhouse](/docs/guides/customer-owned-clickhouse.md) (current page)
- [Observability](/docs/guides/observability.md)

### API

**Practical API guides**

- [Authentication](/docs/api/authentication.md)
- [Errors And Limits](/docs/api/errors-and-limits.md)
- [Health Observability](/docs/api/health-observability.md)
- [Projects Runs](/docs/api/projects-runs.md)
- [Metrics Series](/docs/api/metrics-series.md)
- [Attributes Objects](/docs/api/attributes-objects.md)
- [Artifacts](/docs/api/artifacts.md)
- [Reports](/docs/api/reports.md)
- [Iframe Embeds](/docs/api/iframe-embeds.md)
- [Import Export Usage](/docs/api/import-export-usage.md)
- [Dashboard Control State](/docs/api/dashboard-control-state.md)
**Architecture**

- [System Overview](/docs/architecture/system-overview.md)
- [Service Planes](/docs/architecture/service-planes.md)
- [Storage Model](/docs/architecture/storage-model.md)
- [Google Clickhouse](/docs/architecture/google-clickhouse.md)
- [Auth Tenancy](/docs/architecture/auth-tenancy.md)
- [Schema Reference](/docs/architecture/schema-reference.md)

### API Reference

- [API Reference](/docs/api-reference.md)