BIGQUERY → STACKIT · ARCHITECTURE
BigQuery to STACKIT: designing the data platform
BigQuery is not moved, it is replaced by a design. StackToIT classifies a BigQuery dataset as REDESIGN with LOW confidence and writes no Terraform for it, because a serverless warehouse, its SQL dialect and its billing model have no single STACKIT resource to become. This page sets out the options, what the engine does with the pieces around the warehouse, and how to choose.
A dataset with tables, assessed as one unit
A small analytics estate written for this page and assessed by StackToIT on 4 November 2026: a dataset, a partitioned table, a view, a dataset grant, a bucket and a topic. Left: the dataset and the orders table as the input declares them. Right: an excerpt of the unit in assessment.json, with the values exactly as the run wrote them; the target's service text is left out.
resource "google_bigquery_dataset" "sales" {
dataset_id = "sales"
location = "EU"
default_table_expiration_ms = 31536000000
}
resource "google_bigquery_table" "orders" {
dataset_id = google_bigquery_dataset.sales.dataset_id
table_id = "orders"
deletion_protection = true
time_partitioning {
type = "DAY"
field = "order_date"
}
clustering = ["customer_id"]
schema = jsonencode([
{ name = "order_id", type = "STRING", mode = "REQUIRED" },
{ name = "customer_id", type = "STRING", mode = "REQUIRED" },
{ name = "order_date", type = "DATE", mode = "REQUIRED" },
{ name = "total", type = "NUMERIC", mode = "NULLABLE" }
])
}{
"id": "google_bigquery_dataset.sales",
"kind": "data_warehouse",
"facets": [
"google_bigquery_dataset_iam_member.analysts",
"google_bigquery_table.daily_revenue",
"google_bigquery_table.orders"
],
"result": {
"class": "REDESIGN",
"confidence": "LOW",
"target": {
"resources": [
"stackit_dremio_instance"
]
},
"reason": "BigQuery is a serverless warehouse with its own SQL dialect, storage and billing model. On STACKIT, data lands as files (e.g. Iceberg) in Object Storage queried with Dremio, or in a self-managed warehouse; queries, scheduled queries and BI connections are rewritten."
}
}Tables, view and grant fold into the dataset
The two tables and the dataset grant are listed as facets of the dataset unit, so six resources become three units. What the unit stands for is the whole warehouse slice, not a single object.
Nothing is drafted for the warehouse
The class is REDESIGN and the confidence LOW. The target resource named is a Dremio instance, which the provider marks as part of an opt-in feature that may change significantly; the run lists it as a candidate and writes no block. STACKIT’s Dremio page adds, as of November 2026, that a soft limit on active instances per project applies and engine sizes are capped in the current release phase; more capacity goes through STACKIT support.
Twelve critical settings stay with you
The architect report lists them by kind: identity access (the dataset grant), expiration of data, deletion protection on both tables, the location, three names, the table schema and the partition layout. Each is a decision about the replacement, not a value to copy.
The pieces around the warehouse are drafted
From the same run. A landing bucket and an event topic are ordinary migrations. Left: the input. Then the blocks the engine wrote, one pane per file, without the review comments above them.
resource "google_storage_bucket" "landing" {
name = "shop-analytics-landing"
location = "EUROPE-WEST3"
uniform_bucket_level_access = true
}
resource "google_pubsub_topic" "events" {
name = "order-events"
}resource "stackit_objectstorage_bucket" "landing" {
project_id = var.project_id
name = "shop-analytics-landing"
}resource "stackit_rabbitmq_instance" "messaging" {
project_id = var.project_id
name = "messaging"
plan_name = var.rabbitmq_plan_name
version = var.rabbitmq_version
parameters = {
sgw_acl = join(",", var.rabbitmq_allowed_cidrs)
}
}The bucket is the foundation
A Cloud Storage bucket becomes a STACKIT Object Storage bucket (RECONFIGURE). Whichever warehouse design you choose, the files land here, so this is the first thing to build.
Pub/Sub topic to RabbitMQ
A topic becomes an exchange on one shared RabbitMQ instance (REPLATFORM). The draft creates the instance only; exchanges, queues and the move from the Pub/Sub client to AMQP are yours.
Data movement is an open decision
migration/data-movement.md lists the dataset and the bucket as stateful units with the mechanism still unknown, and notes that the data size and change rate are not in the source.
The options and their trade-offs
| Option | Fits when | Trade-off |
|---|---|---|
| Lakehouse: files in Object Storage, queried with STACKIT Dremio | Many SQL users and BI tools; data can live as open-format files | Queries and views are rewritten to Dremio SQL; the service is opt-in in the provider and available there in eu01 |
| Self-managed warehouse on servers or SKE | You need a specific engine or full control of its tuning | You operate, patch, scale and back it up |
| PostgreSQL Flex for small relational marts | The data is modest and the queries are relational | A single managed instance; it does not scale out like a serverless warehouse |
| Keep this workload on BigQuery for now | The data may stay outside STACKIT for a period | Two platforms to run, and exports remain a future task |
The first two rows are the targets the engine names for BigQuery. The last two are options an architect can weigh; the run does not propose them.
How to decide
Answer these in order; the answers usually pick the row.
Who queries it, and with what
List the analysts, reports and scheduled jobs. BigQuery's SQL dialect, nested and repeated fields and BigQuery ML models are all things you rewrite or replace (general practice, not run output); count them, because they are the real cost.
How big, and how fast does it change
The run says plainly that neither is in the source. Without them the data movement cannot be planned, and an instance cannot be sized.
What must expire or stay
Your dataset sets a default table expiration of one year, which the run flags. On a file-based design, expiration becomes a job or a lifecycle rule you write on purpose.
Who may read what
Dremio documents row and column level access control and sign-in through an OIDC identity provider. Translate each BigQuery dataset grant into one of its rules rather than copying roles.
Moving the data, once you have chosen
General practice, not engine output. The run only records that a mechanism has to be chosen.
Export from BigQuery
A table export writes to Cloud Storage only, as CSV, newline-delimited JSON, Avro or Parquet. The dataset and the bucket must be in the same location, and a single file holds up to 1 GB of table data, so use a wildcard in the file name.
Copy to Object Storage
Copy the exported files to the STACKIT bucket with an S3-compatible tool such as rclone. The bucket speaks the S3 API over HTTPS.
Register the tables
Dremio has native connectors for S3-compatible storage and supports Iceberg and Delta Lake tables. Decide the partition layout deliberately, as the run lists the partition column as open.
Rewrite views and schedules
The daily revenue view is written in BigQuery SQL and is rewritten by hand. STACKIT Workflows, built on Apache Airflow, can run recurring jobs as Python DAGs; its web-based DAG editor is not yet available.
Streaming inputs
Where events used to flow into BigQuery, STACKIT Intake ingests over the Kafka protocol into Iceberg tables in a Dremio catalog. Check that it fits your producers before you rely on it.
Questions data teams ask
Why does StackToIT write no Terraform for BigQuery?
Because the right target depends on a design choice: files queried with Dremio, or a warehouse you operate. The run names the candidates, lists the open settings and leaves the choice to you.
Is there a STACKIT service equivalent to BigQuery?
Not as a single service. The closest managed path in STACKIT's documentation is a lakehouse: open-format files in Object Storage queried with Dremio. It is a different model, so queries and tooling change.
Do BigQuery partitions and clustering carry over?
No setting does by itself. The run lists the partition column, the partition type and the table schema as open items. It reads the clustering on customer_id but raises no item for it, so put clustering on your own list; you decide the layout of the files or tables on the target.
Can we move the data first and design later?
Yes. Exporting to Parquet and copying to an Object Storage bucket does not commit you to an engine. It is a safe first step, because each STACKIT option above reads files or can load them.
Sources
Your next move: STACKIT.
Tell us about your estate. We show you what StackToIT delivers for it: target architecture, Terraform artifacts and a structured migration plan.