Migration roadmap — gcp-candidates/terraform-dynamic-python-webapp/infra
FOR REVIEW — generated by StackToIT Assess. Architect review required before any decision.
Run 6f8f72b2-1c34-4cfc-8483-344c41245364 · source digest sha256:19bfed642d5789ccada81b91d5c36e33175dcafd66dc46b1644ffa105d020000
Partially executable migration package · For review only · Not approved.
The steps of the migration in the order they can be taken: the execution plan (execution-plan.json) laid out in ten waves. A step depends on the steps in its row and on no other; a wave is a starting point for the schedule, not a schedule.
The plan holds 78 steps: 12 automated (10 free of decisions and manual prerequisites, 0 of them can run as written) and 64 manual (a person can start 29 now), 10 after a manual prerequisite, 3 after an automated step, 24 waiting for a decision, 0 not executable whatever is decided and 2 refused.
Steps: partially executable. Execution completeness: COMPLETE (each of the 64 manual steps states what it is, why, who carries it out and when it is done, and the input it needs where it depends on one). 10 of the 12 automated steps (OpenTofu or a script) are free of decisions and manual prerequisites (2 require a design decision) · 0 can run as written · 10 of those 10 wait for a value (project_id, containers_machine_type, containers_nodes_max, containers_nodes_min and 7 more); a person can start 29 of the 64 manual steps now (10 wait for a manual prerequisite, 3 wait for an automated step, 22 require a design decision); 2 steps of the plan are permanently unsupported (refused). Migration-ready: NO — 72 critical settings of 16 source resources have manual or refused steps open (65 manual, 2 refused, 5 with no step); 26 steps of the plan cannot be carried out now (24 wait only for a decision, 2 are refused).
This roadmap holds the steps of the execution plan only. A block of the Terraform draft that no plan lists is not a step here; the draft in terraform/ is the whole of what is written, and technical-sequencing.md orders every source unit by its dependencies.
Wave 0 — Decisions
Questions nobody has answered yet. A step that waits for one cannot be taken before it is answered.
5 decisions (9 questions) must be answered. These are the decisions that steps of the plan wait for; the decisions to make of the register (decisions.json) also count the manual actions and plan statements that need a person, so that number can be larger. A decision of several questions has a row for each question, named by the decision's id and what the question is about.
| Decision | Question | Why it matters | Google Cloud resource | Needed | Blocks |
|---|---|---|---|---|---|
DEC-GEN-CONVERT-API-OPT-IN/google_pubsub_topic.faux | Allow calls of the beta pubsub API v1beta for this migration (the API may change or vanish)? Accept: the pubsub API v1beta path is carried out for google_pubsub_topic.faux: its calls become executable. the RabbitMQ instance the draft writes stays in the draft this run wrote, so its step is held (REGENERATE_REQUIRED) until a draft is generated without it (answer target.messaging.path with pubsub in the inputs file and run again); an instance shared with a queue or topic that has no call of the pubsub API v1beta path stays taken. Refuse: the RabbitMQ instance the draft writes is carried out and the calls and steps of the pubsub API v1beta path are not taken. | the pubsub API v1beta is beta: the execution plan executes no call of an API that is not generally available without an explicit opt-in, so the calls of it and the steps of its path for google_pubsub_topic.faux wait for this decision; the draft's own blocks do not | — | before the pilot; answer with one of its options: "Allow the calls of the API for this migration" | 1 step of the plan |
DEC-GEN-CONVERT-API-OPT-IN/google_secret_manager_secret.django_admin_password | Allow calls of the Secrets Manager Vault-compatible API, whose maturity is not established, for this migration (the API may change or vanish)? Accept: the calls of the Secrets Manager Vault-compatible API (WriteSecret) are carried out for google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings. Refuse: no call is made, a refusal selects no other path (the draft holds no other carrier of what the calls set), and what they set stays open for google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings. | the Secrets Manager Vault-compatible API is of a maturity that is not established (its documentation states no release status and its product documentation does not establish it as generally available): the execution plan executes no call of an API that is not generally available without an explicit opt-in, so the calls of it and the steps of its path for google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings wait for this decision; the draft's own blocks do not | — | before the pilot; answer with one of its options: "Allow the calls of the API for this migration" | 4 steps of the plan |
DEC-GEN-CONVERT-DATA-MOVEMENT | Choose how the data of google_sql_database_instance.postgres moves from google_sql_database_instance to PostgreSQL Flex (stackit_postgresflex_instance.postgres): a dump restored inside the write freeze, the engine's logical replication, or a tool of your own | the plan defines both methods of the engine (a dump restored inside the write freeze, and a logical replication that follows the changes until the freeze) and keeps them open: which one applies depends on the size of the data, the write rate and the downtime the business accepts, and the source states none of them; the steps of the methods wait for the answer | google_sql_database_instance.postgres | before the pilot; answer with one of its options: "Export the data from the source and import it with the clients of the target engine" or "Replicate the data with the replication of the source engine until the cutover" or "Copy the objects of the bucket with rclone, from the source bucket to the STACKIT bucket" or "Copy the objects of the bucket with the AWS CLI, through a staging directory" or "Move the data with a migration tool or a replication of the owner's own" | 13 steps of the plan |
DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_cloudbuild_trigger.activategcb | Decide whether the proposed STACKIT alternative replaces google_cloudbuild_trigger.activategcb: read the module proposals/proposal_activategcb, accept the difference and set its values, or design it another way | the target has no equal of this Cloud Build trigger: starts a build on a source event, a Pub/Sub message, a webhook or by hand, with its steps, substitutions, artifacts, secrets and optional approval: Nothing is converted. The same delivery is a workflow of STACKIT Git Pipelines (written for GitHub Actions; STACKIT-managed or custom runners; secrets from Secrets Manager or the repository; a manual-approval action) or the customer's own CI/CD. Trigger conditions, steps, secrets (created again, never copied), the service account (replaced by the credentials the workflow uses against STACKIT), the log destination and the approval are written again by a person. To decide: Which pipelines are still needed, where they run (STACKIT Git Pipelines or the customer's own CI/CD), how each trigger, step, secret and approval is written again, and who owns the workflow files. | google_cloudbuild_trigger.activategcb | before the pilot; answer with one of its options: "Use the proposed module, with the stated difference" or "Design it another way and leave the module off" | 1 step of the plan |
DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_cloudbuild_trigger.init | Decide whether the proposed STACKIT alternative replaces google_cloudbuild_trigger.init: read the module proposals/proposal_init, accept the difference and set its values, or design it another way | the target has no equal of this Cloud Build trigger: starts a build on a source event, a Pub/Sub message, a webhook or by hand, with its steps, substitutions, artifacts, secrets and optional approval: Nothing is converted. The same delivery is a workflow of STACKIT Git Pipelines (written for GitHub Actions; STACKIT-managed or custom runners; secrets from Secrets Manager or the repository; a manual-approval action) or the customer's own CI/CD. Trigger conditions, steps, secrets (created again, never copied), the service account (replaced by the credentials the workflow uses against STACKIT), the log destination and the approval are written again by a person. To decide: Which pipelines are still needed, where they run (STACKIT Git Pipelines or the customer's own CI/CD), how each trigger, step, secret and approval is written again, and who owns the workflow files. | google_cloudbuild_trigger.init | before the pilot; answer with one of its options: "Use the proposed module, with the stated difference" or "Design it another way and leave the module off" | 1 step of the plan |
DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_cloudbuild_trigger.placeholder | Decide whether the proposed STACKIT alternative replaces google_cloudbuild_trigger.placeholder: read the module proposals/proposal_placeholder, accept the difference and set its values, or design it another way | the target has no equal of this Cloud Build trigger: starts a build on a source event, a Pub/Sub message, a webhook or by hand, with its steps, substitutions, artifacts, secrets and optional approval: Nothing is converted. The same delivery is a workflow of STACKIT Git Pipelines (written for GitHub Actions; STACKIT-managed or custom runners; secrets from Secrets Manager or the repository; a manual-approval action) or the customer's own CI/CD. Trigger conditions, steps, secrets (created again, never copied), the service account (replaced by the credentials the workflow uses against STACKIT), the log destination and the approval are written again by a person. To decide: Which pipelines are still needed, where they run (STACKIT Git Pipelines or the customer's own CI/CD), how each trigger, step, secret and approval is written again, and who owns the workflow files. | google_cloudbuild_trigger.placeholder | before the pilot; answer with one of its options: "Use the proposed module, with the stated difference" or "Design it another way and leave the module off" | 1 step of the plan |
DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_firebase_hosting_site.client | Decide whether the proposed STACKIT alternative replaces google_firebase_hosting_site.client: read the module proposals/proposal_client, accept the difference and set its values, or design it another way | the target has no equal of this static web hosting site: Static files are published into an Object Storage bucket by a pipeline, and a CDN distribution (a beta resource) with the bucket as origin serves them on a custom domain; the redirects of the site become redirect rules of the distribution. Custom response headers, rewrites to Cloud Run services or Cloud Functions, preview channels and the release history have no counterpart and are restated in the pipeline or in the application. To decide: Which bucket is the origin, which domain the distribution serves, and how rewrites and previews are replaced. The CDN resources are beta. | google_firebase_hosting_site.client | before the pilot; answer with one of its options: "Use the proposed module, with the stated difference" or "Design it another way and leave the module off" | 1 step of the plan |
DEC-GEN-CONVERT-SOURCE-PROJECT-MAPPING | State where the units of each source project of the estate go | The source places its units in the project example-project, and in a project the scan cannot read for some units, and the source project is carried nowhere. A person maps the source projects by answering target.project.source_project for each: var.project_id, or the project of the unit's network (network-project). The answer records that mapping and changes nothing in the draft, which keeps its own project resolver. | — | before the pilot; answer with one of its options: "The units of the source project go to the project the variable project_id names" or "The units of the source project go to the project of their network" | 1 step of the plan |
DEC-GEN-CONVERT-SOURCE-REGION-MAPPING | Map each source location of the estate to a STACKIT region | The source places its units in a location the scan cannot read (the value comes from an input a plan can change, or the source does not state it). No mapping between Google and STACKIT regions is evidenced, so the region the draft builds in is not shown to hold the residency of a source location: a person states it once for the estate, by answering target.region.source_location for each location. | — | before the pilot; answer with one of its options: "Map every source location of the estate to the one STACKIT region the draft builds in" or "Choose the STACKIT region of each source location" | 1 step of the plan |
Wave 1 — Landing zone
The project and the preconditions a person provides before the first apply.
The manual steps of this wave in 1 work package (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-01 | Set up the identities and access (prepare) | Identity Architect | 4 | 4 | a person can start it now |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start |
|---|---|---|---|---|---|
MAN/identity/facts:google_cloud_run_v2_job.migrate:google_cloud_run_v2_job.migrate.template.0.template.0.service_account-8dc3e1 | Decide how google_cloud_run_v2_job.migrate gets its application identity on STACKIT (a service account of its own, workload federation): the draft does not write it | google_cloud_run_v2_job.migrate | Manual | — | Can start now |
MAN/identity/facts:google_cloud_run_v2_service.server:google_cloud_run_v2_service.server.template.0.service_account-5ccce9 | Decide how google_cloud_run_v2_service.server gets its application identity on STACKIT (a service account of its own, workload federation): the draft does not write it | google_cloud_run_v2_service.server | Manual | — | Can start now |
MAN/identity/held:principal:google_service_account.automation | Decide who may use the STACKIT service account written for google_service_account.automation, what it may do and which workloads run as it: the draft attaches it to nothing | google_service_account.automation | Manual | — | Can start now |
MAN/identity/held:principal:google_service_account.server | Decide who may use the STACKIT service account written for google_service_account.server, what it may do and which workloads run as it: the draft attaches it to nothing | google_service_account.server | Manual | — | Can start now |
Wave 2 — Network and security
Networks, routing and security groups.
The manual steps of this wave in 4 work packages (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-02 | Decide and set the critical settings (security) | Identity Architect | 6 | 6 | a person can start it now |
WP-03 | Decide and set the critical settings (security) | Security Architect | 2 | 2 | a person can start it now |
WP-04 | Secrets migration user retire (security) | Security Architect | 1 | 2 | waits for decisions |
WP-05 | Secrets read back (security) | Security Architect | 1 | 2 | waits for decisions |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start |
|---|---|---|---|---|---|
MAN/semantic/google_secret_manager_secret.django_admin_password/open-settings-security-architect | Review the 2 open critical settings of google_secret_manager_secret.django_admin_password (google_secret_manager_secret) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (secret-value, tcr-secret-id) | google_secret_manager_secret.django_admin_password | Manual | — | Can start now |
MAN/semantic/google_secret_manager_secret.django_admin_password/secret-settings | Decide what the STACKIT target of google_secret_manager_secret.django_admin_password (google_secret_manager_secret) does for its critical setting secret-settings, which the draft does not carry, and set it up by hand, or record that production does not need it | google_secret_manager_secret.django_admin_password | Manual | — | Can start now |
MAN/semantic/google_secret_manager_secret.django_settings/open-settings-security-architect | Review the 2 open critical settings of google_secret_manager_secret.django_settings (google_secret_manager_secret) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (secret-value, tcr-secret-id) | google_secret_manager_secret.django_settings | Manual | — | Can start now |
MAN/semantic/google_secret_manager_secret.django_settings/secret-settings | Decide what the STACKIT target of google_secret_manager_secret.django_settings (google_secret_manager_secret) does for its critical setting secret-settings, which the draft does not carry, and set it up by hand, or record that production does not need it | google_secret_manager_secret.django_settings | Manual | — | Can start now |
MAN/semantic/google_service_account.automation/open-settings | Review the 2 open critical settings of google_service_account.automation (google_service_account) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (role-bindings, account-name) | google_service_account.automation | Manual | — | Can start now |
MAN/semantic/google_service_account.client/open-settings | Review the 2 open critical settings of google_service_account.client (google_service_account) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (role-bindings, account-name) | google_service_account.client | Manual | — | Can start now |
MAN/semantic/google_service_account.init/open-settings | Review the 2 open critical settings of google_service_account.init (google_service_account) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (role-bindings, account-name) | google_service_account.init | Manual | — | Can start now |
MAN/semantic/google_service_account.server/open-settings | Review the 2 open critical settings of google_service_account.server (google_service_account) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (role-bindings, account-name) | google_service_account.server | Manual | — | Can start now |
TF/stackit_secretsmanager_instance.migrated_secrets | Apply stackit_secretsmanager_instance.migrated_secrets (terraform/security.tf) | google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings | Terraform | — | Can start now |
TF/stackit_secretsmanager_user.migration | Apply stackit_secretsmanager_user.migration (terraform/security.tf) | google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings | Terraform | TF/stackit_secretsmanager_instance.migrated_secrets | Can start now |
API/secrets-manager-vault/WriteSecret/google_secret_manager_secret.django_admin_password | Call the secrets-manager-vault API: WriteSecret | google_secret_manager_secret.django_admin_password | STACKIT API | TF/stackit_secretsmanager_instance.migrated_secrets, TF/stackit_secretsmanager_user.migration | Blocked by DEC-GEN-CONVERT-API-OPT-IN/google_secret_manager_secret.django_admin_password |
API/secrets-manager-vault/WriteSecret/google_secret_manager_secret.django_settings | Call the secrets-manager-vault API: WriteSecret | google_secret_manager_secret.django_settings | STACKIT API | TF/stackit_secretsmanager_instance.migrated_secrets, TF/stackit_secretsmanager_user.migration | Blocked by DEC-GEN-CONVERT-API-OPT-IN/google_secret_manager_secret.django_admin_password |
MAN/path/secrets-migration/read-back | Read each migrated secret (google_secret_manager_secret.django_admin_password and google_secret_manager_secret.django_settings) back from the Secrets Manager instance stackit_secretsmanager_instance.migrated_secrets before the migration user is removed: with its login, send the ReadSecret request of the write step (GET /v1/\<instance id>/data/\<path>, the path the write step names) and check that the secret has a current version, that its keys are the keys of the document you supplied, and that its values equal the document's (compare a hash of each value on your side; print none). | google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings | Manual | API/secrets-manager-vault/WriteSecret/google_secret_manager_secret.django_admin_password, API/secrets-manager-vault/WriteSecret/google_secret_manager_secret.django_settings | Blocked by DEC-GEN-CONVERT-API-OPT-IN/google_secret_manager_secret.django_admin_password |
MAN/path/secrets-migration/retire-user | Remove the migration user stackit_secretsmanager_user.migration of the Secrets Manager instance stackit_secretsmanager_instance.migrated_secrets once the last secret is written and read back: delete its block and its two outputs (stackit_secretsmanager_user_migration_password and stackit_secretsmanager_user_migration_username) from the draft and run tofu apply, so the provider deletes the user. Afterwards the state must not list it (tofu state list shows no stackit_secretsmanager_user.migration) and the instance must list no user of that name; give a workload that reads a secret a user of its own with write_enabled = false, or the instance's role binding. | google_secret_manager_secret.django_admin_password, google_secret_manager_secret.django_settings | Manual | MAN/path/secrets-migration/read-back | Blocked by DEC-GEN-CONVERT-API-OPT-IN/google_secret_manager_secret.django_admin_password |
TF/stackit_service_account.automation | Apply stackit_service_account.automation (terraform/identity.tf) | google_service_account.automation | Terraform | — | Can start now |
TF/stackit_service_account.client | Apply stackit_service_account.client (terraform/identity.tf) | google_service_account.client | Terraform | — | Can start now |
TF/stackit_service_account.init | Apply stackit_service_account.init (terraform/identity.tf) | google_service_account.init | Terraform | — | Can start now |
TF/stackit_service_account.server | Apply stackit_service_account.server (terraform/identity.tf) | google_service_account.server | Terraform | — | Can start now |
Wave 3 — Data services
Databases, object storage and volumes.
The manual steps of this wave in 6 work packages (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-06 | Decide and set the critical settings (data services) | Identity Architect | 2 | 2 | a person can start it now |
WP-07 | Decide and set the critical settings (data services) | Network Architect | 1 | 1 | a person can start it now |
WP-08 | Decide and set the critical settings (data services) | Platform Architect | 4 | 4 | waits for decisions (2 of 4 manual steps) |
WP-09 | Decide and set the critical settings (data services) | Security Architect | 1 | 1 | a person can start it now |
WP-10 | Decide and set the critical settings (data services) | Storage Architect | 1 | 1 | a person can start it now |
WP-11 | Decide on the switched-off modules that hold a STACKIT alternative of what has no equal (data services) | Network Architect | 1 | 1 | waits for decisions |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start |
|---|---|---|---|---|---|
MAN/estate/SOURCE_PROJECT_MAPPING | Answer the estate's question once for 1 unit: it holds 1 open critical setting, and no unit has a step of its own for it | google_sql_database_instance.postgres | Manual | — | Blocked by DEC-GEN-CONVERT-SOURCE-PROJECT-MAPPING |
MAN/estate/SOURCE_REGION_MAPPING | Answer the estate's question once for 4 units: it holds 4 open critical settings, and no unit has a step of its own for it | google_cloud_run_v2_job.migrate, google_cloud_run_v2_service.server, google_sql_database_instance.postgres, google_storage_bucket.media | Manual | — | Blocked by DEC-GEN-CONVERT-SOURCE-REGION-MAPPING |
MAN/semantic/google_sql_database_instance.postgres/database-users | Decide what the STACKIT target of google_sql_database_instance.postgres (google_sql_database_instance) does for its critical setting database-users, which the draft does not carry, and set it up by hand, or record that production does not need it | google_sql_database_instance.postgres | Manual | — | Can start now |
MAN/semantic/google_sql_database_instance.postgres/open-settings-platform-architect | Review the 2 open critical settings of google_sql_database_instance.postgres (google_sql_database_instance) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (tc-name, tcf-sql-database--name) | google_sql_database_instance.postgres | Manual | — | Can start now |
MAN/semantic/google_sql_database_instance.postgres/user-and-database-settings | Decide what the STACKIT target of google_sql_database_instance.postgres (google_sql_database_instance) does for its critical setting user-and-database-settings, which the draft does not carry, and set it up by hand, or record that production does not need it | google_sql_database_instance.postgres | Manual | — | Can start now |
MAN/semantic/google_storage_bucket.media/geo-replication | Decide what the STACKIT target of google_storage_bucket.media (google_storage_bucket) does for its critical setting geo-replication, which the draft does not carry, and set it up by hand, or record that production does not need it | google_storage_bucket.media | Manual | — | Can start now |
MAN/semantic/google_storage_bucket.media/iam-access | Decide what the STACKIT target of google_storage_bucket.media (google_storage_bucket) does for its critical setting iam-access, which the draft does not carry, and set it up by hand, or record that production does not need it | google_storage_bucket.media | Manual | — | Can start now |
MAN/semantic/google_storage_bucket.media/open-settings-identity-architect | Review the 2 open critical settings of google_storage_bucket.media (google_storage_bucket) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (iam-grant, public-access) | google_storage_bucket.media | Manual | — | Can start now |
MAN/semantic/google_storage_bucket.media/tc-storage-bucket-name | Decide what the STACKIT target of google_storage_bucket.media (google_storage_bucket) does for its critical setting tc-storage-bucket-name, which the draft does not carry, and set it up by hand, or record that production does not need it | google_storage_bucket.media | Manual | — | Can start now |
MAN/unit/google_firebase_hosting_site.client | Decide whether the STACKIT alternative in the switched-off module terraform/proposals/proposal_client replaces google_firebase_hosting_site.client (google_firebase_hosting_site): read the module, accept the stated difference, give it its values and set proposal_client_enabled to true, or set the target up another way by hand | google_firebase_hosting_site.client | Manual | — | Blocked by DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_firebase_hosting_site.client |
REF/semantic/google_storage_bucket.media/force-destroy | Not migrated: the critical setting force-destroy of google_storage_bucket.media | google_storage_bucket.media | Refused | — | Refused: A check of the knowledge base (gcs force destroy) found a setting the target has no counterpart for (a finding of this unit describes it). stackit_objectstorage_bucket (pinned provider) "cannot be destroyed if the bucket contains objects" (schema description): force_destroy = false — a destroy that stops at a non-empty bucket — is what STACKIT does, PRESERVED; true has no counterpart, UNSUPPORTED (check gcs-force-destroy). (the target has no counterpart that carries this setting of the source) |
TF/stackit_objectstorage_bucket.media | Apply stackit_objectstorage_bucket.media (terraform/storage.tf) | google_storage_bucket.media | Terraform | — | Can start now |
TF/stackit_postgresflex_instance.postgres | Apply stackit_postgresflex_instance.postgres (terraform/data.tf) | google_sql_database_instance.postgres | Terraform | — | Can start now |
Wave 4 — Platform
Kubernetes, load balancers, logging and the other shared services.
The manual steps of this wave in 7 work packages (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-13 | Decide and set the critical settings (platform services) | Identity Architect | 2 | 2 | a person can start it now |
WP-14 | Decide and set the critical settings (platform services) | Network Architect | 2 | 2 | a person can start it now |
WP-15 | Decide and set the critical settings (platform services) | Platform Architect | 3 | 3 | a person can start it now |
WP-16 | Decide and set the critical settings (platform services) | Security Architect | 2 | 2 | a person can start it now |
WP-17 | Carry by hand what the STACKIT service for pubsub topic does not take from the draft (platform services) | Platform Architect | 1 | 1 | waits for decisions |
WP-18 | Cluster access (platform services) | Platform Architect | 1 | 2 | a person can start it after the automated step it comes after has run |
WP-19 | Messaging topology (platform services) | Platform Engineer | 1 | 1 | a person can start it now |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start |
|---|---|---|---|---|---|
MAN/pubsub.topic/name/google_pubsub_topic.faux | Create the topic under a displayName of 1 to 36 letters, digits, spaces and hyphens and change its publishers to it, or keep the topic on the RabbitMQ path. | google_pubsub_topic.faux | Manual | — | Blocked by DEC-GEN-CONVERT-API-OPT-IN/google_pubsub_topic.faux |
MAN/rabbitmq/topology | Declare the topics and subscriptions of the source on the RabbitMQ instance (exchanges: 1, queues: 0): run rabbitmq/apply-topology.sh, which writes rabbitmq/topology.json through the HTTP API of the broker. Nothing of this was run on STACKIT, and whether the credential may declare them is not stated: the script asks first and stops with the answer | google_pubsub_topic.faux | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_job.migrate/cloud-run-v2-job-network-exposure | Decide what the STACKIT target of google_cloud_run_v2_job.migrate (google_cloud_run_v2_job) does for its critical setting cloud-run-v2-job-network-exposure, which the draft does not carry, and set it up by hand, or record that production does not need it | google_cloud_run_v2_job.migrate | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_job.migrate/open-settings-platform-architect | Review the 6 open critical settings of google_cloud_run_v2_job.migrate (google_cloud_run_v2_job) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (k4-cloud-run-volume-mount-cloud-run-v2-job, terraform-destroy-guard, tc-cloud-run-v2-job-name, tc-cloud-run-v2-job-template--template--volumes--name, deployment-artifact, entry-point) | google_cloud_run_v2_job.migrate | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_job.migrate/secrets | Decide what the STACKIT target of google_cloud_run_v2_job.migrate (google_cloud_run_v2_job) does for its critical setting secrets, which the draft does not carry, and set it up by hand, or record that production does not need it | google_cloud_run_v2_job.migrate | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_job.migrate/service-identity | Decide what the STACKIT target of google_cloud_run_v2_job.migrate (google_cloud_run_v2_job) does for its critical setting service-identity, which the draft does not carry, and set it up by hand, or record that production does not need it | google_cloud_run_v2_job.migrate | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_service.server/cloud-sql-connection | Decide what the STACKIT target of google_cloud_run_v2_service.server (google_cloud_run_v2_service) does for its critical setting cloud-sql-connection, which the draft does not carry, and set it up by hand, or record that production does not need it | google_cloud_run_v2_service.server | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_service.server/open-settings-identity-architect | Review the 2 open critical settings of google_cloud_run_v2_service.server (google_cloud_run_v2_service) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (iam-invocation, service-identity) | google_cloud_run_v2_service.server | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_service.server/open-settings-platform-architect | Review the 6 open critical settings of google_cloud_run_v2_service.server (google_cloud_run_v2_service) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (k4-cloud-run-volume-mount-cloud-run-v2-service, iam-policy-grant, terraform-destroy-guard, tc-cloud-run-v2-service-name, tc-cloud-run-v2-service-template--volumes--name, deployment-artifact) | google_cloud_run_v2_service.server | Manual | — | Can start now |
MAN/semantic/google_cloud_run_v2_service.server/open-settings-security-architect | Review the 2 open critical settings of google_cloud_run_v2_service.server (google_cloud_run_v2_service) that the draft does not carry, and decide for each what the STACKIT target does: set it up by hand, or record that production does not need it (environment, secrets) | google_cloud_run_v2_service.server | Manual | — | Can start now |
MAN/semantic/google_pubsub_topic.faux/tc-name | Decide what the STACKIT target of google_pubsub_topic.faux (google_pubsub_topic) does for its critical setting tc-name, which the draft does not carry, and set it up by hand, or record that production does not need it | google_pubsub_topic.faux | Manual | — | Can start now |
REF/unit/google_firebase_project.default | Not migrated: google_firebase_project.default (google_firebase_project) | google_firebase_project.default | Refused | — | Refused: Nothing on STACKIT holds the registration of an app or the enabling of Firebase on a project. It stays with Google for as long as any Firebase service (push messaging, for example) is used, and a person decides, app by app, which backend replaces the Firebase services the app calls. Adding Firebase to a project cannot be undone. (no STACKIT service or resource carries this source resource) |
TF/stackit_rabbitmq_instance.messaging | Apply stackit_rabbitmq_instance.messaging (terraform/messaging.tf) | google_pubsub_topic.faux | Terraform | — | Can start now |
TF/stackit_ske_cluster.containers | Apply stackit_ske_cluster.containers (terraform/kubernetes.tf) | google_cloud_run_v2_job.migrate, google_cloud_run_v2_service.server | Terraform | — | Can start now |
MAN/draft/handover/cluster-access/kubeconfig-md | Fetch the kubeconfig, check the health, create the namespace and the secrets of the SKE cluster(s) of the draft: the namespace is one you choose for all manifest files, the secrets are migrate-secrets, server-secrets (the keys of each are listed under SECRETS TO CREATE in the header of its manifest file) | google_cloud_run_v2_job.migrate, google_cloud_run_v2_service.server | Manual | TF/stackit_ske_cluster.containers | Can start now |
Wave 5 — Workloads
Servers and what attaches to them.
The manual steps of this wave in 2 work packages (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-20 | Decide on the switched-off modules that hold a STACKIT alternative of what has no equal (workloads) | Platform Architect | 3 | 3 | waits for decisions |
WP-21 | Kubernetes apply (workloads) | Platform Architect | 2 | 2 | a person can start after WP-18 is done |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start |
|---|---|---|---|---|---|
MAN/draft/kubernetes/migrate/manifests-yaml | Fill in the required inputs of kubernetes/migrate/manifests.yaml and apply it to the cluster stackit_ske_cluster.containers: kubectl apply -n \<namespace> -f \<manifest file>; the Job is written suspended and runs when you unsuspend it | google_cloud_run_v2_job.migrate | Manual | MAN/draft/handover/cluster-access/kubeconfig-md, TF/stackit_ske_cluster.containers | Can start after the step "Fetch the kubeconfig, check the health, create the namespace and the secrets of…" is done |
MAN/draft/kubernetes/server/manifests-yaml | Fill in the required inputs of kubernetes/server/manifests.yaml and apply it to the cluster stackit_ske_cluster.containers (put 0 in the number of pods): kubectl apply -n \<namespace> -f \<manifest file>; no pod starts. The pods start when the data they connect to is moved, by the person who runs the cutover (step 2 of the header) | google_cloud_run_v2_service.server | Manual | MAN/draft/handover/cluster-access/kubeconfig-md, TF/stackit_ske_cluster.containers | Can start after the step "Fetch the kubeconfig, check the health, create the namespace and the secrets of…" is done |
MAN/unit/google_cloudbuild_trigger.activategcb | Decide whether the STACKIT alternative in the switched-off module terraform/proposals/proposal_activategcb replaces google_cloudbuild_trigger.activategcb (google_cloudbuild_trigger): read the module, accept the stated difference, give it its values and set proposal_activategcb_enabled to true, or set the target up another way by hand | google_cloudbuild_trigger.activategcb | Manual | — | Blocked by DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_cloudbuild_trigger.activategcb |
MAN/unit/google_cloudbuild_trigger.init | Decide whether the STACKIT alternative in the switched-off module terraform/proposals/proposal_init replaces google_cloudbuild_trigger.init (google_cloudbuild_trigger): read the module, accept the stated difference, give it its values and set proposal_init_enabled to true, or set the target up another way by hand | google_cloudbuild_trigger.init | Manual | — | Blocked by DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_cloudbuild_trigger.init |
MAN/unit/google_cloudbuild_trigger.placeholder | Decide whether the STACKIT alternative in the switched-off module terraform/proposals/proposal_placeholder replaces google_cloudbuild_trigger.placeholder (google_cloudbuild_trigger): read the module, accept the stated difference, give it its values and set proposal_placeholder_enabled to true, or set the target up another way by hand | google_cloudbuild_trigger.placeholder | Manual | — | Blocked by DEC-GEN-CONVERT-PROPOSED-ALTERNATIVE/google_cloudbuild_trigger.placeholder |
Wave 6 — Data migration
Copying the data from the source to the new services.
The manual steps of this wave in 3 work packages (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-12 | Move the data and the objects to the target | Data Architect | 8 | 2 | waits for decisions (2 of 8 manual steps) |
WP-22 | Move the data and the objects to the target | Data Architect | 8 | 2 | waits for decisions (6 of 8 manual steps) |
WP-23 | Move the data and the objects to the target | Migration Owner | 4 | 2 | waits for decisions (3 of 4 manual steps) |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start | ||||
|---|---|---|---|---|---|---|---|---|---|
MAN/data/google_storage_bucket.media/preflight | Close what the plan lists as UNKNOWN, from the move host, before any object moves. The credentials of the target are the environment names TARGET_ACCESS_KEY_ID and TARGET_SECRET_ACCESS_KEY, set in the shell of the move host at run time from the credential of a credentials group; the source uses the Google identity of that shell (the service account the host runs as, or a login you made for the move; rclone's env_auth takes it from the environment). No value is written into a file, a ticket or this plan. In the shell of the move host configure the two remotes from the environment: export RCLONE_CONFIG_SRC_TYPE="google cloud storage" RCLONE_CONFIG_SRC_ENV_AUTH=true RCLONE_CONFIG_DST_TYPE=s3 RCLONE_CONFIG_DST_PROVIDER=Other RCLONE_CONFIG_DST_ENDPOINT=https://object.storage.eu01.onstackit.cloud RCLONE_CONFIG_DST_ACCESS_KEY_ID="$TARGET_ACCESS_KEY_ID" RCLONE_CONFIG_DST_SECRET_ACCESS_KEY="$TARGET_SECRET_ACCESS_KEY" RCLONE_CONFIG_DST_ACL=private. Source: rclone lsd SRC:, rclone size SRC:<source-bucket> --json (the count and the bytes), gcloud storage buckets describe gs://<source-bucket> (versioning, retention policy, default key, requester pays, soft delete policy, storage class), and for the objects the owner names rclone lsjson --metadata SRC:<source-bucket> --files-only -R --max-depth 1. Target (after the draft is applied, the bucket exists and a key of a credentials group is issued): rclone lsd DST: and rclone size DST:<target-bucket> --json: the bucket is empty. Record the number of objects and the bytes beside the size the customer stated (the size of the data is not provided); the plan has no input for the number of objects. Check that the move host reaches both endpoints. Runbook section: terraform/handover/bucket-media/data-copy.md#preflight. | google_storage_bucket.media | Manual | TF/stackit_objectstorage_bucket.media | Can start now | ||||
MAN/data/google_storage_bucket.media/rehearsal | Rehearse the copy with a representative part of the bucket: the objects of one prefix and the largest and the smallest objects, into the prefix stacktoit-rehearsal/ of the target bucket. Note the elapsed time and the bytes a second for small and for large objects, compare rclone lsjson --metadata of a sample on both sides (content type, custom metadata, storage class, modification time), and delete the prefix stacktoit-rehearsal/ when the rehearsal is done. Runbook section: terraform/handover/bucket-media/data-copy.md#rehearsal. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/preflight | Can start after the data preflight of google_storage_bucket.media is done | ||||
MAN/data/google_storage_bucket.media/initial-copy/rclone-copy | While the writers keep writing to the source, copy every live object to the target. The credentials of the target are the environment names TARGET_ACCESS_KEY_ID and TARGET_SECRET_ACCESS_KEY, set in the shell of the move host at run time from the credential of a credentials group; the source uses the Google identity of that shell (the service account the host runs as, or a login you made for the move; rclone's env_auth takes it from the environment). No value is written into a file, a ticket or this plan. With the remotes of the preflight: rclone copy SRC:<source-bucket> DST:<target-bucket> --s3-upload-cutoff 0 --progress --log-file=<log-file> (the form STACKIT's tutorial uses). Read the log for every error before the next step. Runbook section: terraform/handover/bucket-media/data-copy.md#initial-copy-rclone-copy. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/rehearsal | Can start after the data rehearsal of google_storage_bucket.media is done | ||||
MAN/data/google_storage_bucket.media/delta-sync/rclone-copy | Run the same rclone copy again, as often as the business needs the target to be close to the source: each pass skips the objects that are identical (rclone compares size and modification time; --checksum compares size and hash where there is one, and Cloud Storage stores the MD5 natively) and copies the ones that are new or changed. It deletes nothing on the target, so an object deleted in the source stays there until the final pass. Note how many objects and bytes each pass transferred. Runbook section: terraform/handover/bucket-media/data-copy.md#delta-sync-rclone-copy. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/initial-copy/rclone-copy | Can start after the initial copy of the data of google_storage_bucket.media with rclone is done | ||||
MAN/data/google_sql_database_instance.postgres/preflight | Close what the plan lists as UNKNOWN, from the move host, with read-only queries. The move host reaches the source through the connection method you already use for Cloud SQL for PostgreSQL (an authorized network of a public address, a private address inside the VPC network, or a proxy you run). Source: PGPASSWORD="$SOURCE_PGPASSWORD" PGSSLMODE=require psql --host=<source-host> --port=<source-port> --username=<source-user> --dbname=<source-db> --command='SHOW server_version;', SELECT extname, extversion FROM pg_extension ORDER BY 1;, SELECT pg_size_pretty(pg_database_size(current_database)); against the free storage of the instance (at least 25 percent of the database size free before a large export), \du and \dp for the roles and grants, and the users that sign in with an IAM identity. Target (after the draft is applied and the instance, user and database exist): PGPASSWORD="$TARGET_PGPASSWORD" PGSSLMODE=require psql --host=<target-host> --port=<target-port> --username=<target-user> --dbname=<target-db> --command='SHOW extwlist.extensions;' and SELECT version;. Check the move host is in network.acl of the instance, reaches both ends and speaks TLS 1.3. Logical replication: on the source SHOW wal_level;, SHOW max_replication_slots;, SHOW max_wal_senders; and the tables without a primary key (SELECT n.nspname, c.relname FROM pg_class c JOIN pg_namespace n ON n.oid = c.relnamespace WHERE c.relkind = 'r' AND n.nspname NOT IN ('pg_catalog', 'information_schema') AND NOT EXISTS (SELECT 1 FROM pg_index i WHERE i.indrelid = c.oid AND i.indisprimary);); on the empty target the probe CREATE SUBSCRIPTION stacktoit_probe CONNECTION 'host=localhost dbname=x' PUBLICATION p WITH (connect = false); then ALTER SUBSCRIPTION stacktoit_probe SET (slot_name = NONE); and DROP SUBSCRIPTION stacktoit_probe;. Runbook section: terraform/handover/database-postgres/data-move.md#preflight. | google_sql_database_instance.postgres | Manual | TF/stackit_postgresflex_instance.postgres | Can start now | ||||
MAN/data/google_sql_database_instance.postgres/rehearsal | Rehearse the chosen method into a scratch database of the target (a second stackit_postgresflex_database or a clone of the instance) from a copy of the source (a clone or a restored backup of the Cloud SQL for PostgreSQL instance, so that production is not loaded). Note the elapsed time of the dump, the restore, the index build and the verification. Drop the scratch database and the copy afterwards. Runbook section: terraform/handover/database-postgres/data-move.md#rehearsal. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/preflight | Can start after the data preflight of google_sql_database_instance.postgres is done | ||||
MAN/data/google_sql_database_instance.postgres/initial-copy/logical-replication | Source: turn logical decoding on: set the flag cloudsql.logical_decoding to on on the instance (a restart of the instance), create a user for the publication with the REPLICATION attribute that holds CREATE on the database (CREATE ROLE stacktoit_repl WITH LOGIN REPLICATION PASSWORD '<replication-password>';, typed in an interactive session). Every table needs a primary key or a replica identity. Copy the schema (no later DDL on either side): `PGPASSWORD="$SOURCE_PGPASSWORD" pg_dump --schema-only --no-owner --no-acl --host=<source-host> --port=<source-port> --username=<source-user> --dbname=<source-db> \ | PGPASSWORD="$TARGET_PGPASSWORD" PGSSLMODE=require psql --host=<target-host> --port=<target-port> --username=<target-user> --dbname=<target-db>. Publish on the source: CREATE PUBLICATION stacktoit_move FOR ALL TABLES;. Subscribe on the target in an interactive psql session so that no password is stored in a file or the shell history: CREATE SUBSCRIPTION stacktoit_move CONNECTION 'host=<source-host> port=<source-port> dbname=<source-db> user=stacktoit_repl password=<replication-password> sslmode=require' PUBLICATION stacktoit_move;. Wait until SELECT srrelid::regclass, srsubstate FROM pg_subscription_rel; shows r for every table. Runbook section: terraform/handover/database-postgres/data-move.md#initial-copy-logical-replication`. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/rehearsal | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | |||
MAN/data/google_sql_database_instance.postgres/delta-sync/logical-replication | Keep the target apart: the application does not write to it and no DDL runs on either side. Watch the lag and the free storage of the source: SELECT slot_name, active, pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn)) AS lag FROM pg_replication_slots;. Runbook section: terraform/handover/database-postgres/data-move.md#delta-sync-logical-replication. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/initial-copy/logical-replication | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | ||||
MAN/data/google_sql_database_instance.postgres/write-freeze/dump-restore | List who writes (SELECT usename, application_name, client_addr, count(*) FROM pg_stat_activity WHERE datname = '<source-db>' GROUP BY 1, 2, 3 ORDER BY 4 DESC;) and record the connection settings of every client where they can be restored from. Then stop the writers at the application (services, jobs, migrations, cron). Do not make the source read-only and do not change it. Check: SELECT count(*) FROM pg_stat_activity WHERE datname = '<source-db>' AND state <> 'idle' AND pid <> pg_backend_pid(); is 0. Runbook section: terraform/handover/database-postgres/data-move.md#write-freeze-dump-restore. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/rehearsal | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | ||||
MAN/data/google_sql_database_instance.postgres/final-sync/dump-restore | Dump inside the freeze: PGPASSWORD="$SOURCE_PGPASSWORD" PGSSLMODE=require pg_dump --host=<source-host> --port=<source-port> --username=<source-user> --dbname=<source-db> --format=directory --jobs=4 --no-owner --no-acl --file=<dump-dir> (Google's page names --no-owner and --no-acl for a dump of one database and the directory format for a parallel dump; dump one database at a time, since pg_dumpall fails for databases with several owners). Restore as the database owner into the empty database: PGPASSWORD="$TARGET_PGPASSWORD" PGSSLMODE=require pg_restore --host=<target-host> --port=<target-port> --username=<target-user> --dbname=<target-db> --jobs=4 --no-owner --no-acl <dump-dir> 2> restore.log, then read restore.log line by line; create the extensions of the source that the target lists and settle the ones it does not. Runbook section: terraform/handover/database-postgres/data-move.md#final-sync-dump-restore. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/write-freeze/dump-restore | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | ||||
MAN/data/google_sql_database_instance.postgres/reconcile/dump-restore | Both sides, with the writers stopped: per-table counts (`SELECT format('SELECT %L AS tbl, count(*) FROM %I.%I;', schemaname \ | \ | '.' \ | \ | tablename, schemaname, tablename) FROM pg_tables WHERE schemaname NOT IN ('pg_catalog', 'information_schema'); run on each side), an md5 of every keyed table (SELECT count(*), md5(string_agg(t::text, ',' ORDER BY t.<key-column>)) FROM <schema>.<table> t;, a key range for a large table, DateStyle, TimeZone and extra_float_digits equal in both sessions), the sequences, and one read query the application itself runs on its most important data, <application-query>: the query and the answer are the customer's, the source does not state them. Record GO or NO-GO. Runbook section: terraform/handover/database-postgres/data-move.md#reconcile-dump-restore`. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/final-sync/dump-restore | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT |
MAN/data/google_sql_database_instance.postgres/write-freeze/logical-replication | List who writes (SELECT usename, application_name, client_addr, count(*) FROM pg_stat_activity WHERE datname = '<source-db>' GROUP BY 1, 2, 3 ORDER BY 4 DESC;) and record the connection settings of every client where they can be restored from. Then stop the writers at the application (services, jobs, migrations, cron). Do not make the source read-only and do not change it. Check: SELECT count(*) FROM pg_stat_activity WHERE datname = '<source-db>' AND state <> 'idle' AND pid <> pg_backend_pid(); is 0. Runbook section: terraform/handover/database-postgres/data-move.md#write-freeze-logical-replication. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/delta-sync/logical-replication | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | ||||
MAN/data/google_sql_database_instance.postgres/final-sync/logical-replication | Wait until the lag is 0. Copy the sequences: on the source `SELECT format('SELECT setval(%L, %s);', schemaname \ | \ | '.' \ | \ | sequencename, last_value) FROM pg_sequences WHERE last_value IS NOT NULL; with psql --tuples-only --no-align, and run the output on the target. Then disable the subscription on the target: ALTER SUBSCRIPTION stacktoit_move DISABLE; (the slot stays on the source, so the way back stays open). Runbook section: terraform/handover/database-postgres/data-move.md#final-sync-logical-replication`. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/write-freeze/logical-replication | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT |
MAN/data/google_sql_database_instance.postgres/reconcile/logical-replication | Both sides, with the writers stopped: per-table counts (`SELECT format('SELECT %L AS tbl, count(*) FROM %I.%I;', schemaname \ | \ | '.' \ | \ | tablename, schemaname, tablename) FROM pg_tables WHERE schemaname NOT IN ('pg_catalog', 'information_schema'); run on each side), an md5 of every keyed table (SELECT count(*), md5(string_agg(t::text, ',' ORDER BY t.<key-column>)) FROM <schema>.<table> t;, a key range for a large table, DateStyle, TimeZone and extra_float_digits equal in both sessions), the sequences, and one read query the application itself runs on its most important data, <application-query>: the query and the answer are the customer's, the source does not state them. Record GO or NO-GO. Runbook section: terraform/handover/database-postgres/data-move.md#reconcile-logical-replication`. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/final-sync/logical-replication | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT |
MAN/data/google_sql_database_instance.postgres/write-freeze/own-tool | List who writes (SELECT usename, application_name, client_addr, count(*) FROM pg_stat_activity WHERE datname = '<source-db>' GROUP BY 1, 2, 3 ORDER BY 4 DESC;) and record the connection settings of every client where they can be restored from. Then stop the writers at the application (services, jobs, migrations, cron). Do not make the source read-only and do not change it. Check: SELECT count(*) FROM pg_stat_activity WHERE datname = '<source-db>' AND state <> 'idle' AND pid <> pg_backend_pid(); is 0. Runbook section: terraform/handover/database-postgres/data-move.md#write-freeze-own-tool. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/rehearsal | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | ||||
MAN/data/google_sql_database_instance.postgres/final-sync/own-tool | Copy the data of google_sql_database_instance.postgres to stackit_postgresflex_instance.postgres with the migration tool or the replication you chose: write its steps, the checks before the move, the freeze window it needs and the way back. Runbook section: terraform/handover/database-postgres/data-move.md#final-sync-own-tool. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/write-freeze/own-tool | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT | ||||
MAN/data/google_sql_database_instance.postgres/reconcile/own-tool | Both sides, with the writers stopped: per-table counts (`SELECT format('SELECT %L AS tbl, count(*) FROM %I.%I;', schemaname \ | \ | '.' \ | \ | tablename, schemaname, tablename) FROM pg_tables WHERE schemaname NOT IN ('pg_catalog', 'information_schema'); run on each side), an md5 of every keyed table (SELECT count(*), md5(string_agg(t::text, ',' ORDER BY t.<key-column>)) FROM <schema>.<table> t;, a key range for a large table, DateStyle, TimeZone and extra_float_digits equal in both sessions), the sequences, and one read query the application itself runs on its most important data, <application-query>: the query and the answer are the customer's, the source does not state them. Record GO or NO-GO. Runbook section: terraform/handover/database-postgres/data-move.md#reconcile-own-tool`. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/final-sync/own-tool | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT |
MAN/data/google_storage_bucket.media/write-freeze/rclone-copy | List who writes to the bucket: the units the README lists for it, and every job, partner and signed URL outside the scan (if Cloud Audit Logs record data writes for the bucket, its object writes and deletes name the principals; the Terraform does not show whether they are). Record the endpoint, the bucket name and the credential settings of every client where they can be restored from. Then stop the writers at the application (services, jobs, uploads). Do not change the source bucket (no read-only policy, no retention lock): it stays read-write, so the way back needs nothing on the source. Runbook section: terraform/handover/bucket-media/data-copy.md#write-freeze-rclone-copy. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/delta-sync/rclone-copy | Can start after the replication watch of google_storage_bucket.media with rclone is done | ||||
MAN/data/google_storage_bucket.media/final-sync/rclone-copy | Inside the freeze run the rclone copy of the initial copy once more and read how many objects it transferred; run it a second time: it transfers nothing. Then see what the target holds that the source no longer has: rclone sync SRC:<source-bucket> DST:<target-bucket> --dry-run prints what a sync would delete; read the list, and only a person who has read it runs the sync without --dry-run (the target must equal the source at the cutover, and an object deleted in the source after the initial copy is otherwise still served). Runbook section: terraform/handover/bucket-media/data-copy.md#final-sync-rclone-copy. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/write-freeze/rclone-copy | Can start after the write freeze of google_storage_bucket.media with rclone is done | ||||
MAN/data/google_storage_bucket.media/reconcile/rclone-copy | Both sides, with the writers stopped: the number of objects and the bytes (rclone size SRC:<source-bucket> --json, rclone size DST:<target-bucket> --json); the list of keys (`rclone lsf -R --files-only SRC:<source-bucket> \ | sort > source-keys.txt, the same for the target, diff source-keys.txt target-keys.txt is empty); and the content: rclone check SRC:<source-bucket> DST:<target-bucket> --one-way --combined=<check-file> compares size and hash where both sides have one and marks every difference, so read the file and the log; an object for which the two sides share no hash (the target's ETag of a multipart upload is not an MD5) is at best compared by size, which is not its content, so run rclone check ... --download for the prefixes that matter and for a sample of the rest. One read that the application itself does on its most important objects through the STACKIT endpoint with a key of a credentials group: <application-read>, the query and the answer are the customer's, the source does not state them. Record GO or NO-GO. Runbook section: terraform/handover/bucket-media/data-copy.md#reconcile-rclone-copy`. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/final-sync/rclone-copy | Can start after the final copy of the data of google_storage_bucket.media with rclone is done |
Wave 7 — Cutover
Moving traffic to the target and the steps that must be taken before production.
No step of the plan is in this wave.
Wave 8 — Verification
Checking the migrated estate and the work after the cutover.
No step of the plan is in this wave.
Wave 9 — Decommission
Retiring the source.
The manual steps of this wave in 1 work package (execution/manual/actions.md lists every package with its steps):
| Package | What | Owner | Steps | Affected resources | State |
|---|---|---|---|---|---|
WP-24 | Retire the source resources | Data Architect | 3 | 2 | waits for decisions (2 of 3 manual steps) |
The steps of the wave, the manual ones among them:
| Step | What | Google Cloud resource | Mode | Depends on | Can start |
|---|---|---|---|---|---|
MAN/data/google_sql_database_instance.postgres/cleanup/dump-restore | Only after the target has run in production for the period the customer decides and the way back is no longer wanted: remove the move host from network.acl of the instance and from the authorized networks of the source if it was added; delete the dump files and restore.log; keep a final backup of the source for the retention the customer decides before anything is deleted; replace the credentials of the target user if they were ever on a ticket or in a file. Runbook section: terraform/handover/database-postgres/data-move.md#clean-up-dump-restore. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/reconcile/dump-restore | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT |
MAN/data/google_sql_database_instance.postgres/cleanup/logical-replication | Only after the target has run in production for the period the customer decides and the way back is no longer wanted: DROP SUBSCRIPTION stacktoit_move; on the target (it drops the slot on the source if the source is reachable; check SELECT * FROM pg_replication_slots;), DROP PUBLICATION stacktoit_move; and DROP ROLE stacktoit_repl; on the source, set cloudsql.logical_decoding back with a second restart if it was off before the move; remove the move host from network.acl of the instance and from the authorized networks of the source if it was added; delete the dump files and restore.log; keep a final backup of the source for the retention the customer decides before anything is deleted; replace the credentials of the target user if they were ever on a ticket or in a file. Runbook section: terraform/handover/database-postgres/data-move.md#clean-up-logical-replication. | google_sql_database_instance.postgres | Manual | MAN/data/google_sql_database_instance.postgres/reconcile/logical-replication | Blocked by DEC-GEN-CONVERT-DATA-MOVEMENT |
MAN/data/google_storage_bucket.media/cleanup/rclone-copy | Only after the target has run in production for the period the customer decides and the way back is no longer wanted: delete the key lists and the logs of the move (they name every object key); delete the credential made for the move, the Google login or key made for it, and take the environment names out of the shell history; keep the source bucket, with its versions, for the retention the customer decides before any object of it is deleted, and do not delete it in the same hour as the cutover. Runbook section: terraform/handover/bucket-media/data-copy.md#clean-up-rclone-copy. | google_storage_bucket.media | Manual | MAN/data/google_storage_bucket.media/reconcile/rclone-copy | Can start after the data reconciliation of google_storage_bucket.media with rclone is done |