Skip to main content
Production database operation. Coordinate with the team and announce in advance. Although this is designed to be zero-downtime, missteps can cut every service off from the database.

​
When to use this

  • Suspected or confirmed credential leak
  • Routine, scheduled rotation
  • Offboarding / revoking access tied to the current default user

​
Prerequisites

  • Access to the Render Dashboard for the target db instance
  • Access to the HCP Terraform Cloud workspace (production / sandbox / test)
  • A way to query the DB to watch active connections — see Connecting to the prod DB
  • Approval / notification: loop in the on-call engineer and the platform lead

​
Procedure

​
1. Generate a new default credential on Render

  1. Render Dashboard → the db instance → Info page → Credentials section.
  2. Click + New default credential. Leave the username blank so Render generates one, then click Create Credential.
  3. The new user becomes the default user immediately.
The original/previous user is not deleted yet — it keeps working until explicitly deleted in step 6. This is what makes running services (with env vars holding the old credentials) not break and allows the rotation to be done with zero downtime.
See Render’s documentation for more details.

​
2. Apply Terraform to pick up the new credentials

Terraform reads the new default database_user and connection_info.password from Render and propagates them to every service. Inspect the plan output in Terraform Cloud and apply if everything looks good.
  • TODO: confirm the exact set of resources expected to change (env group / service env vars) and add a sample plan snippet.
Sanity-check the plan: it must show env-var updates only. It must not show render_postgres.db being replaced/destroyed (prevent_destroy = true should block that, but stop and investigate if you see it).

​
3. Redeploy services and confirm they’re healthy

  • The DB credentials live in a shared env group, so services may not redeploy automatically when Terraform updates it. For each affected service (API + workers), trigger Manual Deploy → Deploy latest reference in the Render Dashboard.
  • Verify the API + workers come back up on the new credentials.
  • TODO: links / checks — Render service health, sentry, logfire etc.

​
4. Update Metabase credentials

Metabase connects to the database with its own copy of the credentials and is not managed by Terraform, so update it manually:
  • Metabase → Admin → Databases → polar_cpit_p9lf → Edit connection details, and set the new username and password.

​
5. Monitor for use of the old credential

Confirm nothing is still connecting as the old user before deleting it.
  • Wait until the old username shows zero connections.
  • TODO: typical settling time; what to do if a stubborn connection lingers (which service, how to force a redeploy/restart).

​
6. Delete the old user on Render

  1. Render Dashboard → db → Credentials → trashcan icon next to the old user, then confirm.
  2. The user and its credentials are removed immediately.
If the user to delete is the original user then technically Render doesn’t delete the user but rather revokes its login privileges. See Render’s documentation for more details.

​
Verification

  • terraform plan is clean (no pending drift on render_postgres.db)
  • API + workers healthy on the new credential
  • Old user shows zero active connections
  • Old user removed
  • Operation recorded in the on-call log (see Safety Guidelines)

​
Rollback / troubleshooting

There is no rollback, only roll-forward. Render can’t promote existing credentials back to default — the only way to change the default is to create new credentials (see the Background). So the recovery path for a bad rotation is another rotation (create fresh credentials and re-apply), not reverting to the old user. Until you delete the old user in step 6 its credentials still work, so an in-progress rotation can be safely abandoned before that point.
  • TODO: what to do if services fail to connect after apply (re-check plan applied cleanly; the default user is whatever Render last promoted).
  • TODO: behavior if a new default credential was generated but Terraform was not applied (env vars stale → services on old user until apply).

​
Background: how rotation works here

We use Render’s built-in credential rotation: Render designates one role as the instance’s default user and exposes its credentials in the dashboard and connection strings. Creating a new default credential generates a fresh username & password pair and promotes it to the new default. Default credentials are available in terraform and our terraform config populates env vars based on values read from Render:
  • POLAR_POSTGRES_USER & POLAR_POSTGRES_READ_USER are populated from render_postgres.db.database_user which gives the default credentials’ username
  • POLAR_POSTGRES_PWD & POLAR_POSTGRES_READ_PWD are populated from render_postgres.db.connection_info.password which gives the default credentials’ password
Both attributes always reflect the current default credentials, so they stay in sync (no risk of new-user + old-password). This means a single terraform apply re-renders these env vars, replacing Render’s manual “update config” step. Because the variables live in a shared env group, services don’t always pick up the change automatically — you may need to redeploy each one manually (see step 3).
Read replicas reuse local.db_user / local.db_password (the primary’s default credentials), so they rotate together automatically.

​
See also