Migrate date from all-in-one docker to Kubernetes with managed PostgreSQL and MinIO

Technical Help
1

Please fill in the questionnaire below.

Technical Help Questionnaire

Have you read and followed the instructions at: *READ ME FIRST* Technical Help FAQs - #2 by nigel ?

Answer: Yes

How have you self-hosted Baserow.

All-in-one docker

What are the specs of the service or server you are using to host Baserow.

               total        used        free      shared  buff/cache   available
Mem:            15Gi       1.8Gi       7.9Gi        43Mi       5.9Gi        13Gi
Swap:          979Mi          0B       979Mi

Which version of Baserow are you using.

1.26.1 - enterprise

How have you configured your self-hosted installation?

#!/bin/sh
docker run
-d
–name baserow
-e BASEROW_PUBLIC_URL=“xxx”
-e BASEROW_CADDY_ADDRESSES=“xxx”
-e EMAIL_SMTP=true
-e EMAIL_SMTP_HOST=“xxx”
-e EMAIL_SMTP_PORT=“xxxx”
-e FROM_EMAIL=“noreply@eduxs.eu”
-v baserow_data:/baserow/data
-v “$PWD/Caddyfile:/baserow/caddy/Caddyfile”
-p 80:80
-p 443:443
–restart unless-stopped
baserow/baserow:1.26.1

What commands if any did you use to start your Baserow server?

See above.

Describe the problem

We are use the docker all-in-one Baserow at the moment. For backup/restore, we are using the approach provided in the docs which is copying all the data from the mounted volume in the docker container.

Now we want to move our Baserow instance to a Kubernetes cluster with a managed Postgresql and MinIO. So, we do not have access to the volumes where the PostgreSQL and MinIO data is stored. What would be the recommended approach to move our data to the new environment?

2

Hey @jjmurre, I recommend checking out the instructions here Install with Docker. The first step about taking a backup results in a tar containing a PostgreSQL and uploaded media files dump.

When you’ve deployed your Baserow Kubernetes cluster, you can connect directly to the PostgreSQL database and restore the dump there. Same for the media files in the MinIO S3 bucket. This should then restore your complete instance.

If you’re using AWS, Azure, or another cloud provider, then I recommend using their managed PostgreSQL, S3, and Redis from there. That typically comes with additional backup, scaling, and monitoring capabilities.

3

Hi @bram Tx. Ah, I see, I assume I need to use the " Backup only Baserow’s Postgres database" in our case (because we do not have access to the filesystem of the managed postgres).

As for MinIO, I assume we can dump the baserow MiniIO buckets of our docker all-in-one and push those into a bucket on the managed MinIO?

4

Hey @jjmurre, in this case I think you would still need to use the “Backup all of Baserow” because if you’re migrating to another instance you would need to the PostgreSQL dump and also the uploaded files. The uploaded files are the files that the user uploads in the file field, for example. In the all-in-one image, this is just stored in a folder, not on MinIO.

Basically, the MinIO dump you would like to take from the all-in-one image, is already included if you use the “Backup all of Baserow” method to make the backup.

Once you have the two, then you can work on restoring the PostgreSQL dump into the PostgreSQL database and uploaded user files into the MinIO deployments in the Kubernetes cluster.

I hope that makes sense.

5

The problem is in the restore process:

docker run --rm -v new_baserow_data_volume:/results -v $PWD:/backup ubuntu bash -c "mkdir -p /results/ && cd /results && tar xvf /backup/backup.tar --strip 2"

The filesystem where the PostgreSQL data lives needs to be mounted. However, because we are using a managed (by another department) PostgreSQL instance, we are not able to mount the data volume of that instance.