Migrate from 1.23.3 to 2.3.3

Uncategorized
1

I am in need of migrating a 1.23.3 installation to 2.3.3… when i run
docker exec baserow ./baserow.sh backend-cmd manage export_workspace_applications 4 --indent

i get this

OTEL_RESOURCE_ATTRIBUTES=service.namespace=Baserow,service.version=1.23.2,deployment.environment=unknown
Traceback (most recent call last):
File “/baserow/backend/src/baserow/core/handler.py”, line 1575, in export_workspace_applications
exported_application = application_type.export_serialized(
File “/baserow/backend/src/baserow/contrib/database/application_types.py”, line 178, in export_serialized
serialized_tables = self.export_tables_serialized(
File “/baserow/backend/src/baserow/contrib/database/application_types.py”, line 127, in export_tables_serialized
serialized_row[field_name] = field_type.get_export_serialized_value(
File “/baserow/backend/src/baserow/contrib/database/fields/field_types.py”, line 3218, in get_export_serialized_value
files_zip.writestr(file[“name”], storage_file.read())
File “/usr/lib/python3.9/zipfile.py”, line 1802, in writestr
with self.open(zinfo, mode=‘w’) as dest:
File “/usr/lib/python3.9/zipfile.py”, line 1505, in open
return self._open_to_write(zinfo, force_zip64=force_zip64)
File “/usr/lib/python3.9/zipfile.py”, line 1597, in _open_to_write
self._writecheck(zinfo)
File “/usr/lib/python3.9/zipfile.py”, line 1712, in _writecheck
raise LargeZipFile(requires_zip64 +
zipfile.LargeZipFile: Zipfile size would require ZIP64 extensions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/baserow/backend/src/baserow/manage.py”, line 41, in
main()
File “/baserow/backend/src/baserow/manage.py”, line 37, in main
execute_from_command_line(sys.argv)
File “/baserow/venv/lib/python3.9/site-packages/django/core/management/_init_.py”, line 446, in execute_from_command_line
utility.execute()
File “/baserow/venv/lib/python3.9/site-packages/django/core/management/_init_.py”, line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File “/baserow/venv/lib/python3.9/site-packages/django/core/management/base.py”, line 402, in run_from_argv
self.execute(*args, **cmd_options)
File “/baserow/venv/lib/python3.9/site-packages/django/core/management/base.py”, line 448, in execute
output = self.handle(*args, **options)
File “/baserow/backend/src/baserow/core/management/commands/export_workspace_applications.py”, line 67, in handle
exported_applications = CoreHandler().export_workspace_applications(
File “/baserow/backend/src/baserow/core/telemetry/utils.py”, line 72, in _wrapper
raise ex
File “/baserow/backend/src/baserow/core/telemetry/utils.py”, line 68, in _wrapper
result = wrapped_func(*args, **kwargs)
File “/baserow/backend/src/baserow/core/handler.py”, line 1578, in export_workspace_applications
exported_applications.append(exported_application)
File “/usr/lib/python3.9/zipfile.py”, line 1300, in _exit_
self.close()
File “/usr/lib/python3.9/zipfile.py”, line 1825, in close
self._write_end_record()
File “/usr/lib/python3.9/zipfile.py”, line 1900, in _write_end_record
raise LargeZipFile(requires_zip64 +
zipfile.LargeZipFile: Central directory offset would require ZIP64 extensions

indicating baserow has a bug implementing zip64… is there a quick/easy way around this or should i move on with dumping the db and restoring to the new instance?

2

Hello @opentechnw and welcome to the Baserow community :waving_hand:

Thanks for the detailed error report! Before we check this with our team, could you please confirm:

  • The exact Baserow version currently running (the traceback shows 1.23.2, while you mentioned 1.23.3)
  • Whether you’re using the official Baserow Docker installation
  • Approximately how large workspace 4 is, and whether it contains large file attachments

This will help us investigate the error further :slightly_smiling_face:

3

it is 1.23.2, just fat fingered it.

i am using the docker install

there are some attachments >25MB and <30MB so in not sure what large means… but there are ~5600 attachments for a total of 8.9GB

4

any ideas on how to get this migrated?

5

Hello @opentechnw, I’m really sorry for the delay — I was on vacation for the last three weeks. Is this still an issue? We also released Baserow 2.4 today, so perhaps you’ve already tried migrating to the latest version?

6

It is still an issue, I can upgrade my new instance to 2.4 but the issue at this point is in exporting from the old instance

7

Hi @opentechnw, thanks for your patience, and sorry this one took a while.

Good news first: you don’t need the export at all. The error you hit is a limitation in older Baserow versions, where the export tool couldn’t write a ZIP file bigger than 2 GB. With 8.9 GB of attachments it was always going to fail. That was fixed in 1.30, but there’s an easier route for you anyway.

Instead of exporting and importing, you can upgrade your existing installation in place. The new version runs on the same data folder, so everything comes along: users, permissions, API tokens, webhooks, comments, row history, and all your files. We tested this exact path from 1.23.2 to 2.4.0 and it worked cleanly.

Here’s what to do:

  1. Make a backup

Before anything else, back up your data folder. The steps are in our docs under “Backing up and restoring Baserow”: Install with Docker

  1. Stop your current Baserow

docker stop baserow

  1. Upgrade the embedded database

Baserow 1.23 used PostgreSQL 11 internally, and newer versions use PostgreSQL 15. This one-off step converts your data folder. Run it with the same -v and -e arguments you normally use for Baserow:

docker run --name baserow-pgautoupgrade --restart no
<your usual -v and -e arguments>
baserow/baserow-pgautoupgrade:1.24.2

It will print “Upgrade complete” when done. Don’t worry if Docker reports it exited with code 1, that’s expected for this image. You can remove the helper container afterwards with docker rm baserow-pgautoupgrade.

If you connect Baserow to your own external PostgreSQL rather than the built-in one, skip this step.

  1. Start the new Baserow

Start Baserow the way you normally do, just with the new image tag:

docker run -d --name baserow
<your usual -v, -e and -p arguments>
baserow/baserow:2.4.0

Follow the logs with docker logs -f baserow. The upgrade runs automatically on first start and can take a few minutes with a database your size. You may see some red error lines from the background workers while that’s happening. Those are harmless and stop once the upgrade finishes. When you see “Baserow is now available”, you’re done.

Once you’ve confirmed everything looks right, you can remove the old container.

Let us know how it goes!