I would like to import everything I have in our Airtable Enterprise account:
~ 10 databases
100s of thousands of rows in total
100s of tables
500+ fields
10+ million “cells”
lots of formula fields and calculations
In Baserow’s documentation there isn’t any information regarding limitations (or workarounds) for importing that magnitude of data nor of features in Airtable not supported by Baserow and what happens in those cases.
There are some forum discussions related to this although some are quite old or not really about what I’m asking:
Please consider that we would run the import on a self-hosted Baserow instance with an Enterprise license, with whatever infra is necessary to handle the load.
Hi @glarrain-cdd, this is indeed going to be a big migration. If you’re going to use the Airtable import tool, then you might run into a timeout as it will take too long to import. We can help you with instructions on how to perform the import manually via the CLI without that timeout. How are you self-hosting specifically? I’m also curious if you have uploaded files Airtable?
The Airtable import currently imports tables, rows, and most of the field types. It will not include your views, filters, sorts, etc yet. It’s purely about getting the data in. Because the Airtable formula language is incompatible with Baserow’s, those also won’t be imported. We’ve heard that other users successfully experimented with generative AI to convert them from Airtable to Baserow. What I recommend trying is our newest “AI formula generator” tool. You can try to prompt that it’s an Airtable formula that must be converted to Baserow.
When you’re going to run your database with that much data, it will probably require some tweaking and a bigger setup if you want everything to remain performant. I’m happy to help with that. Do you already have insights in how many users, files, API-requests, etc you’re going to make? This will give me better insights when advising.
Thanks a lot for your response. We are not yet self-hosting. We’ve been discussing the Enterprise plan with the sales team.
We will think a little bit more about the planned migration, although replacing Airtable is not our main use case; it is replacing Google Sheets (boy we have a lot of Google Sheets).
Is there any official information or customers’ cases with really large deployments of baserow? I’m think tens of millions of rows.
That makes sense. I’d love to help you with the Airtable migration when the time is right. You are welcome to reach out to me directly.
There are several deployments with customers that have deployed many number of rows. When you’re thinking of tens of millions of rows, are those spread across different workspaces, databases, and tables? Or would that be in one single table, for example? Some context on this really helps in providing you with examples, and advice in terms of what you would need when self-hosting.
Hey @RichardF, what is specifically not working for you? Does the import time out at some point? If so, then I suspect it’s because it’s taking too long to download the 70GB of files.
In that case, we would need to start your Airtable migration in a manual way, where it doesn’t time out. Would you mind sending an email to me (bram@baserow.io)? There are some additional details that I would need, but it potentially involves privacy sensitive data. Then we’ll take it from there.