Self-hosted Airtable Import KeyError(18)

Technical Help
, ,
1

Technical Help Questionnaire

Have you read and followed the instructions at: *READ ME FIRST* Technical Help FAQs - #2 by nigel ?

Answer: Sure did.

Self-Hosted Installation and Setup Questions

How have you self-hosted Baserow.

Docker Compose

services:
  baserow:
    container_name: baserow
    image: baserow/baserow:latest
    environment:
      BASEROW_PUBLIC_URL: 'https://MY_URL'
      WEB_FRONTEND_PORT: 3333
      WEB_FRONTEND_SSL_PORT: 3334
    ports:
      - "3333:80"
      - "3334:443"
    volumes:
      - baserow_data:/baserow/data
volumes:
  baserow_data:

Which version of Baserow are you using.

1.16.0

Describe the problem

I’m unable to import any Airtable bases into my self-hosted Baserow instance. The same bases import just fine to an account on baserow.io, but not my local version. The import process crashes at 42% and I just get the “Something went wrong” message. Logs below.

The bases in question are ~2,500 rows, ~5 tables, ~20 columns so nothing very large.

Please attach full logs from all of Baserow’s services

baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42] [2023-04-19 19:12:34,063: INFO/MainProcess] Task baserow.core.jobs.tasks.run_async_job[e303c9d5-faf7-478b-bf62-aedf29a4109d] received
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42] [2023-04-19 19:12:42,131: ERROR/ForkPoolWorker-1] Task baserow.core.jobs.tasks.run_async_job[e303c9d5-faf7-478b-bf62-aedf29a4109d] raised unexpected: KeyError(18)
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42] Traceback (most recent call last):
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/venv/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     R = retval = fun(*args, **kwargs)
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/venv/lib/python3.9/site-packages/celery/app/trace.py", line 734, in __protected_call__
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     return self.run(*args, **kwargs)
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/core/jobs/tasks.py", line 34, in run_async_job
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     JobHandler().run(job)
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/core/jobs/handler.py", line 59, in run
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     return job_type.run(job, progress)
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/airtable/job_types.py", line 116, in run
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     database = action_type_registry.get(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/airtable/actions.py", line 55, in do
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     database = AirtableHandler.import_from_airtable_to_workspace(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/airtable/handler.py", line 596, in import_from_airtable_to_workspace
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     databases, _ = CoreHandler().import_applications_to_workspace(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 37, in _wrapper
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     raise ex
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 33, in _wrapper
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     result = wrapped_func(*args, **kwargs)
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/core/handler.py", line 1550, in import_applications_to_workspace
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     imported_application = application_type.import_serialized(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/application_types.py", line 459, in import_serialized
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     self.import_tables_serialized(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/application_types.py", line 303, in import_tables_serialized
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     view_type.import_serialized(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/views/view_types.py", line 132, in import_serialized
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     grid_view = super().import_serialized(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/backend/src/baserow/contrib/database/views/registries.py", line 390, in import_serialized
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     value_provider_type.set_import_serialized_value(
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]   File "/baserow/premium/backend/src/baserow_premium/views/decorator_value_provider_types.py", line 133, in set_import_serialized_value
baserow    |  [EXPORT_WORKER][2023-04-19 19:12:42]     new_field_id = id_mapping["database_fields"][filter["field"]]
2

Hi @Manny, firstly, my apologies for the delayed response. It’s strange that the imports work fine on baserow.io, but not in your local environment. I’ve double-checked, and we haven’t introduced changes to the Airtable import on baserow.io since the release of 1.16. Is it a possibility that you share the publicly shared Airtable base with so that I can try to reproduce the problem myself? This will give me more insights in what’s going wrong.

3

Sure thing. Here’s a link to one of the bases. I just re-tested it to make sure I got the same error and it didn’t stop making that strange noise as soon as I got to the mechanic’s.

I’ve also restarted the Docker container, as well as the machine itself a few times to no avail.

4

Hi @Manny, thank you for sharing your base. The good news is that I’ve been able to reproduce your problem. This somehow only seems to happen if you run the Docker all-in-one image. If you run the normal development environment, even on version 1.16.0, it does work as expected. We’re going to look into this.

5

After some debugging, we learned that it’s related to a bug in our code. We’ve already fixed it (Fix wrong empty grid view during airtable import bug (!1413) · Merge requests · Baserow / baserow · GitLab), and it will be released in version 1.17 upcoming Tuesday, May 16. I’m sorry about the inconvenience.

6

Oh, awesome! Thank you for hunting this down. I’ll look forward to the update.

7

Just closing the loop here to confirm that this indeed fixed all the issues I was having. Now everything imports perfectly. Thanks again for the fix.