But as I told you above, all tables and fields were imported from Airtable. After imported, I just removed all rows from tables.
So all tables, fields and structures are very correct.
Sometimes you might see action_type=delete_row error in my logs, but please ignore those because I ran delete rows api as well.
Hi,
I have Country table in airtable and it has 192 records.
So, I created Country table and fields fresh from scratch in baserow like airtable, and I tried to move all of them to baserow using baserow api.
BUT!!!
Only 189 rows were moved, so I traced an error. Meaning, baserow api can’t handle even 192 records.
This makes me crazy. Please help me asap.
[BACKEND][2024-02-28 15:02:06] 407|2024-02-28 15:02:06.673|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=96 action_type=create_row user=2
[BACKEND][2024-02-28 15:02:06] 407|2024-02-28 15:02:06.694|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=96 action_type=create_row user=2
[BACKEND][2024-02-28 15:02:06] ERROR 2024-02-28 15:02:06,691 django.request.log_response:241- Internal Server Error: /api/database/rows/table/533/
[BACKEND][2024-02-28 15:02:06] Traceback (most recent call last):
[BACKEND][2024-02-28 15:02:06] File "/baserow/venv/lib/python3.9/site-packages/django/db/models/options.py", line 669, in get_field
[BACKEND][2024-02-28 15:02:06] return self.fields_map[field_name]
[BACKEND][2024-02-28 15:02:06] KeyError: None
[BACKEND][2024-02-28 15:02:06]
[BACKEND][2024-02-28 15:02:06] During handling of the above exception, another exception occurred:
[BACKEND][2024-02-28 15:02:06]
[BACKEND][2024-02-28 15:02:06] Traceback (most recent call last):
[BACKEND][2024-02-28 15:02:06] File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler
[BACKEND][2024-02-28 15:02:06] raise exc_info[1]
[BACKEND][2024-02-28 15:02:06] File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 43, in inner
[BACKEND][2024-02-28 15:02:06] response = await get_response(request)
[BACKEND][2024-02-28 15:02:06] File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
[BACKEND][2024-02-28 15:02:06] response = await wrapped_callback(
[BACKEND][2024-02-28 15:02:06] File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__
[BACKEND][2024-02-28 15:02:06] ret = await asyncio.wait_for(future, timeout=None)
[BACKEND][2024-02-28 15:02:06] File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
[BACKEND][2024-02-28 15:02:06] return await fut
[BACKEND][2024-02-28 15:02:06] File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run
[BACKEND][2024-02-28 15:02:06] result = self.fn(*self.args, **self.kwargs)
I deleted all rows from Country table in baserow, and retried to import using baserow api.
Airtable has 192 records in Country table so now 191 rows were moved and 1 row with 500 error.
Please help me, this is urgent and super super ULTRA CRITICAL problem for me.
Thanks.
[BACKEND][2024-02-28 15:52:06] 49.13.137.191:0 - "POST /api/database/rows/table/533/?user_field_names=true HTTP/1.1" 200
[BACKEND][2024-02-28 15:52:06] 405|2024-02-28 15:52:06.077|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=96 action_type=create_row user=2
[BACKEND][2024-02-28 15:52:06] 406|2024-02-28 15:52:06.079|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=96 action_type=create_row user=2
[BACKEND][2024-02-28 15:52:06] 406|2024-02-28 15:52:06.083|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=96 action_type=create_row user=2
[BACKEND][2024-02-28 15:52:06] ERROR 2024-02-28 15:52:06,090 django.request.log_response:241- Internal Server Error: /api/database/rows/table/533/
[BACKEND][2024-02-28 15:52:06] Traceback (most recent call last):
[BACKEND][2024-02-28 15:52:06] File "/baserow/venv/lib/python3.9/site-packages/django/db/models/options.py", line 669, in get_field
[BACKEND][2024-02-28 15:52:06] return self.fields_map[field_name]
[BACKEND][2024-02-28 15:52:06] KeyError: None
[BACKEND][2024-02-28 15:52:06]
[BACKEND][2024-02-28 15:52:06] During handling of the above exception, another exception occurred:
[BACKEND][2024-02-28 15:52:06]
[BACKEND][2024-02-28 15:52:06] Traceback (most recent call last):
[BACKEND][2024-02-28 15:52:06] File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler
[BACKEND][2024-02-28 15:52:06] raise exc_info[1]
[BACKEND][2024-02-28 15:52:06] File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 43, in inner
[BACKEND][2024-02-28 15:52:06] response = await get_response(request)
[BACKEND][2024-02-28 15:52:06] File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
[BACKEND][2024-02-28 15:52:06] response = await wrapped_callback(
[BACKEND][2024-02-28 15:52:06] File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__
[BACKEND][2024-02-28 15:52:06] ret = await asyncio.wait_for(future, timeout=None)
[BACKEND][2024-02-28 15:52:06] File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for
[BACKEND][2024-02-28 15:52:06] return await fut
Hello @bram
Please see this post.
This guy is saying
" When field “w_4” is in a form, and referenced by “w4”, and that form is submitted, I get the error.
If I remove the reference to “w_4” from “w4”, I can submit the form without error."
Do I have to remove underline from link or lookup field name?
This is non-sense.
Hello @bram
I realized, Linked or Lookup field is critical problem.
This is Country table, it has Clients Link field. Clients table is available with no rows.
I tested 3 times with Clients Link field and without it.
Before removing Clients field, I keep getting 500 internal error randomly. Sometimes 189 records imported, 170 or etc…
But after removing Clients field, all records were imported successfully from airtable without any errors.
Please help me how to resolve this problem. I MUST use Link and Lookup fields.
Thanks.
I am crazy with Link and Lookup fields, so I decied to share all my credentials with you in private.
My Laravel script to move data from airtable to baserow, and server credential that baserow app installed.
Hi @Long, thanks for sharing the output logs, this is useful for us.
I just wanted to let you know that we’re looking into the problem. Because you’re getting random field errors, I suspect this might be related to a bug we identified recently. It’s fixed, but not yet released. The code changes can be found here: Don't clear Django global apps cache (!2105) · Merge requests · Baserow / baserow · GitLab. If that’s the case, then you would need to wait for the release of next week to fix it.
Does your queue make multiple API requests concurrently? It would also be great if you can share how many API requests you were making. A number of estimated API requests per second would be useful to know.
Would you be able to try to use our hosted version at https://baserow.io, and see if the same problem occurs there as well?
CPU: 8 core
RAM: 15.2G
That should definitely be enough resources.
I am feeling this is public thread, so I am afraid to share my server credential here, right?
No need to share your actual credentials. This is already enough information for me on that front.
This guy is saying
" When field “w_4” is in a form, and referenced by “w4”, and that form is submitted, I get the error.
If I remove the reference to “w_4” from “w4”, I can submit the form without error."
Good catch on finding the error above. However, this specific problem was already fixed a while ago.
Does your queue make multiple API requests concurrently? It would also be great if you can share how many API requests you were making. A number of estimated API requests per second would be useful to know.
I tested with 10861 records from airtable, so sometimes, 12, 20, or 1860 requests at the same time after running the import script.
Is this problem?
If baserow app can’t handle 1000+ requests importing concurrently, how can I import tens of records from airtable?
But what I know is , that the most critical problem is that Link and Lookup field has a critical problem.
Even if I try to create a new row one by one without using job queue, I got 500 internal server error randomly.
It doesn’t make a lot of requests concurrently. It makes 1 - 5 requests per second or less.
So, I can say the most critical problem is that Link and Lookup field has a critical problem.
Hi @Long, concurrent requests are being made at the same time, so not sequential. If you’re making 1 to 5 requests per second, then that should definitely not be a problem. One final question. If you get a 500-internal error, and you retry the same request, do you keep getting a 500 error for that request, or does it eventually work?
