Deadlock detected

[Alex-Clapp]

Are you using our SaaS platform (Baserow.io) or self-hosting Baserow?

Self-hosted

If you are self-hosting, what version of Baserow are you running?

Self hosting 1.30.1

What are the exact steps to reproduce this issue?

When it happens it seems the base is lock: I can’t add a delete row, webhook are not emitted, I can’t publish the app.

I don’t know exactly how to reproduce it, it seems to happen when a row or several rows are created.

Some context

• I have a docker for baserow (8gb) and a separate cloud database hosted on OVH.
• The BASEROW_AMOUNT_OF_GUNICORN_WORKERS environment variable is set to 9
• Some tables have webhook for “all event”. These webhooks are linked to automations that run when rows are created or udated.

It’s verry problematic because baserow is almost unusable when it happend.

Does anyone have an idea about how to solve that ?

Thanks,

Here is the main error

[BACKEND][2025-03-11 08:35:49] ERROR 2025-03-11 08:35:49,095 django.request.log_response:241- Internal Server Error: /api/database/rows/table/38/   
 [BACKEND][2025-03-11 08:35:49] Traceback (most recent call last):  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [BACKEND][2025-03-11 08:35:49]     return self.cursor.execute(sql, params)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49] psycopg2.errors.DeadlockDetected: deadlock detected  
 [BACKEND][2025-03-11 08:35:49] DETAIL:  Process 2539809 waits for ShareLock on transaction 34144581; blocked by process 2530919.  
 [BACKEND][2025-03-11 08:35:49] Process 2530919 waits for ShareLock on transaction 34144557; blocked by process 2539809.  
 [BACKEND][2025-03-11 08:35:49] HINT:  See server log for query details.  
 [BACKEND][2025-03-11 08:35:49] CONTEXT:  while updating tuple (697,2) in relation "database_table_38"  
 [BACKEND][2025-03-11 08:35:49]   
 [BACKEND][2025-03-11 08:35:49]   
 [BACKEND][2025-03-11 08:35:49] The above exception was the direct cause of the following exception:  
 [BACKEND][2025-03-11 08:35:49]   
 [BACKEND][2025-03-11 08:35:49] Traceback (most recent call last):  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 518, in thread_handler  
 [BACKEND][2025-03-11 08:35:49]     raise exc_info[1]  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 42, in inner  
 [BACKEND][2025-03-11 08:35:49]     response = await get_response(request)  
 [BACKEND][2025-03-11 08:35:49]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 518, in thread_handler  
 [BACKEND][2025-03-11 08:35:49]     raise exc_info[1]  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 253, in _get_response_async  
 [BACKEND][2025-03-11 08:35:49]     response = await wrapped_callback(  
 [BACKEND][2025-03-11 08:35:49]                ^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 468, in __call__  
 [BACKEND][2025-03-11 08:35:49]     ret = await asyncio.shield(exec_coro)  
 [BACKEND][2025-03-11 08:35:49]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/current_thread_executor.py", line 40, in run  
 [BACKEND][2025-03-11 08:35:49]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2025-03-11 08:35:49]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 522, in thread_handler  
 [BACKEND][2025-03-11 08:35:49]     return func(*args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper  
 [BACKEND][2025-03-11 08:35:49]     return view_func(request, *args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view  
 [BACKEND][2025-03-11 08:35:49]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2025-03-11 08:35:49]     response = self.handle_exception(exc)  
 [BACKEND][2025-03-11 08:35:49]                ^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2025-03-11 08:35:49]     self.raise_uncaught_exception(exc)  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2025-03-11 08:35:49]     raise exc  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2025-03-11 08:35:49]     response = handler(request, *args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/usr/lib/python3.11/contextlib.py", line 81, in inner  
 [BACKEND][2025-03-11 08:35:49]     return func(*args, **kwds)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/api/decorators.py", line 113, in func_wrapper  
 [BACKEND][2025-03-11 08:35:49]     return func(*args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/api/decorators.py", line 178, in func_wrapper  
 [BACKEND][2025-03-11 08:35:49]     return func(*args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/api/rows/views.py", line 545, in post  
 [BACKEND][2025-03-11 08:35:49]     row = action_type_registry.get_by_type(CreateRowActionType).do(  
 [BACKEND][2025-03-11 08:35:49]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/rows/actions.py", line 90, in do  
 [BACKEND][2025-03-11 08:35:49]     row = RowHandler().create_row(  
 [BACKEND][2025-03-11 08:35:49]           ^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2025-03-11 08:35:49]     raise ex  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2025-03-11 08:35:49]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 704, in create_row  
 [BACKEND][2025-03-11 08:35:49]     return self.force_create_row(  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2025-03-11 08:35:49]     raise ex  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2025-03-11 08:35:49]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 794, in force_create_row  
 [BACKEND][2025-03-11 08:35:49]     fields, dependant_fields = self.update_dependencies_of_rows_created(  
 [BACKEND][2025-03-11 08:35:49]                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2025-03-11 08:35:49]     raise ex  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2025-03-11 08:35:49]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2025-03-11 08:35:49]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 1340, in update_dependencies_of_rows_created  
 [BACKEND][2025-03-11 08:35:49]     update_collector.apply_updates_and_get_updated_fields(field_cache)  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 415, in apply_updates_and_get_updated_fields  
 [BACKEND][2025-03-11 08:35:49]     updated_rows_count = self.apply_updates(field_cache)  
 [BACKEND][2025-03-11 08:35:49]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 399, in apply_updates  
 [BACKEND][2025-03-11 08:35:49]     updated_rows_count = self._update_statement_collector.execute_all(  
 [BACKEND][2025-03-11 08:35:49]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 147, in execute_all  
 [BACKEND][2025-03-11 08:35:49]     updated_rows += sub_path.execute_all(  
 [BACKEND][2025-03-11 08:35:49]                     ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 147, in execute_all  
 [BACKEND][2025-03-11 08:35:49]     updated_rows += sub_path.execute_all(  
 [BACKEND][2025-03-11 08:35:49]                     ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 139, in execute_all  
 [BACKEND][2025-03-11 08:35:49]     updated_rows += self._execute_pending_update_statements(  
 [BACKEND][2025-03-11 08:35:49]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 217, in _execute_pending_update_statements  
 [BACKEND][2025-03-11 08:35:49]     qs.annotate(**annotations)  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [BACKEND][2025-03-11 08:35:49]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [BACKEND][2025-03-11 08:35:49]     cursor = super().execute_sql(result_type)  
 [BACKEND][2025-03-11 08:35:49]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [BACKEND][2025-03-11 08:35:49]     cursor.execute(sql, params)  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [BACKEND][2025-03-11 08:35:49]     return self._execute_with_wrappers(  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [BACKEND][2025-03-11 08:35:49]     return executor(sql, params, many, context)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [BACKEND][2025-03-11 08:35:49]     with self.db.wrap_database_errors:  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [BACKEND][2025-03-11 08:35:49]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2025-03-11 08:35:49]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [BACKEND][2025-03-11 08:35:49]     return self.cursor.execute(sql, params)  
 [BACKEND][2025-03-11 08:35:49]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-11 08:35:49] django.db.utils.OperationalError: deadlock detected  
 [BACKEND][2025-03-11 08:35:49] DETAIL:  Process 2539809 waits for ShareLock on transaction 34144581; blocked by process 2530919.  
 [BACKEND][2025-03-11 08:35:49] Process 2530919 waits for ShareLock on transaction 34144557; blocked by process 2539809.  
 [BACKEND][2025-03-11 08:35:49] HINT:  See server log for query details.  
 [BACKEND][2025-03-11 08:35:49] CONTEXT:  while updating tuple (697,2) in relation "database_table_38"  
 [BACKEND][2025-03-11 08:35:49]   
 [BACKEND][2025-03-11 08:35:51] 34.96.44.77:0 - "POST /api/database/rows/table/38/?user_field_names=true HTTP/1.1" 500  
 [BACKEND][2025-03-11 08:35:54] 92.169.248.58:0 - "GET /api/jobs/542/ HTTP/1.1" 200  
 [BACKEND][2025-03-11 08:35:56] 92.169.248.58:0 - "GET /api/jobs/542/ HTTP/1.1" 200  

[olgatrykush]

Hi @Alex-Clapp, could you please upgrade to version 1.31 and verify if the issue persists?

[Alex-Clapp]

@olgatrykush

Hi @Alex-Clapp, could you please upgrade to version 1.31 and verify if the issue persists?

Hello Olga, Sure, I’ll do that and let you know if it persists

[Alex-Clapp]

Hello Olga,

I’ve updated baserow but the error is still there.

[EXPORT_WORKER][2025-03-17 08:21:02] DETAIL:  Process 940598 waits for ShareLock on transaction 34247748; blocked by process 940591.  
 [EXPORT_WORKER][2025-03-17 08:21:02] Process 940591 waits for ShareLock on transaction 34247749; blocked by process 940598.  
 [EXPORT_WORKER][2025-03-17 08:21:02] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-03-17 08:21:02] CONTEXT:  while updating tuple (210,2) in relation "database_table_38"  
 [BACKEND][2025-03-17 08:21:03] 143.110.174.228:0 - "PATCH /api/database/rows/table/38/3149/?user_field_names=true HTTP/1.1" 200  
 [BACKEND][2025-03-17 08:21:03] ERROR 2025-03-17 08:21:03,416 django.request.log_response:241- Internal Server Error: /api/database/rows/table/37/1229/   
 [BACKEND][2025-03-17 08:21:03] Traceback (most recent call last):  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [BACKEND][2025-03-17 08:21:03]     return self.cursor.execute(sql, params)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03] psycopg2.errors.DeadlockDetected: deadlock detected  
 [BACKEND][2025-03-17 08:21:03] DETAIL:  Process 940591 waits for ShareLock on transaction 34247751; blocked by process 940598.  
 [BACKEND][2025-03-17 08:21:03] Process 940598 waits for ShareLock on transaction 34247748; blocked by process 940591.  
 [BACKEND][2025-03-17 08:21:03] HINT:  See server log for query details.  
 [BACKEND][2025-03-17 08:21:03] CONTEXT:  while updating tuple (65,7) in relation "database_table_38"  
 [BACKEND][2025-03-17 08:21:03]   
 [BACKEND][2025-03-17 08:21:03]   
 [BACKEND][2025-03-17 08:21:03] The above exception was the direct cause of the following exception:  
 [BACKEND][2025-03-17 08:21:03]   
 [BACKEND][2025-03-17 08:21:03] Traceback (most recent call last):  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 518, in thread_handler  
 [BACKEND][2025-03-17 08:21:03]     raise exc_info[1]  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 42, in inner  
 [BACKEND][2025-03-17 08:21:03]     response = await get_response(request)  
 [BACKEND][2025-03-17 08:21:03]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 518, in thread_handler  
 [BACKEND][2025-03-17 08:21:03]     raise exc_info[1]  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 253, in _get_response_async  
 [BACKEND][2025-03-17 08:21:03]     response = await wrapped_callback(  
 [BACKEND][2025-03-17 08:21:03]                ^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 468, in __call__  
 [BACKEND][2025-03-17 08:21:03]     ret = await asyncio.shield(exec_coro)  
 [BACKEND][2025-03-17 08:21:03]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/current_thread_executor.py", line 40, in run  
 [BACKEND][2025-03-17 08:21:03]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2025-03-17 08:21:03]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 522, in thread_handler  
 [BACKEND][2025-03-17 08:21:03]     return func(*args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 65, in _view_wrapper  
 [BACKEND][2025-03-17 08:21:03]     return view_func(request, *args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 104, in view  
 [BACKEND][2025-03-17 08:21:03]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2025-03-17 08:21:03]     response = self.handle_exception(exc)  
 [BACKEND][2025-03-17 08:21:03]                ^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2025-03-17 08:21:03]     self.raise_uncaught_exception(exc)  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2025-03-17 08:21:03]     raise exc  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2025-03-17 08:21:03]     response = handler(request, *args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/usr/lib/python3.11/contextlib.py", line 81, in inner  
 [BACKEND][2025-03-17 08:21:03]     return func(*args, **kwds)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/api/decorators.py", line 113, in func_wrapper  
 [BACKEND][2025-03-17 08:21:03]     return func(*args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/api/decorators.py", line 439, in _wrap  
 [BACKEND][2025-03-17 08:21:03]     return f(_self, request, *args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/api/rows/views.py", line 909, in patch  
 [BACKEND][2025-03-17 08:21:03]     row = action_type_registry.get_by_type(UpdateRowsActionType).do(  
 [BACKEND][2025-03-17 08:21:03]           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/rows/actions.py", line 813, in do  
 [BACKEND][2025-03-17 08:21:03]     result = row_handler.update_rows(  
 [BACKEND][2025-03-17 08:21:03]              ^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 73, in _wrapper  
 [BACKEND][2025-03-17 08:21:03]     raise ex  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 69, in _wrapper  
 [BACKEND][2025-03-17 08:21:03]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 1969, in update_rows  
 [BACKEND][2025-03-17 08:21:03]     return self.force_update_rows(  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 73, in _wrapper  
 [BACKEND][2025-03-17 08:21:03]     raise ex  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 69, in _wrapper  
 [BACKEND][2025-03-17 08:21:03]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 1888, in force_update_rows  
 [BACKEND][2025-03-17 08:21:03]     dependant_fields = self.update_dependencies_of_rows_updated(  
 [BACKEND][2025-03-17 08:21:03]                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 73, in _wrapper  
 [BACKEND][2025-03-17 08:21:03]     raise ex  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 69, in _wrapper  
 [BACKEND][2025-03-17 08:21:03]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2025-03-17 08:21:03]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 1071, in update_dependencies_of_rows_updated  
 [BACKEND][2025-03-17 08:21:03]     update_collector.apply_updates_and_get_updated_fields(  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 447, in apply_updates_and_get_updated_fields  
 [BACKEND][2025-03-17 08:21:03]     updated_rows_count = self.apply_updates(field_cache)  
 [BACKEND][2025-03-17 08:21:03]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 410, in apply_updates  
 [BACKEND][2025-03-17 08:21:03]     updated_rows_count = self._update_statement_collector.execute_all(  
 [BACKEND][2025-03-17 08:21:03]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 147, in execute_all  
 [BACKEND][2025-03-17 08:21:03]     updated_rows += sub_path.execute_all(  
 [BACKEND][2025-03-17 08:21:03]                     ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 147, in execute_all  
 [BACKEND][2025-03-17 08:21:03]     updated_rows += sub_path.execute_all(  
 [BACKEND][2025-03-17 08:21:03]                     ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 147, in execute_all  
 [BACKEND][2025-03-17 08:21:03]     updated_rows += sub_path.execute_all(  
 [BACKEND][2025-03-17 08:21:03]                     ^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 139, in execute_all  
 [BACKEND][2025-03-17 08:21:03]     updated_rows += self._execute_pending_update_statements(  
 [BACKEND][2025-03-17 08:21:03]                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/backend/src/baserow/contrib/database/fields/dependencies/update_collector.py", line 217, in _execute_pending_update_statements  
 [BACKEND][2025-03-17 08:21:03]     qs.annotate(**annotations)  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [BACKEND][2025-03-17 08:21:03]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [BACKEND][2025-03-17 08:21:03]     cursor = super().execute_sql(result_type)  
 [BACKEND][2025-03-17 08:21:03]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [BACKEND][2025-03-17 08:21:03]     cursor.execute(sql, params)  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [BACKEND][2025-03-17 08:21:03]     return self._execute_with_wrappers(  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [BACKEND][2025-03-17 08:21:03]     return executor(sql, params, many, context)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [BACKEND][2025-03-17 08:21:03]     with self.db.wrap_database_errors:  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [BACKEND][2025-03-17 08:21:03]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2025-03-17 08:21:03]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [BACKEND][2025-03-17 08:21:03]     return self.cursor.execute(sql, params)  
 [BACKEND][2025-03-17 08:21:03]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [BACKEND][2025-03-17 08:21:03] django.db.utils.OperationalError: deadlock detected  
 [BACKEND][2025-03-17 08:21:03] DETAIL:  Process 940591 waits for ShareLock on transaction 34247751; blocked by process 940598.  
 [BACKEND][2025-03-17 08:21:03] Process 940598 waits for ShareLock on transaction 34247748; blocked by process 940591.  
 [BACKEND][2025-03-17 08:21:03] HINT:  See server log for query details.  
 [BACKEND][2025-03-17 08:21:03] CONTEXT:  while updating tuple (65,7) in relation "database_table_38"  
 [BACKEND][2025-03-17 08:21:03]   
 [BACKEND][2025-03-17 08:21:07] 195.101.82.105:0 - "PATCH /api/database/rows/table/37/1229/ HTTP/1.1" 500  
 [EXPORT_WORKER][2025-03-17 08:21:12] . Attempting to do per field updates one by one instead...  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,571: WARNING/ForkPoolWorker-2] 27957|2025-03-17 08:21:12.570|ERROR|baserow.contrib.database.search.handler:try_slower_but_best_effort_tsv_update:690 - Failed to update search vector for field with id 652 / type <class 'baserow.contrib.database.fields.models.FormulaField'> because of deadlock detected  
 [EXPORT_WORKER][2025-03-17 08:21:12] DETAIL:  Process 940598 waits for ShareLock on transaction 34247827; blocked by process 940641.  
 [EXPORT_WORKER][2025-03-17 08:21:12] Process 940641 waits for ShareLock on transaction 34247832; blocked by process 940598.  
 [EXPORT_WORKER][2025-03-17 08:21:12] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-03-17 08:21:12] CONTEXT:  while updating tuple (3192,1) in relation "database_table_38"  
 [EXPORT_WORKER][2025-03-17 08:21:12] , field.__str__ is: FormulaField(  
 [EXPORT_WORKER][2025-03-17 08:21:12] formula=join(field('Article'),''),  
 [EXPORT_WORKER][2025-03-17 08:21:12] internal_formula=error_to_null(join(lookup('field_663','field_576'),'')),  
 [EXPORT_WORKER][2025-03-17 08:21:12] formula_type=text,  
 [EXPORT_WORKER][2025-03-17 08:21:12] error=None,  
 [EXPORT_WORKER][2025-03-17 08:21:12] ) and expression is LocalisedSearchVector(Func(Cast(F(field_652)), Value('[@:/+.]|-(?!\\d)|(?<=\\S)-(?=\\d)|\\b-\\b'), Value(' '), Value('g'), function=regexp_replace))  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,575: WARNING/ForkPoolWorker-2] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,576: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,576: WARNING/ForkPoolWorker-2] psycopg2.errors.DeadlockDetected: deadlock detected  
 [EXPORT_WORKER][2025-03-17 08:21:12] DETAIL:  Process 940598 waits for ShareLock on transaction 34247748; blocked by process 940591.  
 [EXPORT_WORKER][2025-03-17 08:21:12] Process 940591 waits for ShareLock on transaction 34247749; blocked by process 940598.  
 [EXPORT_WORKER][2025-03-17 08:21:12] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-03-17 08:21:12] CONTEXT:  while updating tuple (210,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,576: WARNING/ForkPoolWorker-2]   
 [EXPORT_WORKER][2025-03-17 08:21:12] The above exception was the direct cause of the following exception:  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,576: WARNING/ForkPoolWorker-2] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,577: WARNING/ForkPoolWorker-2]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 604, in run_tsvector_update_statement  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return cls.split_update_into_chunks_until_all_background_done(  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,577: WARNING/ForkPoolWorker-2]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 580, in split_update_into_chunks_until_all_background_done  
 [EXPORT_WORKER][2025-03-17 08:21:12]     this_chunk_updated = next_chunk.update(**update_query)  
 [EXPORT_WORKER][2025-03-17 08:21:12]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,577: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [EXPORT_WORKER][2025-03-17 08:21:12]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,577: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [EXPORT_WORKER][2025-03-17 08:21:12]     cursor = super().execute_sql(result_type)  
 [EXPORT_WORKER][2025-03-17 08:21:12]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,577: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [EXPORT_WORKER][2025-03-17 08:21:12]     cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,578: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return self._execute_with_wrappers(  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,578: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return executor(sql, params, many, context)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,578: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     with self.db.wrap_database_errors:  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,578: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [EXPORT_WORKER][2025-03-17 08:21:12]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,578: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,579: WARNING/ForkPoolWorker-2] django.db.utils.OperationalError: deadlock detected  
 [EXPORT_WORKER][2025-03-17 08:21:12] DETAIL:  Process 940598 waits for ShareLock on transaction 34247748; blocked by process 940591.  
 [EXPORT_WORKER][2025-03-17 08:21:12] Process 940591 waits for ShareLock on transaction 34247749; blocked by process 940598.  
 [EXPORT_WORKER][2025-03-17 08:21:12] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-03-17 08:21:12] CONTEXT:  while updating tuple (210,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,579: WARNING/ForkPoolWorker-2]   
 [EXPORT_WORKER][2025-03-17 08:21:12] During handling of the above exception, another exception occurred:  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,579: WARNING/ForkPoolWorker-2] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,579: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,580: WARNING/ForkPoolWorker-2] psycopg2.errors.DeadlockDetected: deadlock detected  
 [EXPORT_WORKER][2025-03-17 08:21:12] DETAIL:  Process 940598 waits for ShareLock on transaction 34247827; blocked by process 940641.  
 [EXPORT_WORKER][2025-03-17 08:21:12] Process 940641 waits for ShareLock on transaction 34247832; blocked by process 940598.  
 [EXPORT_WORKER][2025-03-17 08:21:12] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-03-17 08:21:12] CONTEXT:  while updating tuple (3192,1) in relation "database_table_38"  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,580: WARNING/ForkPoolWorker-2]   
 [EXPORT_WORKER][2025-03-17 08:21:12] The above exception was the direct cause of the following exception:  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,580: WARNING/ForkPoolWorker-2] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,581: WARNING/ForkPoolWorker-2]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 672, in try_slower_but_best_effort_tsv_update  
 [EXPORT_WORKER][2025-03-17 08:21:12]     cls.split_update_into_chunks_until_all_background_done(  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,581: WARNING/ForkPoolWorker-2]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 580, in split_update_into_chunks_until_all_background_done  
 [EXPORT_WORKER][2025-03-17 08:21:12]     this_chunk_updated = next_chunk.update(**update_query)  
 [EXPORT_WORKER][2025-03-17 08:21:12]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,582: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [EXPORT_WORKER][2025-03-17 08:21:12]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,582: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [EXPORT_WORKER][2025-03-17 08:21:12]     cursor = super().execute_sql(result_type)  
 [EXPORT_WORKER][2025-03-17 08:21:12]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,582: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [EXPORT_WORKER][2025-03-17 08:21:12]     cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,583: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return self._execute_with_wrappers(  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,583: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return executor(sql, params, many, context)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,583: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     with self.db.wrap_database_errors:  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,584: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [EXPORT_WORKER][2025-03-17 08:21:12]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,584: WARNING/ForkPoolWorker-2]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-03-17 08:21:12]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-03-17 08:21:12]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-03-17 08:21:12] [2025-03-17 08:21:12,584: WARNING/ForkPoolWorker-2] django.db.utils.OperationalError: deadlock detected  
 [EXPORT_WORKER][2025-03-17 08:21:12] DETAIL:  Process 940598 waits for ShareLock on transaction 34247827; blocked by process 940641.  
 [EXPORT_WORKER][2025-03-17 08:21:12] Process 940641 waits for ShareLock on transaction 34247832; blocked by process 940598.  
 [EXPORT_WORKER][2025-03-17 08:21:12] HINT:  See server log for query details.  
 [BACKEND][2025-03-17 08:21:13] 195.101.82.105:0 - "GET /api/row_comments/37/1229/?offset=0&limit=50 HTTP/1.1" 200  
 [CELERY_WORKER][2025-03-17 08:21:13] [2025-03-17 08:21:00,142: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[fbf4b9c1-f65a-4161-a1ca-e7d9cc01d90a] succeeded in 0.003354310989379883s: None  
 [CELERY_WORKER][2025-03-17 08:21:13] [2025-03-17 08:21:13,131: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[b6ef2997-1e20-47de-89d6-ef2e60d8ae0c] received  
 [CELERY_WORKER][2025-03-17 08:21:13] [2025-03-17 08:21:13,138: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[b6ef2997-1e20-47de-89d6-ef2e60d8ae0c] succeeded in 0.004915378987789154s: None  
 [BACKEND][2025-03-17 08:21:13] 321|2025-03-17 08:21:13.051|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=1 action_type=update_rows user=9  
 [CELERY_WORKER][2025-03-17 08:21:13] [2025-03-17 08:21:13,186: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[987f1e06-d741-4e3e-a678-daeec00388a0] received  
 [CELERY_WORKER][2025-03-17 08:21:13] [2025-03-17 08:21:13,194: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[614b3283-9bff-4c5f-9238-8cef1af05849] received  
 [CELERY_WORKER][2025-03-17 08:21:13] [2025-03-17 08:21:13,198: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[987f1e06-d741-4e3e-a678-daeec00388a0] succeeded in 0.010430015623569489s: None  
 [BACKEND][2025-03-17 08:21:14] 195.101.82.105:0 - "PATCH /api/database/rows/table/37/1229/ HTTP/1.1" 200  
 [BACKEND][2025-03-17 08:21:19] [2025-03-17 08:21:14 +0000] [319] [INFO] connection closed  

Is there a way to know what these tuples are ?

while updating tuple (65,7) in relation "database_table_38"  

or

while updating tuple (3192,1) in relation "database_table_38" 

Thanks

[bram]

Hey @Alex-Clapp, this can happen when multiple processes are trying to update a row at the same time. We’re occasionally seeing them in our production environment as well. If you update a row, then there the following can happen under the hood:

• Update the row itself.
• Update any rows with formula dependencies.
• Update the search index for that row.

If there are many depending on formulas and many API requests are being made, then in some cases, then it can result in that error. Can you give me some more insights in how your database schema looks like, and number of API requests, and how many CPUs and memory your server has?

[Alex-Clapp]

Hello @bram

Thank you for your reply.

The database has around 50 tables with many links and formulas. It’s difficult for me to describe all of it, but I’ll try to explain where the point is, IMO. I can also privately share with you the baserow api documentation that describe it if needed.

Database

The error often happens when we manipulate data with the same schema (mother - children) of data.
For example, we have it with orders and orders line.

The children lines are linked with the “wharehouse” table where we can find all article. Here, stocks are computed with formulas and depend on agency.

Automations

We have automations that runs at line (mother and / or children) creation or update. For example:

• An order is created → update the order number
• Articles are added to the order → create corresponding order lines
• An order is closed → update article prices of all order lines in the warehouse table.
• etc.

Apps

People interact with the baserow front, as well as with the app builder and with an external application that made API requests to fetch information.

Usages

Let’s say ~20 orders are created a day and 10 people use the apps daily. I don’t have the real number, but I would expect few hundreds of API calls daily.

IT

I use the OVH :

• b2-7 instance with 7gm or ram, 2 vCore processor to host the baserow docker, in combination with
• a managed Postgresql database of type Db1-4 composed of 1 node 4go of ram, 2vCoce processor and 80go of storage.

Let me know if you see some improvement that can be made,

Regards,

[bram]

Hey @Alex-Clapp, thank you so much for the detailed response. Your server specifications, number of API requests, users, and database schema seem to be fine. It’s probably a software bug that we need to look into from our side.

It’s probably going to take some time to investigate, so we would need to plan that. We have an issue on the backlog to investigate this problem here Deadlock updating rows (#2969) · Issues · Baserow / baserow · GitLab. There are some other tasks that we need to look into this month, but I’ve planned to look into it in early Q2.

If you encounter the problem when making an API request, then I recommend retrying that specific request, for now. I hope that’s acceptable.

I’ll get back to you as soon as we know more.

[Alex-Clapp]

Hello @bram,

Thank you for your reply.

I have a new case of deadlock which is not linked with API request but with the app builder.

Use case description:
On the app builder, I have a page that shows a list of items, all part of an order. In the database I have a table “Orders” and a table “Items”.
These items can be updated (label and quantity) from the app builder page.
I have a button on each line that trigger the action “update the line” then “refresh data” allowing to update item label and quantity.

When I tried to successively update quantity on serval items, deadlocks appears.

The problem here is not so much that the action does not take place (I can restart it - clicking the button - a few seconds later) but that all the webhooks are blocked for several minutes. Thus blocking all the backend logic.

I saw that this Reduce deadlocks (#3545) · Issues · Baserow / baserow · GitLab seems to be done, and included in the next release, I hope.
But I don’t see anything about the webhook. Do you know how it can be linked ?

Let me know If you have new infos,

Regards,

[Alex-Clapp]

To give more insights:

Here is where the systeme locks

[EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:18,667: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[3abb7f02-aeea-4482-8f64-1aa3b719653b] received  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,180: WARNING/ForkPoolWorker-10] 21027|2025-04-10 13:36:30.179|ERROR|baserow.contrib.database.search.handler:try_slower_but_best_effort_tsv_update:690 - Failed to update search vector for field with id 652 / type <class 'baserow.contrib.database.fields.models.FormulaField'> because of deadlock detected  
 [EXPORT_WORKER][2025-04-10 13:36:30] DETAIL:  Process 552042 waits for ShareLock on transaction 35326866; blocked by process 552100.  
 [EXPORT_WORKER][2025-04-10 13:36:30] Process 552100 waits for ShareLock on transaction 35326871; blocked by process 552042.  
 [EXPORT_WORKER][2025-04-10 13:36:30] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-10 13:36:30] CONTEXT:  while updating tuple (3362,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-10 13:36:30] , field.__str__ is: FormulaField(  
 [EXPORT_WORKER][2025-04-10 13:36:30] formula=join(field('Article'),''),  
 [EXPORT_WORKER][2025-04-10 13:36:30] internal_formula=error_to_null(join(lookup('field_663','field_576'),'')),  
 [EXPORT_WORKER][2025-04-10 13:36:30] formula_type=text,  
 [EXPORT_WORKER][2025-04-10 13:36:30] error=None,  
 [EXPORT_WORKER][2025-04-10 13:36:30] ) and expression is LocalisedSearchVector(Func(Cast(F(field_652)), Value('[@:/+.]|-(?!\\d)|(?<=\\S)-(?=\\d)|\\b-\\b'), Value(' '), Value('g'), function=regexp_replace))  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,186: WARNING/ForkPoolWorker-10] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,186: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,187: WARNING/ForkPoolWorker-10] psycopg2.errors.DeadlockDetected: deadlock detected  
 [EXPORT_WORKER][2025-04-10 13:36:30] DETAIL:  Process 552042 waits for ShareLock on transaction 35326742; blocked by process 552033.  
 [EXPORT_WORKER][2025-04-10 13:36:30] Process 552033 waits for ShareLock on transaction 35326743; blocked by process 552042.  
 [EXPORT_WORKER][2025-04-10 13:36:30] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-10 13:36:30] CONTEXT:  while updating tuple (47,1) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,187: WARNING/ForkPoolWorker-10]   
 [EXPORT_WORKER][2025-04-10 13:36:30] The above exception was the direct cause of the following exception:  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,187: WARNING/ForkPoolWorker-10] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 604, in run_tsvector_update_statement  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return cls.split_update_into_chunks_until_all_background_done(  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 580, in split_update_into_chunks_until_all_background_done  
 [EXPORT_WORKER][2025-04-10 13:36:30]     this_chunk_updated = next_chunk.update(**update_query)  
 [EXPORT_WORKER][2025-04-10 13:36:30]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [EXPORT_WORKER][2025-04-10 13:36:30]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [EXPORT_WORKER][2025-04-10 13:36:30]     cursor = super().execute_sql(result_type)  
 [EXPORT_WORKER][2025-04-10 13:36:30]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [EXPORT_WORKER][2025-04-10 13:36:30]     cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return self._execute_with_wrappers(  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return executor(sql, params, many, context)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,189: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     with self.db.wrap_database_errors:  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [EXPORT_WORKER][2025-04-10 13:36:30]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10] django.db.utils.OperationalError: deadlock detected  
 [EXPORT_WORKER][2025-04-10 13:36:30] DETAIL:  Process 552042 waits for ShareLock on transaction 35326742; blocked by process 552033.  
 [EXPORT_WORKER][2025-04-10 13:36:30] Process 552033 waits for ShareLock on transaction 35326743; blocked by process 552042.  
 [EXPORT_WORKER][2025-04-10 13:36:30] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-10 13:36:30] CONTEXT:  while updating tuple (47,1) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10]   
 [EXPORT_WORKER][2025-04-10 13:36:30] During handling of the above exception, another exception occurred:  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,190: WARNING/ForkPoolWorker-10] psycopg2.errors.DeadlockDetected: deadlock detected  
 [EXPORT_WORKER][2025-04-10 13:36:30] DETAIL:  Process 552042 waits for ShareLock on transaction 35326866; blocked by process 552100.  
 [EXPORT_WORKER][2025-04-10 13:36:30] Process 552100 waits for ShareLock on transaction 35326871; blocked by process 552042.  
 [EXPORT_WORKER][2025-04-10 13:36:30] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-10 13:36:30] CONTEXT:  while updating tuple (3362,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,191: WARNING/ForkPoolWorker-10]   
 [EXPORT_WORKER][2025-04-10 13:36:30] The above exception was the direct cause of the following exception:  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,191: WARNING/ForkPoolWorker-10] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 672, in try_slower_but_best_effort_tsv_update  
 [EXPORT_WORKER][2025-04-10 13:36:30]     cls.split_update_into_chunks_until_all_background_done(  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 580, in split_update_into_chunks_until_all_background_done  
 [EXPORT_WORKER][2025-04-10 13:36:30]     this_chunk_updated = next_chunk.update(**update_query)  
 [EXPORT_WORKER][2025-04-10 13:36:30]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [EXPORT_WORKER][2025-04-10 13:36:30]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [EXPORT_WORKER][2025-04-10 13:36:30]     cursor = super().execute_sql(result_type)  
 [EXPORT_WORKER][2025-04-10 13:36:30]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [EXPORT_WORKER][2025-04-10 13:36:30]     cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return self._execute_with_wrappers(  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return executor(sql, params, many, context)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     with self.db.wrap_database_errors:  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [EXPORT_WORKER][2025-04-10 13:36:30]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,192: WARNING/ForkPoolWorker-10]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-10 13:36:30]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-10 13:36:30]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,193: WARNING/ForkPoolWorker-10] django.db.utils.OperationalError: deadlock detected  
 [EXPORT_WORKER][2025-04-10 13:36:30] DETAIL:  Process 552042 waits for ShareLock on transaction 35326866; blocked by process 552100.  
 [EXPORT_WORKER][2025-04-10 13:36:30] Process 552100 waits for ShareLock on transaction 35326871; blocked by process 552042.  
 [EXPORT_WORKER][2025-04-10 13:36:30] HINT:  See server log for query details.  
 [BACKEND][2025-04-10 13:36:30] 92.169.248.58:0 - "OPTIONS /api/builder/workflow_action/20969/dispatch/ HTTP/1.1" 200  
 [CELERY_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:18,756: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[11156822-d81b-45b7-9c77-290d3adffb7c] succeeded in 0.004044726490974426s: None  
 [CELERY_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,614: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[fda92793-d3e5-4cc1-9fa3-6a010f861107] received  
 [CELERY_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,628: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[fda92793-d3e5-4cc1-9fa3-6a010f861107] succeeded in 0.010705932974815369s: None  
 [CELERY_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,650: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[33159e20-7bef-4f54-b124-00d50dfaaf61] received  
 [CELERY_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,658: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[3643ac4f-7702-419f-a21b-a1d4e1476d66] received  
 [BACKEND][2025-04-10 13:36:30] 323|2025-04-10 13:36:30.529|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=1 action_type=update_rows user=1  
 [CELERY_WORKER][2025-04-10 13:36:30] [2025-04-10 13:36:30,658: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[33159e20-7bef-4f54-b124-00d50dfaaf61] succeeded in 0.0073369890451431274s: None  

[Alex-Clapp]

Here is where it unlocks and release the webhooks (30 min later)

[BACKEND][2025-04-10 14:06:16] 92.169.248.58:0 - "GET /api/database/views/grid/127/aggregations/ HTTP/1.1" 200  
 [EXPORT_WORKER][2025-04-10 14:06:16] CONTEXT:  while updating tuple (3362,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-10 14:06:16] [2025-04-10 14:06:16,613: ERROR/MainProcess] Task handler raised error: TimeLimitExceeded(1800)  
 [EXPORT_WORKER][2025-04-10 14:06:16] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-10 14:06:16]   File "/baserow/venv/lib/python3.11/site-packages/billiard/pool.py", line 683, in on_hard_timeout  
 [EXPORT_WORKER][2025-04-10 14:06:16]     raise TimeLimitExceeded(job._timeout)  
 [EXPORT_WORKER][2025-04-10 14:06:16] billiard.einfo.ExceptionWithTraceback:   
 [EXPORT_WORKER][2025-04-10 14:06:16] """  
 [EXPORT_WORKER][2025-04-10 14:06:16] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-10 14:06:16]   File "/baserow/venv/lib/python3.11/site-packages/billiard/pool.py", line 683, in on_hard_timeout  
 [EXPORT_WORKER][2025-04-10 14:06:16]     raise TimeLimitExceeded(job._timeout)  
 [EXPORT_WORKER][2025-04-10 14:06:16] billiard.exceptions.TimeLimitExceeded: TimeLimitExceeded(1800,)  
 [EXPORT_WORKER][2025-04-10 14:06:16] """  
 [EXPORT_WORKER][2025-04-10 14:06:17] [2025-04-10 14:06:16,614: ERROR/MainProcess] Hard time limit (1800s) exceeded for baserow.contrib.database.search.tasks.async_update_tsvector_columns[7e774e00-220d-4b60-879e-8a9b66915df9]  
 [EXPORT_WORKER][2025-04-10 14:06:17] [2025-04-10 14:06:17,518: ERROR/MainProcess] Process 'ForkPoolWorker-10' pid:21027 exited with 'signal 9 (SIGKILL)'  
 [EXPORT_WORKER][2025-04-10 14:06:17] [2025-04-10 14:06:17,954: WARNING/ForkPoolWorker-11] Not configuring telemetry due to BASEROW_ENABLE_OTEL not being set.  
 [EXPORT_WORKER][2025-04-10 14:06:17] [2025-04-10 14:06:17,967: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:17.967|INFO|baserow.core.telemetry.telemetry:setup_logging:72 - Logger setup.  
 [BACKEND][2025-04-10 14:06:18] 127.0.0.1:48218 - "GET /api/_health/ HTTP/1.1" 200  
 [EXPORT_WORKER][2025-04-10 14:06:18] [2025-04-10 14:06:17,975: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[b1ff1c20-5c13-4397-bee7-d80c30c2031c] received  
 [EXPORT_WORKER][2025-04-10 14:06:18] [2025-04-10 14:06:18,234: INFO/ForkPoolWorker-11] Task baserow.contrib.database.webhooks.tasks.call_webhook[07f99117-1ebb-40ce-a2dc-4922eb71e9a4] succeeded in 0.2635445147752762s: None  
 [EXPORT_WORKER][2025-04-10 14:06:18] [2025-04-10 14:06:18,240: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[fcc876c0-f099-4f52-9db0-d8f1e607592e] received  
 [EXPORT_WORKER][2025-04-10 14:06:18] [2025-04-10 14:06:18,778: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:18.777|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:18] [2025-04-10 14:06:18,780: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0106b907-5ed2-467f-9753-3280a04a6ff8] succeeded in 0.542194589972496s: None  
 [EXPORT_WORKER][2025-04-10 14:06:19] [2025-04-10 14:06:18,788: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[8d02d2b6-0b37-4b59-9a16-d2708ccd7719] received  
 [EXPORT_WORKER][2025-04-10 14:06:19] [2025-04-10 14:06:19,217: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:19.217|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 37's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:19] [2025-04-10 14:06:19,219: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[40178c52-1225-4ede-a46f-d4f61892d395] succeeded in 0.43638066947460175s: None  
 [EXPORT_WORKER][2025-04-10 14:06:19] [2025-04-10 14:06:19,224: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[46f5dcd2-b051-401d-a892-f02fd0d8a580] received  
 [EXPORT_WORKER][2025-04-10 14:06:19] [2025-04-10 14:06:19,436: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:19.435|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 37's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:19] [2025-04-10 14:06:19,438: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[3abb7f02-aeea-4482-8f64-1aa3b719653b] succeeded in 0.2130948305130005s: None  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:19,443: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[1bdcf880-32d9-4c52-8d82-b0a7dc69ef0f] received  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,040: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:20.040|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,042: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[b1ff1c20-5c13-4397-bee7-d80c30c2031c] succeeded in 0.600210577249527s: None  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,049: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[89613c01-6099-4dca-afce-a439300e35f6] received  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,252: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:20.251|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,255: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[fcc876c0-f099-4f52-9db0-d8f1e607592e] succeeded in 0.2095649391412735s: None  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,262: INFO/MainProcess] Task baserow.contrib.database.webhooks.tasks.call_webhook[33da43ef-e58a-4f2a-acfb-df52bffad1eb] received  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,465: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:20.464|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 37's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,467: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[8d02d2b6-0b37-4b59-9a16-d2708ccd7719] succeeded in 0.20847268402576447s: None  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,471: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[c8fa1d69-21a6-4d0a-b8f8-f558e1ca5e06] received  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,655: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:20.654|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,657: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[46f5dcd2-b051-401d-a892-f02fd0d8a580] succeeded in 0.18763506412506104s: None  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,662: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[8df3bb74-a345-4efa-9391-ce7dc9a8b3b3] received  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,886: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:20.886|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:20] [2025-04-10 14:06:20,888: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[1bdcf880-32d9-4c52-8d82-b0a7dc69ef0f] succeeded in 0.22756658494472504s: None  
 [EXPORT_WORKER][2025-04-10 14:06:21] [2025-04-10 14:06:20,892: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[01d86a27-38fa-4ef1-9e70-0cb6edd95f71] received  
 [EXPORT_WORKER][2025-04-10 14:06:21] [2025-04-10 14:06:21,727: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:21.727|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:21] [2025-04-10 14:06:21,729: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[89613c01-6099-4dca-afce-a439300e35f6] succeeded in 0.836350604891777s: None  
 [EXPORT_WORKER][2025-04-10 14:06:21] [2025-04-10 14:06:21,734: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0f853564-d87e-4ba9-bcf9-4795e197c479] received  
 [EXPORT_WORKER][2025-04-10 14:06:21] [2025-04-10 14:06:21,978: INFO/ForkPoolWorker-11] Task baserow.contrib.database.webhooks.tasks.call_webhook[33da43ef-e58a-4f2a-acfb-df52bffad1eb] succeeded in 0.24348723888397217s: None  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:21,982: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[d6f0bebc-042b-4786-9a62-9898dfba0f11] received  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,150: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:22.149|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,153: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[c8fa1d69-21a6-4d0a-b8f8-f558e1ca5e06] succeeded in 0.1724197417497635s: None  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,157: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[12a27b65-90c6-4a43-ab2e-0dc8a2ec156d] received  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,311: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:22.311|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,313: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[8df3bb74-a345-4efa-9391-ce7dc9a8b3b3] succeeded in 0.15734434127807617s: None  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,318: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[1c837055-3024-4d95-8bb0-25c4e5375e0c] received  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,537: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:22.537|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,539: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[01d86a27-38fa-4ef1-9e70-0cb6edd95f71] succeeded in 0.22249141335487366s: None  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,546: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[751b2310-ed30-494b-9636-f82c76ff1ce1] received  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,701: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:22.701|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,703: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0f853564-d87e-4ba9-bcf9-4795e197c479] succeeded in 0.1610346883535385s: None  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,713: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0e41d9db-68f9-4783-a4a6-399972a50d0f] received  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,867: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:22.867|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:22] [2025-04-10 14:06:22,870: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[d6f0bebc-042b-4786-9a62-9898dfba0f11] succeeded in 0.16231389343738556s: None  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:22,875: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[23fc7f06-7264-4085-9b03-670788b8556e] received  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,089: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:23.089|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,091: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[12a27b65-90c6-4a43-ab2e-0dc8a2ec156d] succeeded in 0.2169242799282074s: None  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,095: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[bf073969-f69c-4659-a4d6-b0826e604a03] received  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,543: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:23.543|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,544: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[1c837055-3024-4d95-8bb0-25c4e5375e0c] succeeded in 0.44857046008110046s: None  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,550: INFO/MainProcess] Task baserow.contrib.database.webhooks.tasks.call_webhook[cb0be93e-8258-4eb1-86d3-1d3284cb3a57] received  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,727: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:23.726|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,727: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[751b2310-ed30-494b-9636-f82c76ff1ce1] succeeded in 0.18042519688606262s: None  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,731: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[eb8f8e53-5ed2-48cc-93fd-ff356751f7a4] received  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,865: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:23.865|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 38's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:23] [2025-04-10 14:06:23,866: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0e41d9db-68f9-4783-a4a6-399972a50d0f] succeeded in 0.13574224710464478s: None  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:23,870: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[9c5377cd-5195-4664-94f0-9e108d3a34b5] received  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,063: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:24.063|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,065: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[23fc7f06-7264-4085-9b03-670788b8556e] succeeded in 0.1959005445241928s: None  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,071: INFO/MainProcess] Task baserow.contrib.database.webhooks.tasks.call_webhook[f233e248-e78d-4351-a9b7-bd03849f5ea0] received  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,287: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:24.286|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 37's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,289: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[bf073969-f69c-4659-a4d6-b0826e604a03] succeeded in 0.2188338041305542s: None  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,298: INFO/MainProcess] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[42d286c4-f294-4561-a38d-180e6a3b6afa] received  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,531: INFO/ForkPoolWorker-11] Task baserow.contrib.database.webhooks.tasks.call_webhook[cb0be93e-8258-4eb1-86d3-1d3284cb3a57] succeeded in 0.23657812178134918s: None  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,542: INFO/MainProcess] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[ce2d54fd-c955-4e66-a6c1-c766d266d9d1] received  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,645: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:24.645|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 75's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,646: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[eb8f8e53-5ed2-48cc-93fd-ff356751f7a4] succeeded in 0.11018936336040497s: None  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,650: INFO/MainProcess] Task baserow.contrib.database.export.tasks.clean_up_old_jobs[223a148e-6806-40e0-941b-329599bcbdcf] received  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,841: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:24.841|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 63's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-10 14:06:24] [2025-04-10 14:06:24,843: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[9c5377cd-5195-4664-94f0-9e108d3a34b5] succeeded in 0.19377851486206055s: None  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:24,847: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0489d12a-5839-4af7-b6bc-997829f10ec5] received  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,060: INFO/ForkPoolWorker-11] Task baserow.contrib.database.webhooks.tasks.call_webhook[f233e248-e78d-4351-a9b7-bd03849f5ea0] succeeded in 0.21206693351268768s: None  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,065: INFO/MainProcess] Task baserow.contrib.database.webhooks.tasks.call_webhook[4b472ad9-fc8d-4965-9341-2126d93b3cb4] received  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,075: INFO/ForkPoolWorker-11] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[42d286c4-f294-4561-a38d-180e6a3b6afa] succeeded in 0.010239452123641968s: None  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,078: INFO/MainProcess] Task baserow.core.action.tasks.cleanup_old_actions[99ce8e42-6fdb-4834-8e8e-6e78eb051fc7] received  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,145: INFO/ForkPoolWorker-11] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[ce2d54fd-c955-4e66-a6c1-c766d266d9d1] succeeded in 0.06588354706764221s: None  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,151: INFO/MainProcess] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[ad7b9fc2-7f97-4807-9ea8-1a47c237528b] received  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,189: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:25.189|INFO|baserow.contrib.database.export.handler:clean_up_old_jobs:188 - Cleaning up 0 old jobs  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,194: INFO/ForkPoolWorker-11] Task baserow.contrib.database.export.tasks.clean_up_old_jobs[223a148e-6806-40e0-941b-329599bcbdcf] succeeded in 0.041799113154411316s: None  
 [BACKEND][2025-04-10 14:06:25] 127.0.0.1:48232 - "GET /api/_health/ HTTP/1.1" 200  
 [BACKEND][2025-04-10 14:06:25] 167.99.92.103:0 - "GET /api/database/rows/table/38/3350/?user_field_names=true HTTP/1.1" 200  
 [BACKEND][2025-04-10 14:06:25] 167.99.92.103:0 - "GET /api/database/rows/table/37/1322/?user_field_names=true HTTP/1.1" 200  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,199: INFO/MainProcess] Task baserow.core.import_export.tasks.delete_marked_import_export_resources[931ddd17-b0ea-42e7-8b61-6127aeba943a] received  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,884: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:25.884|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:488 - Updated table 63's tsvs for all rows with optional field filter of [1162].  
 [EXPORT_WORKER][2025-04-10 14:06:25] [2025-04-10 14:06:25,886: INFO/ForkPoolWorker-11] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[0489d12a-5839-4af7-b6bc-997829f10ec5] succeeded in 0.6841247230768204s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:25,893: INFO/MainProcess] Task baserow.core.import_export.tasks.mark_import_export_resources_for_deletion[3f5d9480-b2ec-4d1d-b06d-d7a7f234d924] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,133: INFO/ForkPoolWorker-11] Task baserow.contrib.database.webhooks.tasks.call_webhook[4b472ad9-fc8d-4965-9341-2126d93b3cb4] succeeded in 0.24177560210227966s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,138: INFO/MainProcess] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[45844522-350e-4082-94d2-b126adc51d0c] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,174: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:26.174|INFO|baserow.core.action.handler:clean_up_old_undoable_actions:306 - Cleaned up 0 actions.  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,175: INFO/ForkPoolWorker-11] Task baserow.core.action.tasks.cleanup_old_actions[99ce8e42-6fdb-4834-8e8e-6e78eb051fc7] succeeded in 0.03837206959724426s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,179: INFO/MainProcess] Task baserow.core.jobs.tasks.clean_up_jobs[6453f8d8-19e5-4ef6-9d96-ad8a76b746ef] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,180: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:26.179|ERROR|baserow.core.notifications.tasks:beat_send_instant_notifications_summary_by_email:105 - Cannot run `send_instant_notifications_email_to_users` more than once at the same time.  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,180: INFO/ForkPoolWorker-11] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[ad7b9fc2-7f97-4807-9ea8-1a47c237528b] succeeded in 0.0030047595500946045s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,183: INFO/MainProcess] Task baserow.core.trash.tasks.mark_old_trash_for_permanent_deletion[9f252d9c-eecf-41cb-ac4a-b3cbc037499f] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,229: INFO/ForkPoolWorker-11] Task baserow.core.import_export.tasks.delete_marked_import_export_resources[931ddd17-b0ea-42e7-8b61-6127aeba943a] succeeded in 0.04667995870113373s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,232: INFO/MainProcess] Task baserow.core.trash.tasks.permanently_delete_marked_trash[37c11162-2b14-4a50-85f5-32eee39cb3f4] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,270: INFO/ForkPoolWorker-11] Task baserow.core.import_export.tasks.mark_import_export_resources_for_deletion[3f5d9480-b2ec-4d1d-b06d-d7a7f234d924] succeeded in 0.039946720004081726s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,273: INFO/MainProcess] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[4ed05b48-ace7-453b-a7f3-879bb9873047] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,314: INFO/ForkPoolWorker-11] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[45844522-350e-4082-94d2-b126adc51d0c] succeeded in 0.04229845106601715s: None  
 [BACKEND][2025-04-10 14:06:26] 167.99.92.103:0 - "GET /api/database/rows/table/38/3351/?user_field_names=true HTTP/1.1" 200  
 [BACKEND][2025-04-10 14:06:26] WARNING 2025-04-10 14:06:26,319 django.request.log_response:241- Not Found: /api/database/rows/table/63/120/   
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,317: INFO/MainProcess] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[9951e486-d647-4533-8a20-344fe55d5fdc] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,416: INFO/ForkPoolWorker-11] Task baserow.core.jobs.tasks.clean_up_jobs[6453f8d8-19e5-4ef6-9d96-ad8a76b746ef] succeeded in 0.10029949247837067s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,422: INFO/MainProcess] Task baserow.contrib.database.fields.tasks.run_periodic_fields_updates[d30729a5-05e5-4456-827a-3cc06accd983] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,459: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:26.459|INFO|baserow.core.trash.handler:mark_old_trash_for_permanent_deletion:223 - Successfully marked 0 old trash items for deletion as they were older than 720 hours.  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,461: INFO/ForkPoolWorker-11] Task baserow.core.trash.tasks.mark_old_trash_for_permanent_deletion[9f252d9c-eecf-41cb-ac4a-b3cbc037499f] succeeded in 0.04032960534095764s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,468: INFO/MainProcess] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[0e926bf2-60b4-482c-91f1-f3e2fb5e4301] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,509: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:26.509|INFO|baserow.core.trash.handler:permanently_delete_marked_trash:305 - Successfully deleted 0 trash entries and their associated trashed items.  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,509: INFO/ForkPoolWorker-11] Task baserow.core.trash.tasks.permanently_delete_marked_trash[37c11162-2b14-4a50-85f5-32eee39cb3f4] succeeded in 0.040376245975494385s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,513: INFO/MainProcess] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[050d0b24-f9b3-4095-a0cd-21f32bcbf4bc] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,514: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:26.514|ERROR|baserow.core.notifications.tasks:beat_send_instant_notifications_summary_by_email:105 - Cannot run `send_instant_notifications_email_to_users` more than once at the same time.  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,514: INFO/ForkPoolWorker-11] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[4ed05b48-ace7-453b-a7f3-879bb9873047] succeeded in 0.0012990683317184448s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,517: INFO/MainProcess] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[8d08846c-5a0f-41e3-b231-d7e59fd3cb28] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,558: INFO/ForkPoolWorker-11] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[9951e486-d647-4533-8a20-344fe55d5fdc] succeeded in 0.04072330892086029s: None  
 [CELERY_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:12,134: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[5fdbc816-7ff9-48da-8d2e-93d470d2b2c0] succeeded in 0.009552404284477234s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,562: INFO/MainProcess] Task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due[dba4f3db-d7f6-4fd7-bddc-611ee45757e3] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,749: INFO/ForkPoolWorker-11] Task baserow.contrib.database.fields.tasks.run_periodic_fields_updates[d30729a5-05e5-4456-827a-3cc06accd983] succeeded in 0.18597038090229034s: None  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,754: INFO/MainProcess] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[0e9ab2ff-c515-4a5d-af58-70fa0a797b17] received  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,755: WARNING/ForkPoolWorker-11] 29982|2025-04-10 14:06:26.755|ERROR|baserow.core.notifications.tasks:beat_send_instant_notifications_summary_by_email:105 - Cannot run `send_instant_notifications_email_to_users` more than once at the same time.  
 [EXPORT_WORKER][2025-04-10 14:06:26] [2025-04-10 14:06:26,755: INFO/ForkPoolWorker-11] Task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email[0e926bf2-60b4-482c-91f1-f3e2fb5e4301] succeeded in 0.0022276192903518677s: None  

[bram]

Hey @Alex-Clapp, thank you for sharing the detailed information. The database team is indeed working on solving the deadlock problems at the moment. In your situation, it seems to be related to the search index update part. If all goes well, this one will also be solved in the next release.

[Alex-Clapp]

Copied, that would be great! Can you please let me know the relase number ?

If you need some logs, it happends again this morning:

[EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:31,909: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[ec889b6e-65f6-420c-b1be-c8f3c7b5c77e] received  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,353: WARNING/ForkPoolWorker-11] 29982|2025-04-14 09:52:38.352|ERROR|baserow.contrib.database.search.handler:try_slower_but_best_effort_tsv_update:690 - Failed to update search vector for field with id 652 / type <class 'baserow.contrib.database.fields.models.FormulaField'> because of deadlock detected  
 [EXPORT_WORKER][2025-04-14 09:52:38] DETAIL:  Process 2219187 waits for ShareLock on transaction 35381822; blocked by process 2219222.  
 [EXPORT_WORKER][2025-04-14 09:52:38] Process 2219222 waits for ShareLock on transaction 35381825; blocked by process 2219187.  
 [EXPORT_WORKER][2025-04-14 09:52:38] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-14 09:52:38] CONTEXT:  while updating tuple (2487,3) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-14 09:52:38] , field.__str__ is: FormulaField(  
 [EXPORT_WORKER][2025-04-14 09:52:38] formula=join(field('Article'),''),  
 [EXPORT_WORKER][2025-04-14 09:52:38] internal_formula=error_to_null(join(lookup('field_663','field_576'),'')),  
 [EXPORT_WORKER][2025-04-14 09:52:38] formula_type=text,  
 [EXPORT_WORKER][2025-04-14 09:52:38] error=None,  
 [EXPORT_WORKER][2025-04-14 09:52:38] ) and expression is LocalisedSearchVector(Func(Cast(F(field_652)), Value('[@:/+.]|-(?!\\d)|(?<=\\S)-(?=\\d)|\\b-\\b'), Value(' '), Value('g'), function=regexp_replace))  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,358: WARNING/ForkPoolWorker-11] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,360: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,360: WARNING/ForkPoolWorker-11] psycopg2.errors.DeadlockDetected: deadlock detected  
 [EXPORT_WORKER][2025-04-14 09:52:38] DETAIL:  Process 2219187 waits for ShareLock on transaction 35381775; blocked by process 2219171.  
 [EXPORT_WORKER][2025-04-14 09:52:38] Process 2219171 waits for ShareLock on transaction 35381776; blocked by process 2219187.  
 [EXPORT_WORKER][2025-04-14 09:52:38] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-14 09:52:38] CONTEXT:  while updating tuple (419,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,360: WARNING/ForkPoolWorker-11]   
 [EXPORT_WORKER][2025-04-14 09:52:38] The above exception was the direct cause of the following exception:  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,360: WARNING/ForkPoolWorker-11] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 604, in run_tsvector_update_statement  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return cls.split_update_into_chunks_until_all_background_done(  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 580, in split_update_into_chunks_until_all_background_done  
 [EXPORT_WORKER][2025-04-14 09:52:38]     this_chunk_updated = next_chunk.update(**update_query)  
 [EXPORT_WORKER][2025-04-14 09:52:38]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [EXPORT_WORKER][2025-04-14 09:52:38]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [EXPORT_WORKER][2025-04-14 09:52:38]     cursor = super().execute_sql(result_type)  
 [EXPORT_WORKER][2025-04-14 09:52:38]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [EXPORT_WORKER][2025-04-14 09:52:38]     cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return self._execute_with_wrappers(  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return executor(sql, params, many, context)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     with self.db.wrap_database_errors:  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,361: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [EXPORT_WORKER][2025-04-14 09:52:38]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11] django.db.utils.OperationalError: deadlock detected  
 [EXPORT_WORKER][2025-04-14 09:52:38] DETAIL:  Process 2219187 waits for ShareLock on transaction 35381775; blocked by process 2219171.  
 [EXPORT_WORKER][2025-04-14 09:52:38] Process 2219171 waits for ShareLock on transaction 35381776; blocked by process 2219187.  
 [EXPORT_WORKER][2025-04-14 09:52:38] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-14 09:52:38] CONTEXT:  while updating tuple (419,2) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   
 [EXPORT_WORKER][2025-04-14 09:52:38] During handling of the above exception, another exception occurred:  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11] psycopg2.errors.DeadlockDetected: deadlock detected  
 [EXPORT_WORKER][2025-04-14 09:52:38] DETAIL:  Process 2219187 waits for ShareLock on transaction 35381822; blocked by process 2219222.  
 [EXPORT_WORKER][2025-04-14 09:52:38] Process 2219222 waits for ShareLock on transaction 35381825; blocked by process 2219187.  
 [EXPORT_WORKER][2025-04-14 09:52:38] HINT:  See server log for query details.  
 [EXPORT_WORKER][2025-04-14 09:52:38] CONTEXT:  while updating tuple (2487,3) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   
 [EXPORT_WORKER][2025-04-14 09:52:38] The above exception was the direct cause of the following exception:  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 672, in try_slower_but_best_effort_tsv_update  
 [EXPORT_WORKER][2025-04-14 09:52:38]     cls.split_update_into_chunks_until_all_background_done(  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/backend/src/baserow/contrib/database/search/handler.py", line 580, in split_update_into_chunks_until_all_background_done  
 [EXPORT_WORKER][2025-04-14 09:52:38]     this_chunk_updated = next_chunk.update(**update_query)  
 [EXPORT_WORKER][2025-04-14 09:52:38]                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1253, in update  
 [EXPORT_WORKER][2025-04-14 09:52:38]     rows = query.get_compiler(self.db).execute_sql(CURSOR)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql  
 [EXPORT_WORKER][2025-04-14 09:52:38]     cursor = super().execute_sql(result_type)  
 [EXPORT_WORKER][2025-04-14 09:52:38]              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql  
 [EXPORT_WORKER][2025-04-14 09:52:38]     cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return self._execute_with_wrappers(  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,362: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return executor(sql, params, many, context)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,363: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     with self.db.wrap_database_errors:  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,363: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__  
 [EXPORT_WORKER][2025-04-14 09:52:38]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,363: WARNING/ForkPoolWorker-11]   File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute  
 [EXPORT_WORKER][2025-04-14 09:52:38]     return self.cursor.execute(sql, params)  
 [EXPORT_WORKER][2025-04-14 09:52:38]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 [EXPORT_WORKER][2025-04-14 09:52:38] [2025-04-14 09:52:38,363: WARNING/ForkPoolWorker-11] django.db.utils.OperationalError: deadlock detected  
 [EXPORT_WORKER][2025-04-14 09:52:38] DETAIL:  Process 2219187 waits for ShareLock on transaction 35381822; blocked by process 2219222.  
 [EXPORT_WORKER][2025-04-14 09:52:38] Process 2219222 waits for ShareLock on transaction 35381825; blocked by process 2219187.  
 [EXPORT_WORKER][2025-04-14 09:52:38] HINT:  See server log for query details. 

And

[EXPORT_WORKER][2025-04-14 10:22:29] CONTEXT:  while updating tuple (2487,3) in relation "database_table_38"  
 [EXPORT_WORKER][2025-04-14 10:22:29] [2025-04-14 10:22:29,864: ERROR/MainProcess] Task handler raised error: TimeLimitExceeded(1800)  
 [EXPORT_WORKER][2025-04-14 10:22:29] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-14 10:22:29]   File "/baserow/venv/lib/python3.11/site-packages/billiard/pool.py", line 683, in on_hard_timeout  
 [EXPORT_WORKER][2025-04-14 10:22:29]     raise TimeLimitExceeded(job._timeout)  
 [EXPORT_WORKER][2025-04-14 10:22:29] billiard.einfo.ExceptionWithTraceback:   
 [EXPORT_WORKER][2025-04-14 10:22:29] """  
 [EXPORT_WORKER][2025-04-14 10:22:29] Traceback (most recent call last):  
 [EXPORT_WORKER][2025-04-14 10:22:29]   File "/baserow/venv/lib/python3.11/site-packages/billiard/pool.py", line 683, in on_hard_timeout  
 [EXPORT_WORKER][2025-04-14 10:22:29]     raise TimeLimitExceeded(job._timeout)  
 [EXPORT_WORKER][2025-04-14 10:22:29] billiard.exceptions.TimeLimitExceeded: TimeLimitExceeded(1800,)  
 [EXPORT_WORKER][2025-04-14 10:22:29] """  
 [BEAT_WORKER][2025-04-14 10:22:30] [2025-04-14 10:22:05,552: INFO/MainProcess] Scheduler: Sending due task baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due() (baserow_enterprise.data_sync.tasks.call_periodic_data_sync_syncs_that_are_due)  
 [EXPORT_WORKER][2025-04-14 10:22:34] [2025-04-14 10:22:29,866: ERROR/MainProcess] Hard time limit (1800s) exceeded for baserow.contrib.database.search.tasks.async_update_tsvector_columns[44fa4693-8a36-4cba-bc70-f822488bbff8]  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:34,770: ERROR/MainProcess] Process 'ForkPoolWorker-11' pid:29982 exited with 'signal 9 (SIGKILL)'  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,308: WARNING/ForkPoolWorker-12] Not configuring telemetry due to BASEROW_ENABLE_OTEL not being set.  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,320: WARNING/ForkPoolWorker-12] 15293|2025-04-14 10:22:35.319|INFO|baserow.core.telemetry.telemetry:setup_logging:72 - Logger setup.  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,330: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[1ec2bcc6-feac-4a4e-89e6-1e6db9887ad9] received  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,613: INFO/ForkPoolWorker-12] Task baserow.contrib.database.webhooks.tasks.call_webhook[77f1203c-9703-4957-a3bc-9b6651e96fd4] succeeded in 0.2878044694662094s: None  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,623: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[23134843-8670-4996-b8e6-fb24e8eafca6] received  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,667: WARNING/ForkPoolWorker-12] 15293|2025-04-14 10:22:35.667|INFO|baserow.contrib.database.export.handler:clean_up_old_jobs:188 - Cleaning up 0 old jobs  
 [EXPORT_WORKER][2025-04-14 10:22:35] [2025-04-14 10:22:35,671: INFO/ForkPoolWorker-12] Task baserow.contrib.database.export.tasks.clean_up_old_jobs[c818d73b-8bc5-42a1-96d4-5b65de788572] succeeded in 0.05279289186000824s: None  
 [EXPORT_WORKER][2025-04-14 10:22:36] [2025-04-14 10:22:35,676: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[9260a0d6-fdde-43ee-adc8-251969920a43] received  
 [EXPORT_WORKER][2025-04-14 10:22:36] [2025-04-14 10:22:36,151: WARNING/ForkPoolWorker-12] 15293|2025-04-14 10:22:36.151|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 34's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-14 10:22:36] [2025-04-14 10:22:36,153: INFO/ForkPoolWorker-12] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[2d3d6e97-f1a3-4675-8d2b-49098f28c6e4] succeeded in 0.47858287394046783s: None  
 [EXPORT_WORKER][2025-04-14 10:22:36] [2025-04-14 10:22:36,159: INFO/MainProcess] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[55015339-f837-487c-b200-429cb610ff64] received  
 [EXPORT_WORKER][2025-04-14 10:22:36] [2025-04-14 10:22:36,575: WARNING/ForkPoolWorker-12] 15293|2025-04-14 10:22:36.575|INFO|baserow.contrib.database.search.handler:update_tsvector_columns:495 - Updated a unknown number of rows in table 37's tsvs with optional field filter of no fields.  
 [EXPORT_WORKER][2025-04-14 10:22:36] [2025-04-14 10:22:36,577: INFO/ForkPoolWorker-12] Task baserow.contrib.database.search.tasks.async_update_tsvector_columns[ec889b6e-65f6-420c-b1be-c8f3c7b5c77e] succeeded in 0.42225968837738037s: None  

[bram]

Hey @Alex-Clapp, the release will be 1.33.0, and will happen in about two weeks from now.

[Alex-Clapp]

Hello,

I wanted to inform you that the error still occurs frequently especially as the number of users continues to grow. Additionally, when this happens, it’s not only the webhooks that are blocked, but also the App Builder, which displays an error message in the top right corner when a button is pressed. No action is possible at this time.

Do you know if the next release will solve this problem ?

Thank you for your continued assistance.

Alex