I’m running a baserow instance using the AIO docker image deployed in a digital ocean droplet. The instance has been fine for many weeks but suddenly I’ve been getting errors.
I just hit this API endpoint on my instance: /api/database/fields/table/4059/to get some columns but failed. Its not limited to that API call, also happens when updating rows via API. It also doesn’t just happen on just 1 table or database, happens to many but not always. I don’t know what’s causing it
I have baserow v1.23 and earlier today, I upgraded to 1.29 to see if it fixes the issue - it didn’t. When I upgraded to v1.29 I put MIGRATE_ON_STARTUP=true and BASEROW_TRIGGER_SYNC_TEMPLATES_AFTER_MIGRATION=false – I had both as false because I’ve been testing a few configurations when I started self-hosting.
I’m not too familiar with python so I have no Idea why this happens.
Database is in Supabase
Digital Ocean storage/s3
self-hosted in a Droplet with docker using the all-in-one image
ERROR 2024-11-15 15:09:30,908 django.request.log_response:241- Internal Server Error: /api/database/fields/table/4059/
[BACKEND][2024-11-15 15:09:50] Traceback (most recent call last):
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
[BACKEND][2024-11-15 15:09:50] return self.cursor.execute(sql, params)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] psycopg2.OperationalError: SSL connection has been closed unexpectedly
full log:
[BACKEND][2024-11-15 15:09:50] ERROR 2024-11-15 15:09:30,908 django.request.log_response:241- Internal Server Error: /api/database/fields/table/4059/
[BACKEND][2024-11-15 15:09:50] Traceback (most recent call last):
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
[BACKEND][2024-11-15 15:09:50] return self.cursor.execute(sql, params)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] psycopg2.OperationalError: SSL connection has been closed unexpectedly
[BACKEND][2024-11-15 15:09:50]
[BACKEND][2024-11-15 15:09:50]
[BACKEND][2024-11-15 15:09:50] The above exception was the direct cause of the following exception:
[BACKEND][2024-11-15 15:09:50]
[BACKEND][2024-11-15 15:09:50] Traceback (most recent call last):
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 486, in thread_handler
[BACKEND][2024-11-15 15:09:50] raise exc_info[1]
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 43, in inner
[BACKEND][2024-11-15 15:09:50] response = await get_response(request)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 486, in thread_handler
[BACKEND][2024-11-15 15:09:50] raise exc_info[1]
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
[BACKEND][2024-11-15 15:09:50] response = await wrapped_callback(
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 448, in __call__
[BACKEND][2024-11-15 15:09:50] ret = await asyncio.wait_for(future, timeout=None)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/usr/lib/python3.11/asyncio/tasks.py", line 442, in wait_for
[BACKEND][2024-11-15 15:09:50] return await fut
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/asgiref/current_thread_executor.py", line 22, in run
[BACKEND][2024-11-15 15:09:50] result = self.fn(*self.args, **self.kwargs)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/asgiref/sync.py", line 490, in thread_handler
[BACKEND][2024-11-15 15:09:50] return func(*args, **kwargs)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/views/decorators/csrf.py", line 55, in wrapped_view
[BACKEND][2024-11-15 15:09:50] return view_func(*args, **kwargs)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/views/generic/base.py", line 103, in view
[BACKEND][2024-11-15 15:09:50] return self.dispatch(request, *args, **kwargs)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 509, in dispatch
[BACKEND][2024-11-15 15:09:50] response = self.handle_exception(exc)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
[BACKEND][2024-11-15 15:09:50] self.raise_uncaught_exception(exc)
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
[BACKEND][2024-11-15 15:09:50] raise exc
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 497, in dispatch
[BACKEND][2024-11-15 15:09:50] self.initial(request, *args, **kwargs)
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 414, in initial
[BACKEND][2024-11-15 15:09:50] self.perform_authentication(request)
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/views.py", line 324, in perform_authentication
[BACKEND][2024-11-15 15:09:50] request.user
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/request.py", line 227, in user
[BACKEND][2024-11-15 15:09:50] self._authenticate()
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/rest_framework/request.py", line 380, in _authenticate
[BACKEND][2024-11-15 15:09:50] user_auth_tuple = authenticator.authenticate(self)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/backend/src/baserow/contrib/database/api/tokens/authentications.py", line 40, in authenticate
[BACKEND][2024-11-15 15:09:50] token = handler.get_by_key(decoded_key)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/backend/src/baserow/contrib/database/tokens/handler.py", line 45, in get_by_key
[BACKEND][2024-11-15 15:09:50] token = Token.objects.select_related("workspace").get(key=key)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 646, in get
[BACKEND][2024-11-15 15:09:50] num = len(clone)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 376, in __len__
[BACKEND][2024-11-15 15:09:50] self._fetch_all()
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 1867, in _fetch_all
[BACKEND][2024-11-15 15:09:50] self._result_cache = list(self._iterable_class(self))
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/models/query.py", line 87, in __iter__
[BACKEND][2024-11-15 15:09:50] results = compiler.execute_sql(
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
[BACKEND][2024-11-15 15:09:50] cursor.execute(sql, params)
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
[BACKEND][2024-11-15 15:09:50] return self._execute_with_wrappers(
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
[BACKEND][2024-11-15 15:09:50] return executor(sql, params, many, context)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
[BACKEND][2024-11-15 15:09:50] with self.db.wrap_database_errors:
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
[BACKEND][2024-11-15 15:09:50] raise dj_exc_value.with_traceback(traceback) from exc_value
[BACKEND][2024-11-15 15:09:50] File "/baserow/venv/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
[BACKEND][2024-11-15 15:09:50] return self.cursor.execute(sql, params)
[BACKEND][2024-11-15 15:09:50] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[BACKEND][2024-11-15 15:09:50] django.db.utils.OperationalError: SSL connection has been closed unexpectedly