Import .csv is stuck in importing

Rob192 · October 12, 2023, 9:37am

Please fill in the questionnaire below.

Technical Help Questionnaire

Have you read and followed the instructions at: *READ ME FIRST* Technical Help FAQs - #2 by nigel ?

Answer: Yes I did !

Self-Hosted Installation and Setup Questions

Delete this section if you are using Baserow.io.

How have you self-hosted Baserow.

I am using a docker container. The data inside the data base is migrated from a previous instance set up with docker 1.11.0 and migrated per your instructions.

version: '3.1'

services:
  baserow:
    image: baserow/baserow:1.20.1
    container_name: baserow
    environment:
      - BASEROW_PUBLIC_URL=https://xxx
      - BASEROW_CADDY_ADDRESSES=https://xxx
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - baserow_data:/baserow/data
    restart: unless-stopped

volumes:
  baserow_data:
    external: true

What are the specs of the service or server you are using to host Baserow.

I am using 4GB. it’s a t2.medium from AWS EC2

Which version of Baserow are you using.

1.20.1

How have you configured your self-hosted installation?

cf. above

What commands if any did you use to start your Baserow server?

docker compose up

Describe the problem

I am using baserow to log data from our factory jigs + from user app though API calls.
I had some issues with baserow being very slow recently after the update to 1.20.1 from 1.11.0.
I decided to try to split the services and create one instance for the jigs and one instance for the users as I feared I had too many API calls on the database connected to our user app.
In the process of doing that something went wrong (murphy’s law) and I had to recreate a new instance for the jigs with old data. Because I could still access the recently created data in my previous instance, i extracted some csv (biggest is 40 Mo, 1500 lines) so I could import them again in my new and working instance.
The problem is that when I tried to import the biggest csv file, the importation never ended.
Now when I try to import other csv, the UI is telling me the below. Also I feel the whole instance is slow again.
At this point I am afraid to stop the instance as I fear this job could compromise the database and I could not access the data anymore (I have no XP in working with databases )

Already running

Another import job is already running. You need to wait for that one to finish before starting another.

My question : do you know what could render baserow slow ? Do you know how to kill a csv import job to try to chunk the file in smallest files ?
Please help

Provide screenshots or include share links showing:

How many rows in total do you have in your Baserow tables?

109k + 16k + 81k + 265k + 214k + 24k + 14k = 723k lines in total

Please attach full logs from all of Baserow’s services

extract of the logs

 [BACKEND][2023-10-12 09:20:06] 581|2023-10-12 09:20:05.787|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [BACKEND][2023-10-12 09:20:09] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:09] [2023-10-12 09:20:06,882: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[d15abae5-d8ab-4109-ae8f-a68710742b31] succeeded in 0.0044973359908908606s: None
 [CELERY_WORKER][2023-10-12 09:20:09] [2023-10-12 09:20:09,673: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[c63d12ad-eee8-433f-b664-e69d844b210b] received
 [BACKEND][2023-10-12 09:20:09] 581|2023-10-12 09:20:09.067|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=move_row user=1
 [BACKEND][2023-10-12 09:20:09] 78.202.149.100:0 - "PATCH /api/database/rows/table/343/154/move/?before_id=137 HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:10] [2023-10-12 09:20:09,680: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[c63d12ad-eee8-433f-b664-e69d844b210b] succeeded in 0.006649849994573742s: None
 [BACKEND][2023-10-12 09:20:10] 581|2023-10-12 09:20:09.821|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [CELERY_WORKER][2023-10-12 09:20:10] [2023-10-12 09:20:10,528: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[8fcb9f56-25b0-4b63-b393-2be54d12a600] received
 [BACKEND][2023-10-12 09:20:12] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [BACKEND][2023-10-12 09:20:12] 127.0.0.1:36480 - "GET /api/_health/ HTTP/1.1" 200
 [BACKEND][2023-10-12 09:20:13] 581|2023-10-12 09:20:12.480|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=move_row user=1
 [CELERY_WORKER][2023-10-12 09:20:14] [2023-10-12 09:20:10,536: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[8fcb9f56-25b0-4b63-b393-2be54d12a600] succeeded in 0.007375040033366531s: None
 [CELERY_WORKER][2023-10-12 09:20:14] [2023-10-12 09:20:14,126: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[9404d734-ec68-484d-ac50-1f54bb4c8df7] received
 [BACKEND][2023-10-12 09:20:14] 580|2023-10-12 09:20:13.173|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [CELERY_WORKER][2023-10-12 09:20:15] [2023-10-12 09:20:14,134: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[9404d734-ec68-484d-ac50-1f54bb4c8df7] succeeded in 0.006156745017506182s: None
 [CELERY_WORKER][2023-10-12 09:20:15] [2023-10-12 09:20:15,345: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[d397348d-f907-43e4-97a2-fd08b7a67362] received
 [BACKEND][2023-10-12 09:20:15] 78.202.149.100:0 - "PATCH /api/database/rows/table/343/154/move/?before_id=135 HTTP/1.1" 200
 [BACKEND][2023-10-12 09:20:17] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [BACKEND][2023-10-12 09:20:25] 127.0.0.1:45680 - "GET /api/_health/ HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:25] [2023-10-12 09:20:15,351: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[d397348d-f907-43e4-97a2-fd08b7a67362] succeeded in 0.004331787000410259s: None
 [CELERY_WORKER][2023-10-12 09:20:25] [2023-10-12 09:20:25,257: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[810e23f3-9bd9-435d-a291-b31e17c814fa] received
 [BACKEND][2023-10-12 09:20:25] 580|2023-10-12 09:20:25.041|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [BACKEND][2023-10-12 09:20:28] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:28] [2023-10-12 09:20:25,262: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[810e23f3-9bd9-435d-a291-b31e17c814fa] succeeded in 0.004611869982909411s: None
 [BACKEND][2023-10-12 09:20:28] 582|2023-10-12 09:20:28.451|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [CELERY_WORKER][2023-10-12 09:20:28] [2023-10-12 09:20:28,814: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[15374e75-becd-446d-81b1-cf1e6a19c490] received
 [BACKEND][2023-10-12 09:20:32] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [BACKEND][2023-10-12 09:20:34] 127.0.0.1:36110 - "GET /api/_health/ HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:35] [2023-10-12 09:20:28,820: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[15374e75-becd-446d-81b1-cf1e6a19c490] succeeded in 0.005640959017910063s: None
 [BACKEND][2023-10-12 09:20:35] 581|2023-10-12 09:20:34.553|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [CELERY_WORKER][2023-10-12 09:20:35] [2023-10-12 09:20:35,381: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[445f841a-ee57-418d-9b7e-14c23eee2958] received
 [BACKEND][2023-10-12 09:20:36] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:36] [2023-10-12 09:20:35,389: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[445f841a-ee57-418d-9b7e-14c23eee2958] succeeded in 0.006583498034160584s: None
 [CELERY_WORKER][2023-10-12 09:20:36] [2023-10-12 09:20:36,710: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[a8646d4c-ac07-44d8-9bb4-5e1a1b4af942] received
 [BACKEND][2023-10-12 09:20:36] 581|2023-10-12 09:20:36.179|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [BACKEND][2023-10-12 09:20:41] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:42] [2023-10-12 09:20:36,715: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[a8646d4c-ac07-44d8-9bb4-5e1a1b4af942] succeeded in 0.004254892992321402s: None
 [BACKEND][2023-10-12 09:20:42] 580|2023-10-12 09:20:41.479|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [CELERY_WORKER][2023-10-12 09:20:42] [2023-10-12 09:20:42,028: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[0707e09a-ca27-46e1-a224-7d896fc864b8] received
 [BACKEND][2023-10-12 09:20:45] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [CELERY_WORKER][2023-10-12 09:20:46] [2023-10-12 09:20:42,033: INFO/ForkPoolWorker-1] Task baserow.ws.tasks.broadcast_to_channel_group[0707e09a-ca27-46e1-a224-7d896fc864b8] succeeded in 0.004466312995646149s: None
 [CELERY_WORKER][2023-10-12 09:20:46] [2023-10-12 09:20:46,845: INFO/MainProcess] Task baserow.ws.tasks.broadcast_to_channel_group[a528dad8-a8ab-480d-9809-76cb1a9776dc] received
 [BACKEND][2023-10-12 09:20:46] 581|2023-10-12 09:20:45.723|INFO|baserow.core.action.signals:log_action_receiver:28 - do: workspace=60 action_type=create_row user=1
 [BACKEND][2023-10-12 09:20:52] 183.17.64.188:0 - "POST /api/database/rows/table/316/?user_field_names=true HTTP/1.1" 200
 [BEAT_WORKER][2023-10-12 09:21:00] [2023-10-12 09:20:00,022: INFO/MainProcess] Scheduler: Sending due task baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email() (baserow.core.notifications.tasks.beat_send_instant_notifications_summary_by_email)

processes running

USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           2  0.0  0.0      0     0 ?        S    Oct06   0:00 [kthreadd]
root           3  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [rcu_gp]
root           4  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [rcu_par_gp]
root           5  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [slub_flushwq]
root           6  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [netns]
root           8  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kworker/0:0H-events_highpri]
root          10  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [mm_percpu_wq]
root          11  0.0  0.0      0     0 ?        I    Oct06   0:00  \_ [rcu_tasks_rude_kthread]
root          12  0.0  0.0      0     0 ?        I    Oct06   0:00  \_ [rcu_tasks_trace_kthread]
root          13  0.0  0.0      0     0 ?        S    Oct06   0:14  \_ [ksoftirqd/0]
root          14  0.0  0.0      0     0 ?        I    Oct06   2:59  \_ [rcu_sched]
root          15  0.0  0.0      0     0 ?        S    Oct06   0:01  \_ [migration/0]
root          16  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [idle_inject/0]
root          18  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [cpuhp/0]
root          19  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [cpuhp/1]
root          20  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [idle_inject/1]
root          21  0.0  0.0      0     0 ?        S    Oct06   0:02  \_ [migration/1]
root          22  0.0  0.0      0     0 ?        S    Oct06   0:26  \_ [ksoftirqd/1]
root          24  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kworker/1:0H-kblockd]
root          25  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [kdevtmpfs]
root          26  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [inet_frag_wq]
root          27  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [kauditd]
root          29  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [khungtaskd]
root          31  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [oom_reaper]
root          32  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [writeback]
root          34  0.0  0.0      0     0 ?        S    Oct06   3:46  \_ [kcompactd0]
root          35  0.0  0.0      0     0 ?        SN   Oct06   0:00  \_ [ksmd]
root          36  0.0  0.0      0     0 ?        SN   Oct06   0:04  \_ [khugepaged]
root          37  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kintegrityd]
root          38  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kblockd]
root          39  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [blkcg_punt_bio]
root          40  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [xen-balloon]
root          41  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [tpm_dev_wq]
root          42  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [ata_sff]
root          43  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [md]
root          44  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [edac-poller]
root          45  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [devfreq_wq]
root          46  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [watchdogd]
root          47  0.0  0.0      0     0 ?        I<   Oct06   5:00  \_ [kworker/0:1H-kblockd]
root          48  0.1  0.0      0     0 ?        S    Oct06   9:04  \_ [kswapd0]
root          49  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [ecryptfs-kthread]
root          50  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kthrotld]
root          51  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [acpi_thermal_pm]
root          52  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [xenbus]
root          53  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [xenwatch]
root          54  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [nvme-wq]
root          55  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [nvme-reset-wq]
root          56  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [nvme-delete-wq]
root          57  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [nvme-auth-wq]
root          58  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [scsi_eh_0]
root          59  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [scsi_tmf_0]
root          60  0.0  0.0      0     0 ?        S    Oct06   0:00  \_ [scsi_eh_1]
root          61  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [scsi_tmf_1]
root          62  0.0  0.0      0     0 ?        I<   Oct06   4:13  \_ [kworker/1:1H-kblockd]
root          64  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [mld]
root          65  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [ipv6_addrconf]
root          72  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kstrp]
root          74  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [zswap-shrink]
root          75  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kworker/u31:0]
root          79  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [charger_manager]
root          80  0.0  0.0      0     0 ?        S    Oct06   0:41  \_ [jbd2/xvda1-8]
root          81  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [ext4-rsv-conver]
root         152  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kaluad]
root         156  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kmpath_rdacd]
root         157  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kmpathd]
root         158  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [kmpath_handlerd]
root         191  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [cryptd]
root         552  0.0  0.0      0     0 ?        I<   Oct06   0:00  \_ [tls-strp]
root      947561  0.0  0.0      0     0 ?        D    Oct09   2:44  \_ [kworker/u30:5+flush-202:0]
root     1200443  0.0  0.0      0     0 ?        I    Oct11   0:02  \_ [kworker/0:1-events]
root     1235556  0.0  0.0      0     0 ?        I    Oct11   0:01  \_ [kworker/1:2-events]
root     1306353  0.0  0.0      0     0 ?        I    08:34   0:00  \_ [kworker/u30:2-events_unbound]
root     1310113  0.0  0.0      0     0 ?        I    09:00   0:00  \_ [kworker/u30:0-writeback]
root     1311836  0.0  0.0      0     0 ?        I    09:14   0:00  \_ [kworker/u30:3-flush-202:0]
root     1312837  0.0  0.0      0     0 ?        I    09:21   0:00  \_ [kworker/0:2-cgroup_destroy]
root     1312839  0.0  0.0      0     0 ?        I    09:22   0:00  \_ [kworker/1:1-cgroup_destroy]
root     1314148  0.0  0.0      0     0 ?        I    09:30   0:00  \_ [kworker/0:0-cgroup_destroy]
root     1314206  0.0  0.0      0     0 ?        I    09:30   0:00  \_ [kworker/u30:1-flush-202:0]
root     1314207  0.0  0.0      0     0 ?        I    09:30   0:00  \_ [kworker/u30:4-events_unbound]
root     1314764  0.0  0.0      0     0 ?        I    09:34   0:00  \_ [kworker/0:3]
root           1  0.0  0.2 167488 10328 ?        Ss   Oct06   0:51 /sbin/init
root         121  0.0  0.3  72764 13440 ?        S<s  Oct06   0:08 /lib/systemd/systemd-journald
root         159  0.0  0.6 289312 27392 ?        SLsl Oct06   0:36 /sbin/multipathd -d -s
root         170  0.0  0.1  11476  4768 ?        Ss   Oct06   0:01 /lib/systemd/systemd-udevd
systemd+     323  0.0  0.1  16244  4992 ?        Ss   Oct06   0:03 /lib/systemd/systemd-networkd
systemd+     325  0.0  0.2  25660  8804 ?        Ss   Oct06   0:02 /lib/systemd/systemd-resolved
root         359  0.0  0.0   2812  2048 ?        Ss   Oct06   0:00 /usr/sbin/acpid
root         365  0.0  0.0   7284  2560 ?        Ss   Oct06   0:00 /usr/sbin/cron -f -P
message+     366  0.0  0.1   8716  4224 ?        Ss   Oct06   0:02 @dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
root         373  0.0  0.0  82696  3328 ?        Ssl  Oct06   0:14 /usr/sbin/irqbalance --foreground
root         376  0.0  0.3  33080 13440 ?        Ss   Oct06   0:00 /usr/bin/python3 /usr/bin/networkd-dispatcher --run-startup-triggers
syslog       380  0.0  0.1 222400  5248 ?        Ssl  Oct06   0:00 /usr/sbin/rsyslogd -n -iNONE
_chrony      387  0.0  0.0  18888  3100 ?        S    Oct06   0:09 /usr/sbin/chronyd -F 1
_chrony      405  0.0  0.0  10560  1988 ?        S    Oct06   0:00  \_ /usr/sbin/chronyd -F 1
root         408  0.0  0.1  16000  6144 ?        Ss   Oct06   0:01 /lib/systemd/systemd-logind
root         414  0.0  0.4 1356764 19644 ?       Ssl  Oct06   3:38 /usr/bin/containerd
root         467  0.0  0.0   6216  2176 ttyS0    Ss+  Oct06   0:00 /sbin/agetty -o -p -- \u --keep-baud 115200,57600,38400,9600 ttyS0 vt220
root         477  0.0  0.3 110084 12288 ?        Ssl  Oct06   0:00 /usr/bin/python3 /usr/share/unattended-upgrades/unattended-upgrade-shutdown --wait-for-signal
root         492  0.0  0.0   6172  1920 tty1     Ss+  Oct06   0:00 /sbin/agetty -o -p -- \u --noclear tty1 linux
root         533  0.0  0.1 235476  5972 ?        Ssl  Oct06   0:00 /usr/libexec/polkitd --no-debug
root         550  0.0  1.0 1832756 40156 ?       Ssl  Oct06   6:30 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
root      946836  0.0  0.0 1155756 2176 ?        Sl   Oct09   0:00  \_ /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 443 -container-ip 172.18.0.2 -container-port 443
root      946842  0.0  0.0 1082024 2176 ?        Sl   Oct09   0:00  \_ /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 443 -container-ip 172.18.0.2 -container-port 443
root      946857  0.0  0.0 1229744 2176 ?        Sl   Oct09   0:00  \_ /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.18.0.2 -container-port 80
root      946863  0.0  0.0 1082024 2176 ?        Sl   Oct09   0:00  \_ /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 80 -container-ip 172.18.0.2 -container-port 80
root         554  0.0  0.1  15420  5760 ?        Ss   Oct06   0:00 sshd: /usr/sbin/sshd -D -o AuthorizedKeysCommand /usr/share/ec2-instance-connect/eic_run_authorized_keys %u %f -o Authoriz
root     1314140  0.0  0.2  16916 10368 ?        Ss   09:30   0:00  \_ sshd: ubuntu [priv]
ubuntu   1314204  0.0  0.1  17308  7800 ?        S    09:30   0:00      \_ sshd: ubuntu@pts/0
ubuntu   1314205  0.0  0.1   9280  5120 pts/0    Ss   09:30   0:00          \_ -bash
ubuntu   1314993  0.0  0.0  10928  3712 pts/0    R+   09:36   0:00              \_ ps auxf
root      151944  0.0  0.2 1389020 9796 ?        Ssl  Oct06   2:12 /snap/amazon-ssm-agent/7628/amazon-ssm-agent
root      946883  0.0  0.1 719852  5640 ?        Sl   Oct09   0:39 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 126d81d9527c35a91775e908f5f590c7d985fe041c2da56ac238f532d1529482 -add
root      946904  0.0  0.5  29408 21108 ?        Ss   Oct09   1:06  \_ /usr/bin/python3 /usr/bin/supervisord --configuration /baserow/supervisor/supervisor.conf
root      947095  0.0  0.0   5788  3072 ?        S    Oct09   0:00      \_ bash /baserow/supervisor/stop-supervisor.sh
9999      947096  0.0  0.0   5920  3200 ?        S    Oct09   0:12      \_ bash /baserow/supervisor/baserow-watcher.sh
9999      947167  0.0  0.0   5788  1472 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh BOLD BASEROW-WATCHER /baserow/supervisor/baserow-watcher.sh
9999      947178  0.0  0.0  10228  2688 ?        S    Oct09   0:00      |   |   \_ gawk -vRS=[\r\n] -vORS= {  print "?[1m",strftime("[BASEROW-WATCHER][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , R
9999     1314936  0.0  0.0   4284  1664 ?        S    09:35   0:00      |   \_ sleep 20
root      947097  0.0  0.4 738956 18124 ?        Sl   Oct09   2:20      \_ /usr/bin/caddy run --config /baserow/caddy/Caddyfile
root      947176  0.0  0.0   5788  1600 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh PURPLE CADDY /usr/bin/caddy run --config /baserow/caddy/Caddyfile
root      947181  0.0  0.0  10228  2688 ?        S    Oct09   0:00      |       \_ gawk -vRS=[\r\n] -vORS= {  print "?[35m",strftime("[CADDY][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT; fflush
message+  947098  0.0  0.5 211592 23296 ?        S    Oct09   0:56      \_ /usr/lib/postgresql/11/bin/postgres -c config_file=/etc/postgresql/11/main/postgresql.conf
message+  947142  0.0  0.0   5788  1472 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh PURPLE POSTGRES /usr/lib/postgresql/11/bin/postgres -c config_file=/etc/po
message+  947145  0.0  0.0  10228  2816 ?        S    Oct09   0:00      |   |   \_ gawk -vRS=[\r\n] -vORS= {  print "?[35m",strftime("[POSTGRES][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT; ffl
message+  947462  0.0  3.6 214984 146996 ?       Ds   Oct09   2:44      |   \_ postgres: 11/main: checkpointer
message+  947463  0.1  3.6 213516 146936 ?       Ds   Oct09   3:55      |   \_ postgres: 11/main: background writer
message+  947464  0.1  0.2 211592  9720 ?        Ds   Oct09   4:14      |   \_ postgres: 11/main: walwriter
message+  947465  0.0  0.6 211996 24952 ?        Ss   Oct09   0:04      |   \_ postgres: 11/main: autovacuum launcher
message+  947466  0.0  0.1  68576  5880 ?        Ss   Oct09   0:35      |   \_ postgres: 11/main: stats collector
message+  947467  0.0  0.1 211996  7288 ?        Ss   Oct09   0:00      |   \_ postgres: 11/main: logical replication launcher
message+  964364  0.2  4.0 223048 160692 ?       Ds   Oct09   9:08      |   \_ postgres: 11/main: baserow baserow 127.0.0.1(46676) UPDATE
message+  987315  0.6  4.0 222580 160404 ?       Ds   Oct09  23:09      |   \_ postgres: 11/main: baserow baserow 127.0.0.1(59304) UPDATE
message+ 1314346  0.5  0.6 280396 27096 ?        Ds   09:31   0:01      |   \_ postgres: 11/main: autovacuum worker   baserow
message+ 1314930  0.5  0.8 225128 34344 ?        Ss   09:35   0:00      |   \_ postgres: 11/main: autovacuum worker   baserow
message+ 1314990  1.0  0.9 214144 36600 ?        Ds   09:36   0:00      |   \_ postgres: 11/main: baserow baserow 127.0.0.1(32868) SELECT
message+ 1314991  0.0  0.5 212912 20344 ?        Ds   09:36   0:00      |   \_ postgres: 11/main: parallel worker for PID 313898
message+ 1314992  0.0  0.4 212912 18936 ?        Ds   09:36   0:00      |   \_ postgres: 11/main: parallel worker for PID 313898
systemd+  947099  0.1  0.5  82132 22808 ?        Sl   Oct09   6:55      \_ /usr/bin/redis-server *:6379
systemd+  947177  0.0  0.0   5788  1600 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh PURPLE REDIS /usr/bin/redis-server --requirepass 7reyaerq3hkg1amiujoz4ks4w
systemd+  947185  0.0  0.0  10228  2560 ?        S    Oct09   0:00      |       \_ gawk -vRS=[\r\n] -vORS= {  print "?[35m",strftime("[REDIS][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT; fflush
9999      947102  0.1  3.3 173688 134356 ?       S    Oct09   5:12      \_ /baserow/venv/bin/python /baserow/venv/bin/celery -A baserow worker --concurrency 1 -l INFO -Q celery -n default-w
9999      947192  0.0  0.0   5788  1680 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh CYAN CELERY_WORKER ./docker/docker-entrypoint.sh celery-worker
9999      947197  0.0  0.0  10228  2688 ?        S    Oct09   0:01      |   |   \_ gawk -vRS=[\r\n] -vORS= {  print "?[36m",strftime("[CELERY_WORKER][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT
9999      947323  0.0  3.6 408556 144816 ?       S    Oct09   0:50      |   \_ /baserow/venv/bin/python /baserow/venv/bin/celery -A baserow worker --concurrency 1 -l INFO -Q celery -n defau
9999      947104  0.0  3.3 172332 132796 ?       S    Oct09   3:06      \_ /baserow/venv/bin/python /baserow/venv/bin/celery -A baserow worker --concurrency 1 -l INFO -Q export -n export-wo
9999      947184  0.0  0.0   5788  1600 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh CYAN EXPORT_WORKER ./docker/docker-entrypoint.sh celery-exportworker
9999      947194  0.0  0.0  10228  2688 ?        S    Oct09   0:00      |   |   \_ gawk -vRS=[\r\n] -vORS= {  print "?[36m",strftime("[EXPORT_WORKER][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT
9999      968763  0.0  3.5 182976 140896 ?       S    Oct09   0:00      |   \_ /baserow/venv/bin/python /baserow/venv/bin/celery -A baserow worker --concurrency 1 -l INFO -Q export -n expor
9999      947110  0.8  3.9 11522848 156532 ?     Sl   Oct09  32:25      \_ node /baserow/web-frontend/node_modules/.bin/nuxt start --hostname 127.0.0.1 --port 3000 --config-file ./config/nu
9999      947210  0.0  0.0   5788  1472 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh YELLOW WEBFRONTEND ./docker/docker-entrypoint.sh nuxt-local
9999      947235  0.0  0.0  10228  2560 ?        S    Oct09   0:00      |       \_ gawk -vRS=[\r\n] -vORS= {  print "?[33m",strftime("[WEBFRONTEND][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT;
9999      947113  0.0  3.2 170620 131196 ?       S    Oct09   0:30      \_ /baserow/venv/bin/python /baserow/venv/bin/celery -A baserow beat -l INFO -S redbeat.RedBeatScheduler
9999      947228  0.0  0.0   5788  1600 ?        S    Oct09   0:00      |   \_ bash /baserow/supervisor/wrapper.sh CYAN BEAT_WORKER ./docker/docker-entrypoint.sh celery-beat
9999      947250  0.0  0.0  10228  2688 ?        S    Oct09   0:00      |       \_ gawk -vRS=[\r\n] -vORS= {  print "?[36m",strftime("[BEAT_WORKER][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT;
9999      947474  0.0  0.5  31564 22664 ?        S    Oct09   0:25      \_ /baserow/venv/bin/python /baserow/venv/bin/gunicorn --workers=3 --worker-tmp-dir /dev/shm --log-file=- --access-lo
9999      947483  0.0  0.0   5788  1600 ?        S    Oct09   0:00          \_ bash /baserow/supervisor/wrapper.sh BLUE BACKEND ./docker/docker-entrypoint.sh gunicorn
9999      947484  0.0  0.0  10228  2688 ?        S    Oct09   0:03          |   \_ gawk -vRS=[\r\n] -vORS= {  print "?[34m",strftime("[BACKEND][%Y-%m-%d %H:%M:%S]"), $0, "?(B?[m" , RT; fflu
9999      947513  0.5  4.8 722552 194824 ?       Sl   Oct09  20:48          \_ /baserow/venv/bin/python /baserow/venv/bin/gunicorn --workers=3 --worker-tmp-dir /dev/shm --log-file=- --acces
9999      947514  0.2  4.3 569732 175856 ?       Sl   Oct09  10:51          \_ /baserow/venv/bin/python /baserow/venv/bin/gunicorn --workers=3 --worker-tmp-dir /dev/shm --log-file=- --acces
9999      947515  0.2  4.9 702168 199480 ?       Sl   Oct09   8:57          \_ /baserow/venv/bin/python /baserow/venv/bin/gunicorn --workers=3 --worker-tmp-dir /dev/shm --log-file=- --acces
root     1235234  0.0  0.3 1393096 14940 ?       Ssl  Oct11   0:15 /usr/lib/snapd/snapd
ubuntu   1314147  0.0  0.2  16928  9216 ?        Ss   09:30   0:00 /lib/systemd/systemd --user
ubuntu   1314149  0.0  0.1 170400  6516 ?        S    09:30   0:00  \_ (sd-pam)
root     1314763  0.0  0.4 296040 19328 ?        Ssl  09:34   0:00 /usr/libexec/packagekitd

Rob192 · October 13, 2023, 2:41pm

Do you think there is a risk for the database if I just restart the docker to cancel the operation ?

bram · October 16, 2023, 8:21am

Hi @Rob192, I’m sorry to hear that you’re having some problems with your Baserow instance.

do you know what could render baserow slow ?

There could be various reasons why your Baserow environment is slow. Do you have any insights in the CPU and memory usage of your server? Do you have any insights in how many API requests you’re making per second?

The problem is that when I tried to import the biggest csv file, the importation never ended.

Did you see the import percentage going up, or did it remain stuck at 0%? If so, would you mind checking if the real-time collaboration works? Just open two browser tabs, change a cell in one tab, and see it being updated in the other? The reason I’m asking is that I want to make sure async jobs are working. Although judging from your logs, it does sound like they it works.

Do you know how to kill a csv import job to try to chunk the file in smallest files ?

1500 lines doesn’t seem like that much, and you should be able to import that quickly in any environment. Out of curiosity, how many columns does the CSV have? It’s not easily possible to cancel the job, but restarting the Baserow Docker container should kill the execution of it. The job itself might remain pending in the PostgreSQL database, but if I’m correct, that will automatically be cleaned up within 24 hours. Is it already possible to start the job again?

Do you think there is a risk for the database if I just restart the docker to cancel the operation ?

It should normally not be a problem to restart your Docker container.

Separately, can you share an overview of all the environment variables you’ve set?

Rob192 · October 24, 2023, 7:48am

Hello @bram ,

Thanks for reaching out to me ! I had a hard time restarting the Baserow instance on my end (with Panic mode activated because we use it for our business).
I had issues for the Baserow instance not being able to restart because the database could not start. I ended up using this Investigate, document and prevent or fix postgres database corruption when two all-in-one images run at the same time with same volume (#1524) · Issues · Baserow / baserow · GitLab and it worked I think it would be good to add this to the documentation for the backup of the database.

Unfortunately they are all described in the docker compose file I sent
environment:
- BASEROW_PUBLIC_URL=https://xxx
- BASEROW_CADDY_ADDRESSES=https://xxx
I realized it’s poorly configurated

We were making approx 1 api call per sec with a filter on the mac address on the 256k row table.

It remained stuck at 50%. I was still unable to import csv after the restart of the instance. However, after the manipulation outlined in there : Investigate, document and prevent or fix postgres database corruption when two all-in-one images run at the same time with same volume (#1524) · Issues · Baserow / baserow · GitLab I was able to start again the import.
BUT, it is now stuck again although I used a small file (212Ko) .

Also, now one of my table displays this error message

View not found.

The page you are looking for has not been found. This might be because URL is incorrect or that you don’t have permission to view this page.

(I should have permissions as I am the only user )

I feel like the database has issue but this is a bit of a black box to me.

Please help

Rob192 · October 24, 2023, 8:29am

Regarding the error for the table 323 that I cannot reach anymore, I have the following log when I try to send data to it :


[POSTGRES][2023-10-24 08:24:31] 2023-10-24 08:23:01.866 UTC [61836] baserow@baserow STATEMENT:  INSERT INTO "database_table_323" ("created_on", "updated_on", "trashed", "field_2668", "field_2672", "field_2673", "field_2674", "field_2675", "field_2676", "field_2677", "field_2678", "field_2679", "field_2693", "field_2835", "order", "tsv_field_2668", "tsv_field_2672", "tsv_field_2673", "tsv_field_2674", "tsv_field_2675", "tsv_field_2676", "tsv_field_2677", "tsv_field_2678", "tsv_field_2679", "tsv_field_2693", "tsv_field_2835", "needs_background_update") VALUES ('2023-10-24T08:23:01.612979+00:00'::timestamptz, '2023-10-24T08:23:01.612979+00:00'::timestamptz, false, NULL, '2023-10-24T08:22:54.079972+00:00'::timestamptz, 1698135774.079972, 'E5:45:E0:B8:5E:EE', '{''result'': [''7.59'', ''37.16'', ''19.81'', ''97.99'', ''27.22'', ''171.15'', ''5.19'', ''60.30'', ''26.40'', ''236.92'', ''13.89'', ''334.53''], ''data'': [[0.011255025863647461, 0.033661842346191406, 0.04976391792297363, 0.07225799560546875, **.... BIG CHUNK OF DATA **
 [BACKEND][2023-10-24 08:24:31] 113.118.233.239:0 - "POST /api/database/rows/table/317/?user_field_names=true HTTP/1.1" 200  
 [BACKEND][2023-10-24 08:24:31] ERROR 2023-10-24 08:24:31,170 django.request.log_response:224- Internal Server Error: /api/database/views/grid/1213/   
 [BACKEND][2023-10-24 08:24:31] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31] psycopg2.errors.InternalError_: missing chunk number 32 for toast value 378150 in pg_toast_31140  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] The above exception was the direct cause of the following exception:  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler  
 [BACKEND][2023-10-24 08:24:31]     raise exc_info[1]  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 38, in inner  
 [BACKEND][2023-10-24 08:24:31]     response = await get_response(request)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 233, in _get_response_async  
 [BACKEND][2023-10-24 08:24:31]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__  
 [BACKEND][2023-10-24 08:24:31]     ret = await asyncio.wait_for(future, timeout=None)  
 [BACKEND][2023-10-24 08:24:31]   File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for  
 [BACKEND][2023-10-24 08:24:31]     return await fut  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run  
 [BACKEND][2023-10-24 08:24:31]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view  
 [BACKEND][2023-10-24 08:24:31]     return view_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view  
 [BACKEND][2023-10-24 08:24:31]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2023-10-24 08:24:31]     response = self.handle_exception(exc)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2023-10-24 08:24:31]     self.raise_uncaught_exception(exc)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2023-10-24 08:24:31]     raise exc  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2023-10-24 08:24:31]     response = handler(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 107, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 340, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 172, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/contrib/database/api/views/grid/views.py", line 289, in get  
 [BACKEND][2023-10-24 08:24:31]     page = paginator.paginate_queryset(queryset, request, self)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/pagination.py", line 395, in paginate_queryset  
 [BACKEND][2023-10-24 08:24:31]     return list(queryset[self.offset:self.offset + self.limit])  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 262, in __len__  
 [BACKEND][2023-10-24 08:24:31]     self._fetch_all()  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all  
 [BACKEND][2023-10-24 08:24:31]     self._result_cache = list(self._iterable_class(self))  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__  
 [BACKEND][2023-10-24 08:24:31]     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql  
 [BACKEND][2023-10-24 08:24:31]     cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute  
 [BACKEND][2023-10-24 08:24:31]     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers  
 [BACKEND][2023-10-24 08:24:31]     return executor(sql, params, many, context)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__  
 [BACKEND][2023-10-24 08:24:31]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31] django.db.utils.InternalError: missing chunk number 32 for toast value 378150 in pg_toast_31140  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] ERROR 2023-10-24 08:24:31,170 django.request.log_response:224- Internal Server Error: /api/database/views/grid/1213/   
 [BACKEND][2023-10-24 08:24:31] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31] psycopg2.errors.InternalError_: missing chunk number 32 for toast value 378150 in pg_toast_31140  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] The above exception was the direct cause of the following exception:  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler  
 [BACKEND][2023-10-24 08:24:31]     raise exc_info[1]  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 38, in inner  
 [BACKEND][2023-10-24 08:24:31]     response = await get_response(request)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 233, in _get_response_async  
 [BACKEND][2023-10-24 08:24:31]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__  
 [BACKEND][2023-10-24 08:24:31]     ret = await asyncio.wait_for(future, timeout=None)  
 [BACKEND][2023-10-24 08:24:31]   File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for  
 [BACKEND][2023-10-24 08:24:31]     return await fut  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run  
 [BACKEND][2023-10-24 08:24:31]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view  
 [BACKEND][2023-10-24 08:24:31]     return view_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view  
 [BACKEND][2023-10-24 08:24:31]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2023-10-24 08:24:31]     response = self.handle_exception(exc)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2023-10-24 08:24:31]     self.raise_uncaught_exception(exc)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2023-10-24 08:24:31]     raise exc  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2023-10-24 08:24:31]     response = handler(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 107, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 340, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 172, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/contrib/database/api/views/grid/views.py", line 289, in get  
 [BACKEND][2023-10-24 08:24:31]     page = paginator.paginate_queryset(queryset, request, self)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/pagination.py", line 395, in paginate_queryset  
 [BACKEND][2023-10-24 08:24:31]     return list(queryset[self.offset:self.offset + self.limit])  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 262, in __len__  
 [BACKEND][2023-10-24 08:24:31]     self._fetch_all()  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all  
 [BACKEND][2023-10-24 08:24:31]     self._result_cache = list(self._iterable_class(self))  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__  
 [BACKEND][2023-10-24 08:24:31]     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql  
 [BACKEND][2023-10-24 08:24:31]     cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute  
 [BACKEND][2023-10-24 08:24:31]     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers  
 [BACKEND][2023-10-24 08:24:31]     return executor(sql, params, many, context)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__  
 [BACKEND][2023-10-24 08:24:31]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31] django.db.utils.InternalError: missing chunk number 32 for toast value 378150 in pg_toast_31140  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] ERROR 2023-10-24 08:24:31,170 django.request.log_response:224- Internal Server Error: /api/database/views/grid/1213/   
 [BACKEND][2023-10-24 08:24:31] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31] psycopg2.errors.InternalError_: missing chunk number 32 for toast value 378150 in pg_toast_31140  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] The above exception was the direct cause of the following exception:  
 [BACKEND][2023-10-24 08:24:31]   
 [BACKEND][2023-10-24 08:24:31] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler  
 [BACKEND][2023-10-24 08:24:31]     raise exc_info[1]  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 38, in inner  
 [BACKEND][2023-10-24 08:24:31]     response = await get_response(request)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 233, in _get_response_async  
 [BACKEND][2023-10-24 08:24:31]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__  
 [BACKEND][2023-10-24 08:24:31]     ret = await asyncio.wait_for(future, timeout=None)  
 [BACKEND][2023-10-24 08:24:31]   File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for  
 [BACKEND][2023-10-24 08:24:31]     return await fut  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run  
 [BACKEND][2023-10-24 08:24:31]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view  
 [BACKEND][2023-10-24 08:24:31]     return view_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view  
 [BACKEND][2023-10-24 08:24:31]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2023-10-24 08:24:31]     response = self.handle_exception(exc)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2023-10-24 08:24:31]     self.raise_uncaught_exception(exc)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2023-10-24 08:24:31]     raise exc  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2023-10-24 08:24:31]     response = handler(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 107, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 340, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/api/decorators.py", line 172, in func_wrapper  
 [BACKEND][2023-10-24 08:24:31]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/backend/src/baserow/contrib/database/api/views/grid/views.py", line 289, in get  
 [BACKEND][2023-10-24 08:24:31]     page = paginator.paginate_queryset(queryset, request, self)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/pagination.py", line 395, in paginate_queryset  
 [BACKEND][2023-10-24 08:24:31]     return list(queryset[self.offset:self.offset + self.limit])  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 262, in __len__  
 [BACKEND][2023-10-24 08:24:31]     self._fetch_all()  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1324, in _fetch_all  
 [BACKEND][2023-10-24 08:24:31]     self._result_cache = list(self._iterable_class(self))  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 51, in __iter__  
 [BACKEND][2023-10-24 08:24:31]     results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1175, in execute_sql  
 [BACKEND][2023-10-24 08:24:31]     cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute  
 [BACKEND][2023-10-24 08:24:31]     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers  
 [BACKEND][2023-10-24 08:24:31]     return executor(sql, params, many, context)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__  
 [BACKEND][2023-10-24 08:24:31]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2023-10-24 08:24:31]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:24:31]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:24:31] django.db.utils.InternalError: missing chunk number 32 for toast value 378150 in pg_toast_31140
[POSTGRES][2023-10-24 08:24:31] 2023-10-24 08:24:31.169 UTC [62012] baserow@baserow ERROR:  missing chunk number 32 for toast value 378150 in pg_toast_31140  

**BIG CHUNK OF DATA CONTINUES** 3.313476821129692]], ''param'': {''T05_pressure'': [[0, 20], [25, 65], [0, 40], [75, 125], [0, 50], [135, 200], [0, 20], [35, 80], [0, 50], [190, 265], [0, 40], [320, 380]], ''T30_pressure'': [[0, 20], [35, 60], [0, 35], [80, 100], [0, 45], [135, 160], [0, 20], [55, 80], [0, 35], [150, 175], [0, 45], [280, 310]], ''T10_board_info'': {''board_version'': ''S00P00'', ''board'': 0, ''batch'': 11}, ''Serial_port'': {}, ''pump_cycles'': {''initial_cycles_nb'': 5, ''test_cycles_nb'': 5, ''nb_peaks_to_avg'': 5}, ''JFlash'': {''macos_path'': ''/Applications/SEGGER/JLink_V780c/JFlash.app'', ''win_path'': ''C:\\Program Files\\SEGGER\\JLink\\JFlash.exe''}, ''opto_calibration_distance'': 40, ''opto_xtalk_calib_value'': 0, ''serial_port'': {''port_breast_pump'': ''COM8'', ''port_bench'': ''COM6''}}}', 'V41', 'PerifitPump_V32', '2310b00113573', 334.530000, true, '0011', 26719.00000000000000000000, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, true) RETURNING "database_table_323"."id"

Rob192 · October 24, 2023, 8:37am

And also this log

POSTGRES][2023-10-24 08:35:15] 2023-10-24 08:33:27.424 UTC [63099] baserow@baserow STATEMENT:  SELECT "database_table_323"."id", "database_table_323"."created_on", "database_table_323"."updated_on", "database_table_323"."trashed", "database_table_323"."field_2668", "database_table_323"."field_2672", "database_table_323"."field_2673", "database_table_323"."field_2674", "database_table_323"."field_2675", "database_table_323"."field_2676", "database_table_323"."field_2677", "database_table_323"."field_2678", "database_table_323"."field_2679", "database_table_323"."field_2693", "database_table_323"."field_2835", "database_table_323"."order", "database_table_323"."needs_background_update" FROM "database_table_323" WHERE NOT "database_table_323"."trashed" ORDER BY "database_table_323"."field_2672" DESC NULLS LAST, "database_table_323"."order" ASC NULLS FIRST, "database_table_323"."id" ASC NULLS FIRST LIMIT 80  
 [POSTGRES][2023-10-24 08:35:15] 2023-10-24 08:35:15.772 UTC [63344] baserow@baserow ERROR:  failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15] 127.0.0.1:51612 - "GET /api/_health/ HTTP/1.1" 200  
 [BACKEND][2023-10-24 08:35:15] ERROR 2023-10-24 08:35:15,777 django.request.log_response:224- Internal Server Error: /api/database/rows/table/323/   
 [BACKEND][2023-10-24 08:35:15] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15] psycopg2.errors.InternalError_: failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] The above exception was the direct cause of the following exception:  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler  
 [BACKEND][2023-10-24 08:35:15]     raise exc_info[1]  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 38, in inner  
 [BACKEND][2023-10-24 08:35:15]     response = await get_response(request)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 233, in _get_response_async  
 [BACKEND][2023-10-24 08:35:15]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__  
 [BACKEND][2023-10-24 08:35:15]     ret = await asyncio.wait_for(future, timeout=None)  
 [BACKEND][2023-10-24 08:35:15]   File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for  
 [BACKEND][2023-10-24 08:35:15]     return await fut  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run  
 [BACKEND][2023-10-24 08:35:15]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view  
 [BACKEND][2023-10-24 08:35:15]     return view_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view  
 [BACKEND][2023-10-24 08:35:15]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2023-10-24 08:35:15]     response = self.handle_exception(exc)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2023-10-24 08:35:15]     self.raise_uncaught_exception(exc)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2023-10-24 08:35:15]     raise exc  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2023-10-24 08:35:15]     response = handler(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/usr/lib/python3.9/contextlib.py", line 79, in inner  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwds)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/api/decorators.py", line 107, in func_wrapper  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/api/decorators.py", line 172, in func_wrapper  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/api/rows/views.py", line 491, in post  
 [BACKEND][2023-10-24 08:35:15]     row = action_type_registry.get_by_type(CreateRowActionType).do(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/actions.py", line 73, in do  
 [BACKEND][2023-10-24 08:35:15]     row = RowHandler().create_row(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     raise ex  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 715, in create_row  
 [BACKEND][2023-10-24 08:35:15]     return self.force_create_row(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     raise ex  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 762, in force_create_row  
 [BACKEND][2023-10-24 08:35:15]     instance = model.objects.create(**row_values)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method  
 [BACKEND][2023-10-24 08:35:15]     return getattr(self.get_queryset(), name)(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 453, in create  
 [BACKEND][2023-10-24 08:35:15]     obj.save(force_insert=True, using=self.db)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 739, in save  
 [BACKEND][2023-10-24 08:35:15]     self.save_base(using=using, force_insert=force_insert,  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 776, in save_base  
 [BACKEND][2023-10-24 08:35:15]     updated = self._save_table(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 881, in _save_table  
 [BACKEND][2023-10-24 08:35:15]     results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 919, in _do_insert  
 [BACKEND][2023-10-24 08:35:15]     return manager._insert(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method  
 [BACKEND][2023-10-24 08:35:15]     return getattr(self.get_queryset(), name)(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1270, in _insert  
 [BACKEND][2023-10-24 08:35:15]     return query.get_compiler(using=using).execute_sql(returning_fields)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql  
 [BACKEND][2023-10-24 08:35:15]     cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute  
 [BACKEND][2023-10-24 08:35:15]     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers  
 [BACKEND][2023-10-24 08:35:15]     return executor(sql, params, many, context)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__  
 [BACKEND][2023-10-24 08:35:15]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15] django.db.utils.InternalError: failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] ERROR 2023-10-24 08:35:15,777 django.request.log_response:224- Internal Server Error: /api/database/rows/table/323/   
 [BACKEND][2023-10-24 08:35:15] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15] psycopg2.errors.InternalError_: failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] The above exception was the direct cause of the following exception:  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler  
 [BACKEND][2023-10-24 08:35:15]     raise exc_info[1]  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 38, in inner  
 [BACKEND][2023-10-24 08:35:15]     response = await get_response(request)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 233, in _get_response_async  
 [BACKEND][2023-10-24 08:35:15]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__  
 [BACKEND][2023-10-24 08:35:15]     ret = await asyncio.wait_for(future, timeout=None)  
 [BACKEND][2023-10-24 08:35:15]   File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for  
 [BACKEND][2023-10-24 08:35:15]     return await fut  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run  
 [BACKEND][2023-10-24 08:35:15]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view  
 [BACKEND][2023-10-24 08:35:15]     return view_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view  
 [BACKEND][2023-10-24 08:35:15]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2023-10-24 08:35:15]     response = self.handle_exception(exc)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2023-10-24 08:35:15]     self.raise_uncaught_exception(exc)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2023-10-24 08:35:15]     raise exc  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2023-10-24 08:35:15]     response = handler(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/usr/lib/python3.9/contextlib.py", line 79, in inner  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwds)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/api/decorators.py", line 107, in func_wrapper  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/api/decorators.py", line 172, in func_wrapper  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/api/rows/views.py", line 491, in post  
 [BACKEND][2023-10-24 08:35:15]     row = action_type_registry.get_by_type(CreateRowActionType).do(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/actions.py", line 73, in do  
 [BACKEND][2023-10-24 08:35:15]     row = RowHandler().create_row(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     raise ex  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 715, in create_row  
 [BACKEND][2023-10-24 08:35:15]     return self.force_create_row(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     raise ex  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 762, in force_create_row  
 [BACKEND][2023-10-24 08:35:15]     instance = model.objects.create(**row_values)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method  
 [BACKEND][2023-10-24 08:35:15]     return getattr(self.get_queryset(), name)(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 453, in create  
 [BACKEND][2023-10-24 08:35:15]     obj.save(force_insert=True, using=self.db)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 739, in save  
 [BACKEND][2023-10-24 08:35:15]     self.save_base(using=using, force_insert=force_insert,  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 776, in save_base  
 [BACKEND][2023-10-24 08:35:15]     updated = self._save_table(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 881, in _save_table  
 [BACKEND][2023-10-24 08:35:15]     results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 919, in _do_insert  
 [BACKEND][2023-10-24 08:35:15]     return manager._insert(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method  
 [BACKEND][2023-10-24 08:35:15]     return getattr(self.get_queryset(), name)(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1270, in _insert  
 [BACKEND][2023-10-24 08:35:15]     return query.get_compiler(using=using).execute_sql(returning_fields)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql  
 [BACKEND][2023-10-24 08:35:15]     cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute  
 [BACKEND][2023-10-24 08:35:15]     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers  
 [BACKEND][2023-10-24 08:35:15]     return executor(sql, params, many, context)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__  
 [BACKEND][2023-10-24 08:35:15]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15] django.db.utils.InternalError: failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] ERROR 2023-10-24 08:35:15,777 django.request.log_response:224- Internal Server Error: /api/database/rows/table/323/   
 [BACKEND][2023-10-24 08:35:15] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15] psycopg2.errors.InternalError_: failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] The above exception was the direct cause of the following exception:  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:15] Traceback (most recent call last):  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 486, in thread_handler  
 [BACKEND][2023-10-24 08:35:15]     raise exc_info[1]  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/exception.py", line 38, in inner  
 [BACKEND][2023-10-24 08:35:15]     response = await get_response(request)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/core/handlers/base.py", line 233, in _get_response_async  
 [BACKEND][2023-10-24 08:35:15]     response = await wrapped_callback(request, *callback_args, **callback_kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 448, in __call__  
 [BACKEND][2023-10-24 08:35:15]     ret = await asyncio.wait_for(future, timeout=None)  
 [BACKEND][2023-10-24 08:35:15]   File "/usr/lib/python3.9/asyncio/tasks.py", line 442, in wait_for  
 [BACKEND][2023-10-24 08:35:15]     return await fut  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/current_thread_executor.py", line 22, in run  
 [BACKEND][2023-10-24 08:35:15]     result = self.fn(*self.args, **self.kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/asgiref/sync.py", line 490, in thread_handler  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view  
 [BACKEND][2023-10-24 08:35:15]     return view_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/views/generic/base.py", line 70, in view  
 [BACKEND][2023-10-24 08:35:15]     return self.dispatch(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch  
 [BACKEND][2023-10-24 08:35:15]     response = self.handle_exception(exc)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception  
 [BACKEND][2023-10-24 08:35:15]     self.raise_uncaught_exception(exc)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception  
 [BACKEND][2023-10-24 08:35:15]     raise exc  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch  
 [BACKEND][2023-10-24 08:35:15]     response = handler(request, *args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/usr/lib/python3.9/contextlib.py", line 79, in inner  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwds)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/api/decorators.py", line 107, in func_wrapper  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/api/decorators.py", line 172, in func_wrapper  
 [BACKEND][2023-10-24 08:35:15]     return func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/api/rows/views.py", line 491, in post  
 [BACKEND][2023-10-24 08:35:15]     row = action_type_registry.get_by_type(CreateRowActionType).do(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/actions.py", line 73, in do  
 [BACKEND][2023-10-24 08:35:15]     row = RowHandler().create_row(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     raise ex  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 715, in create_row  
 [BACKEND][2023-10-24 08:35:15]     return self.force_create_row(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 72, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     raise ex  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/core/telemetry/utils.py", line 68, in _wrapper  
 [BACKEND][2023-10-24 08:35:15]     result = wrapped_func(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/backend/src/baserow/contrib/database/rows/handler.py", line 762, in force_create_row  
 [BACKEND][2023-10-24 08:35:15]     instance = model.objects.create(**row_values)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method  
 [BACKEND][2023-10-24 08:35:15]     return getattr(self.get_queryset(), name)(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 453, in create  
 [BACKEND][2023-10-24 08:35:15]     obj.save(force_insert=True, using=self.db)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 739, in save  
 [BACKEND][2023-10-24 08:35:15]     self.save_base(using=using, force_insert=force_insert,  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 776, in save_base  
 [BACKEND][2023-10-24 08:35:15]     updated = self._save_table(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 881, in _save_table  
 [BACKEND][2023-10-24 08:35:15]     results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/base.py", line 919, in _do_insert  
 [BACKEND][2023-10-24 08:35:15]     return manager._insert(  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method  
 [BACKEND][2023-10-24 08:35:15]     return getattr(self.get_queryset(), name)(*args, **kwargs)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/query.py", line 1270, in _insert  
 [BACKEND][2023-10-24 08:35:15]     return query.get_compiler(using=using).execute_sql(returning_fields)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql  
 [BACKEND][2023-10-24 08:35:15]     cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute  
 [BACKEND][2023-10-24 08:35:15]     return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers  
 [BACKEND][2023-10-24 08:35:15]     return executor(sql, params, many, context)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__  
 [BACKEND][2023-10-24 08:35:15]     raise dj_exc_value.with_traceback(traceback) from exc_value  
 [BACKEND][2023-10-24 08:35:15]   File "/baserow/venv/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute  
 [BACKEND][2023-10-24 08:35:15]     return self.cursor.execute(sql, params)  
 [BACKEND][2023-10-24 08:35:15] django.db.utils.InternalError: failed to re-find parent key in index "pg_toast_31140_index" for split pages 5094/5095  
 [BACKEND][2023-10-24 08:35:15]   
 [BACKEND][2023-10-24 08:35:16] 78.202.149.100:0 - "POST /api/database/rows/table/323/?user_field_names=true HTTP/1.1" 500  
 [BACKEND][2023-10-24 08:35:22] 219.133.67.244:0 - "GET /api/database/rows/table/333/?user_field_names=true&filter__field_2740__equal=C0:AF:C6:1C:28:E4&filter__field_2741__equal=autoTestPressure&order_by=-Date&include=Date,Mac%20adresse,Test%20name,Pump&size=1 HTTP/1.1" 200

Rob192 · October 24, 2023, 8:39am

There might be something to do with this post : Postgresql error: failed to re-find parent key in index - Database Administrators Stack Exchange

REINDEX SYSTEM; -- run once
REINDEX DATABASE <your dbname>; -- run for each database
VACUUM (FULL VERBOSE ANALYZE); -- also run for each database after reindexing

How can I do that with the baserow database ?

bram · October 25, 2023, 8:01am

Hey @Rob192, I’m sorry to hear that things are not working as expected for you. I’ve looked into everything you said, and responded to your questions.

Separately, if you want to self-host Baserow worry free, I recommend looking into Cloudron (Installation on Cloudron // Baserow). It will allow you to easily install application, Baserow as well, make backups, do automated updates, etc.

We were making approx 1 api call per sec with a filter on the mac address on the 256k row table.

A 256k table can be quite a lot for a no-code database. The performance depends on how many fields that table, but also what kind of hardware you are running. Is there anything you can share about the type of CPU cores you’re using, how many fields you have in that table. Any insights in how your database is structured in Baserow could be useful.

It remained stuck at 50%. I was still unable to import csv after the restart of the instance. However, after the manipulation outlined in there : Investigate, document and prevent or fix postgres database corruption when two all-in-one images run at the same time with same volume (#1524) · Issues · Baserow / baserow · GitLab 2 I was able to start again the import.

How many minutes did the import already run for? By default, background jobs can’t run longer than 30 minutes. You can try to set the BASEROW_JOB_SOFT_TIME_LIMIT=3600 to increase it to 60 minutes.

Regarding the error for the table 323 that I cannot reach anymore, I have the following log when I try to send data to it :

Judging from the error message, it seems like your PostgreSQL database is somehow currupted. Maybe this resource can be of use to you: Postgres error: Missing chunk 0 for toast value in pg_toast · GitHub?

There might be something to do with this post : Postgresql error: failed to re-find parent key in index - Database Administrators Stack Exchange How can I do that with the baserow database ?

It depends a bit on how you’re self hosting, but this is something you would have to execute on the underlying PostgreSQL database. You would have to connect to it using a PostgreSQL client, and then execute those queries. More about that can be found here: database - Run PostgreSQL queries from the command line - Stack Overflow.

I hope this will help you with fixing your environment.

Rob192 · October 30, 2023, 8:52am

Is there anything you can share about the type of CPU cores you’re using, how many fields you have in that table. Any insights in how your database is structured in Baserow could be useful.

I am using a t2-medium instance (2 CPU, 4 GB RAM) with Ubuntu in AWS. My databases have approx 12 fields. One of them is used to store approx 100 Ko of text.

I am trying to move this data out of Baserow by doing API calls and storing the data in s3 bucket but it takes very long. Baserow often becomes “unhealthy”.

I am a bit lost in my options, when I am trying to do something it either takes for ages or does not improve the speed of my instance.

The reason I chose to self host was because I did not want to have the limitation on the number of rows. Is there a procedure to move the existing data of my instance to cloudtron ?

bram · October 30, 2023, 9:45am

It almost seems like your server can’t handle the load. 2CPU cores and 4GB of RAM should be enough to run a table with 12 fields and 256k rows. Are there any other applications running on the same server, or do you have any resource constraints defined? I don’t have any insights in your server, but it might be worth upgrading the hardware.

When making an API request, how long does it take for the server to respond?
How do you know it becomes unhealthy?
Can you share some of your access logs with me?
What happens if you set add the following environment variables?
- BASEROW_AMOUNT_OF_GUNICORN_WORKERS=5
- BASEROW_AMOUNT_OF_WORKERS=2

There is a procedure to migrate your data from your current instance to Cloudron. Depending on whether you want to keep your data structured or not, I’d first like to know the answers to the questions above.

Rob192 · October 30, 2023, 10:09am

Hello Bram,

Are there any other applications running on the same server, or do you have any resource constraints defined? → nothing else running (although I occasionally connect with vscode in ssh but remove the pid afterwards)
When making an API request, how long does it take for the server to respond? depends, usually 1 sec but for exemple while trying to send this request:
update_response = requests.patch(f"{API_ENDPOINT}{row_id}/?user_field_names=true",
headers=headers,
json=updated_row_data,
timeout=10)
the timeout is often hit!
How do you know it becomes unhealthy? I read the docker logs and it states “[BASEROW-WATCHER][2023-10-30 02:53:28] [32mWARNING: Baserow has become unhealthy.”
This is when I launch my script to update the rows and remove data (I have a sleep time of 5 sec between patch requests!)
Can you share some of your access logs with me? of course !
Here are some errors I recently noticed.
[POSTGRES][2023-10-30 09:51:23] 2023-10-30 09:44:47.163 UTC [87609] LOG: failed to re-find parent key in index “database_table_734_trashed_756d6321” for deletion target page 4
[POSTGRES][2023-10-30 09:51:23] 2023-10-30 09:51:23.047 UTC [88055] LOG: right sibling’s left-link doesn’t match: right sibling 30 of target 25 with leafblkno 25 and scanblkno 25 spuriously links to non-target 31 on level 0 of index “core_action_user_id_eed182ff”

[POSTGRES][2023-10-30 09:52:18] 2023-10-30 09:51:23.056 UTC [88055] LOG: no left sibling (concurrent deletion?) of block 31 in “core_action_user_id_eed182ff”
[POSTGRES][2023-10-30 09:52:18] 2023-10-30 09:52:18.901 UTC [86619] ERROR: right sibling of GIN page is of different type

link to log

What happens if you set add the following environment variables?
- BASEROW_AMOUNT_OF_GUNICORN_WORKERS=5
- BASEROW_AMOUNT_OF_WORKERS=2
  → I just changed that I will monitor !!

Rob192 · October 30, 2023, 10:52am

Hello ! I changed the env values but got the same issue:

see the logs :

bram · October 30, 2023, 1:41pm

Hey @Rob192, I think I know what might be causing your problems. Would you mind adding this environment variable: BASEROW_USE_PG_FULLTEXT_SEARCH=false? Since version 1.19 we’ve introduced an automatic full-text search index. You have a table with quite some rows, and after updating, Baserow tries to create the full-text search index for you in one query. This probably slows down your PostgreSQL so much that it doesn’t work anymore. I suspect it even times out, so it doesn’t complete the task.

Rob192 · October 30, 2023, 2:15pm

Hey @bram !
thanks a lot for your fast support
I did change that but I am still getting some PATCH requests not sent
See for exemple this log where the last api call I am able to successfully send is : “PATCH /api/database/rows/table/333/1006/?user_field_names=true HTTP/1.1”
Then I try to send “PATCH /api/database/rows/table/333/1009/?user_field_names=true HTTP/1.1”
but the 10s timeout of my request was triggered.

I am seeing a lot of these :
django.db.utils.InternalError: missing chunk number 32 for toast value 378150 in pg_toast_31140

so it may be my broken database again

bram · October 31, 2023, 11:16am

Hey @Rob192, I’m sorry to hear that you still have corrupted database problems. Unfortunately, this is a bit outside the scope of Baserow, as that could have been caused by hardware failure, or many other unrelated things. There are a couple of things you can do:

Reach out in a PostgreSQL community, and see if someone can help you.
Try to dump and restore your database.
Migrate to a different server.
Search online for solutions to that problem.

Rob192 · October 31, 2023, 1:13pm

Ok @bram,
Could you please point me to the procedure to migrate your data from your current instance to Cloudron ?
thanks

bram · November 1, 2023, 1:59pm

We don’t have a ready step-by-step procedure for this at the moment, but in your case it’s important that you can make a PostgreSQL database dump. The reason I’m asking is that your database is corrupted, and maybe this is not possible. Would you mind checking if you can do a pg_dump of your PostgreSQL database before I write the next steps out for you?

Rob192 · November 7, 2023, 9:55pm

Thanks for your answer Bram,
Yes I can do this pg_dump !

bram · November 8, 2023, 9:45am

That’s great, then it actually means you can actually move all your data into a Cloudron instance. This also means you don’t have to worry about your corrupted data. What I recommend doing is the following:

Create a pg_dump of the PostgreSQL database of your Baserow instance. Copy the exported file to your local machine.
Setup a new server, and install Cloudron on it: Cloudron - Install Cloudron.
Install Baserow on your Cloudron server: Baserow on Cloudron.
Use these instructions to import the PostgreSQL database into your Baserow instance running on Cloudron: Import PostgresSQL - Cloudron Docs.

All should then work as expected.

Rob192 · November 27, 2023, 12:51pm

Hello,

I managed to remove the errors in T05 with the script you shared to find the corrupted lines

Also some issues and delays where actually coming from the long data I was storing in one column type “Long Text”. In Baserow, are you planning to add for the jsonb field type ?