Baserow restoration issues

Sterfield · May 17, 2024, 6:17pm

Please fill in the questionnaire below.

Technical Help Questionnaire

Have you read and followed the instructions at: *READ ME FIRST* Technical Help FAQs - #2 by nigel ?

Answer: Yes

Self-Hosted Installation and Setup Questions

How have you self-hosted Baserow.

docker-compose.yml :

version: "3.4"

# See https://baserow.io/docs/installation%2Fconfiguration for more details on these
# backend environment variables, their defaults if left blank etc.
x-backend-variables: &backend-variables
  # Most users should only need to set these first four variables.
  SECRET_KEY: ${SECRET_KEY:?}
  BASEROW_JWT_SIGNING_KEY: ${BASEROW_JWT_SIGNING_KEY:-}
  DATABASE_PASSWORD: ${DATABASE_PASSWORD:?}
  REDIS_PASSWORD: ${REDIS_PASSWORD:?}
  # If you manually change this line make sure you also change the duplicate line in
  # the web-frontend service.
  BASEROW_PUBLIC_URL: ${BASEROW_PUBLIC_URL-http://localhost}

  # Set these if you want to use an external postgres instead of the db service below.
  DATABASE_USER: ${DATABASE_USER:-baserow}
  DATABASE_NAME: ${DATABASE_NAME:-baserow}
  DATABASE_HOST:
  DATABASE_PORT:
  DATABASE_OPTIONS:
  DATABASE_URL:

 .. / ..

  # Misc settings see https://baserow.io/docs/installation%2Fconfiguration for info
  BASEROW_AMOUNT_OF_WORKERS:
  BASEROW_ROW_PAGE_SIZE_LIMIT:
  BATCH_ROWS_SIZE_LIMIT:
  INITIAL_TABLE_DATA_LIMIT:
  BASEROW_FILE_UPLOAD_SIZE_LIMIT_MB:
  BASEROW_UNIQUE_ROW_VALUES_SIZE_LIMIT:

  BASEROW_EXTRA_ALLOWED_HOSTS:
  ADDITIONAL_APPS:
  BASEROW_PLUGIN_GIT_REPOS:
  BASEROW_PLUGIN_URLS:

  BASEROW_ENABLE_SECURE_PROXY_SSL_HEADER:
  MIGRATE_ON_STARTUP: ${MIGRATE_ON_STARTUP:-true}
  SYNC_TEMPLATES_ON_STARTUP: ${SYNC_TEMPLATES_ON_STARTUP:-true}
  DONT_UPDATE_FORMULAS_AFTER_MIGRATION:
  BASEROW_TRIGGER_SYNC_TEMPLATES_AFTER_MIGRATION:
  BASEROW_SYNC_TEMPLATES_TIME_LIMIT:

  BASEROW_BACKEND_DEBUG:
  BASEROW_BACKEND_LOG_LEVEL:
  FEATURE_FLAGS:
  BASEROW_ENABLE_OTEL:
  BASEROW_DEPLOYMENT_ENV:
  OTEL_EXPORTER_OTLP_ENDPOINT:
  OTEL_RESOURCE_ATTRIBUTES:
  POSTHOG_PROJECT_API_KEY:
  POSTHOG_HOST:

  PRIVATE_BACKEND_URL: http://backend:8000
  PUBLIC_BACKEND_URL:
  PUBLIC_WEB_FRONTEND_URL:
  MEDIA_URL:
  MEDIA_ROOT:

.. / ...

services:
  # A caddy reverse proxy sitting in-front of all the services. Responsible for routing
  # requests to either the backend or web-frontend and also serving user uploaded files
  # from the media volume.
  caddy:
    image: caddy:2
    restart: unless-stopped
    environment:
      # Controls what port the Caddy server binds to inside its container.
      BASEROW_CADDY_ADDRESSES: ${BASEROW_CADDY_ADDRESSES:-:80}
      PRIVATE_WEB_FRONTEND_URL: ${PRIVATE_WEB_FRONTEND_URL:-http://web-frontend:3000}
      PRIVATE_BACKEND_URL: ${PRIVATE_BACKEND_URL:-http://backend:8000}
    ports:
      - "${HOST_PUBLISH_IP:-0.0.0.0}:${WEB_FRONTEND_PORT:-80}:80"
      - "${HOST_PUBLISH_IP:-0.0.0.0}:${WEB_FRONTEND_SSL_PORT:-443}:443"
    volumes:
      - $PWD/Caddyfile:/etc/caddy/Caddyfile
      - media:/baserow/media
      - caddy_config:/config
      - caddy_data:/data
    networks:
      local:

  backend:
    image: baserow/backend:1.23.2
    restart: unless-stopped
    environment:
      <<: *backend-variables
    depends_on:
      - db
      - redis
    volumes:
      - media:/baserow/media
    networks:
      local:

  web-frontend:
    image: baserow/web-frontend:1.23.2
    restart: unless-stopped
    environment:
      BASEROW_PUBLIC_URL: ${BASEROW_PUBLIC_URL-http://localhost}
      PRIVATE_BACKEND_URL: ${PRIVATE_BACKEND_URL:-http://backend:8000}
      PUBLIC_BACKEND_URL:
      PUBLIC_WEB_FRONTEND_URL:
      BASEROW_DISABLE_PUBLIC_URL_CHECK:
      INITIAL_TABLE_DATA_LIMIT:
      DOWNLOAD_FILE_VIA_XHR:
      BASEROW_DISABLE_GOOGLE_DOCS_FILE_PREVIEW:
      HOURS_UNTIL_TRASH_PERMANENTLY_DELETED:
      DISABLE_ANONYMOUS_PUBLIC_VIEW_WS_CONNECTIONS:
      FEATURE_FLAGS:
      ADDITIONAL_MODULES:
      BASEROW_MAX_IMPORT_FILE_SIZE_MB:
      BASEROW_MAX_SNAPSHOTS_PER_GROUP:
      BASEROW_ENABLE_OTEL:
      BASEROW_DEPLOYMENT_ENV:
      BASEROW_OSS_ONLY:
      BASEROW_USE_PG_FULLTEXT_SEARCH:
      POSTHOG_PROJECT_API_KEY:
      POSTHOG_HOST:
      BASEROW_UNIQUE_ROW_VALUES_SIZE_LIMIT:
      BASEROW_ROW_PAGE_SIZE_LIMIT:
      BASEROW_BUILDER_DOMAINS:
      BASEROW_FRONTEND_SAME_SITE_COOKIE:
      SENTRY_DSN:
    depends_on:
      - backend
    networks:
      local:

  celery:
    image: baserow/backend:1.23.2
    restart: unless-stopped
    environment:
      <<: *backend-variables
    command: celery-worker
    # The backend image's baked in healthcheck defaults to the django healthcheck
    # override it to the celery one here.
    healthcheck:
      test: [ "CMD-SHELL", "/baserow/backend/docker/docker-entrypoint.sh celery-worker-healthcheck" ]
    depends_on:
      - backend
    volumes:
      - media:/baserow/media
    networks:
      local:

  celery-export-worker:
    image: baserow/backend:1.23.2
    restart: unless-stopped
    command: celery-exportworker
    environment:
      <<: *backend-variables
    # The backend image's baked in healthcheck defaults to the django healthcheck
    # override it to the celery one here.
    healthcheck:
      test: [ "CMD-SHELL", "/baserow/backend/docker/docker-entrypoint.sh celery-exportworker-healthcheck" ]
    depends_on:
      - backend
    volumes:
      - media:/baserow/media
    networks:
      local:

  celery-beat-worker:
    image: baserow/backend:1.23.2
    restart: unless-stopped
    command: celery-beat
    environment:
      <<: *backend-variables
    # See https://github.com/sibson/redbeat/issues/129#issuecomment-1057478237
    stop_signal: SIGQUIT
    depends_on:
      - backend
    volumes:
      - media:/baserow/media
    networks:
      local:

  db:
    image: postgres:11
    restart: unless-stopped
    environment:
      - POSTGRES_USER=${DATABASE_USER:-baserow}
      - POSTGRES_PASSWORD=${DATABASE_PASSWORD:?}
      - POSTGRES_DB=${DATABASE_NAME:-baserow}
    healthcheck:
      test: [ "CMD-SHELL", "su postgres -c \"pg_isready -U ${DATABASE_USER:-baserow}\"" ]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      local:
    volumes:
      - pgdata:/var/lib/postgresql/data

  redis:
    image: redis:6
    command: redis-server --requirepass ${REDIS_PASSWORD:?}
    healthcheck:
      test: [ "CMD", "redis-cli", "ping" ]
    networks:
      local:

  # By default, the media volume will be owned by root on startup. Ensure it is owned by
  # the same user that django is running as, so it can write user files.
  volume-permissions-fixer:
    image: bash:4.4
    command: chown 9999:9999 -R /baserow/media
    volumes:
      - media:/baserow/media
    networks:
      local:

volumes:
  pgdata:
  media:
  caddy_data:
  caddy_config:

networks:
  local:
    driver: bridge

.env :

SECRET_KEY=***************************
DATABASE_PASSWORD=******************
REDIS_PASSWORD=***************************
BASEROW_JWT_SIGNING_KEY=*****************************
BASEROW_PUBLIC_URL=http://localhost

What are the specs of the service or server you are using to host Baserow.

selfhosted baserow

tested on Linux mint 16 Go RAM 4 CPUs
also tested on Linux mint 32 Go 6 CPUs via Virtualbox > Ubuntu 20.2 16Go - 6 theads

Which version of Baserow are you using.

1.23.2
same issue with 1.24.2 and postgreSQL 11

How have you configured your self-hosted installation?

***_backend_1                    /usr/bin/tini -- /bin/bash ...   Up (healthy)                                                                           
***_caddy_1                      caddy run --config /etc/ca ...   Up                      2019/tcp, 127.0.0.1:443->443/tcp, 443/udp, 127.0.0.1:80->80/tcp
***_celery-beat-worker_1         /usr/bin/tini -- /bin/bash ...   Up (health: starting)                                                                  
***_celery-export-worker_1       /usr/bin/tini -- /bin/bash ...   Up (healthy)                                                                           
***_celery_1                     /usr/bin/tini -- /bin/bash ...   Up (healthy)                                                                           
***_db_1                         docker-entrypoint.sh postgres    Up (healthy)            5432/tcp                                                       
***_redis_1                      docker-entrypoint.sh redis ...   Up (healthy)            6379/tcp                                                       
***_volume-permissions-fixer_1   docker-entrypoint.sh chown ...   Exit 0                                                                                 
***_web-frontend_1               /usr/bin/tini -- /bin/bash ...   Up (healthy)            3000/tcp

What commands if any did you use to start your Baserow server?

PWD=$PWD HOST_PUBLISH_IP=127.0.0.1 docker-compose up -d

Describe the problem

I can’ restore baserow with this command :

docker-compose run -v ~/baserow_backups:/baserow/backups backend restore -f /baserow/backups/baserow_backup.tar.gz

WARNING: errors ignored on restore: 1088
CommandError: The back-up failed because of the failure of the following sub-command, please read the output of the failed command above to see what went wrong. 
The sub-command which failed was:
pg_restore --host=db --dbname=baserow --port=5432 --username=baserow -Fd --jobs=1 -w /tmp/tmphkiu4710/20240515-1-baserow_backup.tar.gz/everything_but_user_tables/
It failed with a return code of: 1
ERROR: 1

How many rows in total do you have in your Baserow tables?

about 9000

Please attach full logs from all of Baserow’s services

*****_db1_log:

pg_restore: [archiver (db)] Error from TOC entry 19198; 2606 18304 FK CONSTRAINT database_rollupfield database_rollupfield_through_field_id_a35fa95f_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_rollupfield_through_field_id_a35fa95f_fk_database_" for relation "database_rollupfield" already exists
    Command was: ALTER TABLE ONLY public.database_rollupfield
    ADD CONSTRAINT database_rollupfield_through_field_id_a35fa95f_fk_database_ FOREIGN KEY (through_field_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19240; 2606 18833 FK CONSTRAINT database_rowcomment_mentions database_rowcomment__rowcomment_id_dc56cf9b_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_rowcomment__rowcomment_id_dc56cf9b_fk_database_" for relation "database_rowcomment_mentions" already exists
    Command was: ALTER TABLE ONLY public.database_rowcomment_mentions
    ADD CONSTRAINT database_rowcomment__rowcomment_id_dc56cf9b_fk_database_ FOREIGN KEY (rowcomment_id) REFERENCES public.database_rowcomment(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19241; 2606 18838 FK CONSTRAINT database_rowcomment_mentions database_rowcomment_mentions_user_id_49a137b9_fk_auth_user_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_rowcomment_mentions_user_id_49a137b9_fk_auth_user_id" for relation "database_rowcomment_mentions" already exists
    Command was: ALTER TABLE ONLY public.database_rowcomment_mentions
    ADD CONSTRAINT database_rowcomment_mentions_user_id_49a137b9_fk_auth_user_id FOREIGN KEY (user_id) REFERENCES public.auth_user(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19227; 2606 18691 FK CONSTRAINT database_rowcomment database_rowcomment_table_id_5878d1c7_fk_database_table_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_rowcomment_table_id_5878d1c7_fk_database_table_id" for relation "database_rowcomment" already exists
    Command was: ALTER TABLE ONLY public.database_rowcomment
    ADD CONSTRAINT database_rowcomment_table_id_5878d1c7_fk_database_table_id FOREIGN KEY (table_id) REFERENCES public.database_table(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19228; 2606 18778 FK CONSTRAINT database_rowcomment database_rowcomment_user_id_d0408bb6_fk_auth_user_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_rowcomment_user_id_d0408bb6_fk_auth_user_id" for relation "database_rowcomment" already exists
    Command was: ALTER TABLE ONLY public.database_rowcomment
    ADD CONSTRAINT database_rowcomment_user_id_d0408bb6_fk_auth_user_id FOREIGN KEY (user_id) REFERENCES public.auth_user(id) DEFERRABLE INITIALLY DEFERRED;




pg_restore: [archiver (db)] Error from TOC entry 19119; 2606 17209 FK CONSTRAINT database_tablewebhook database_tablewebhook_table_id_bded0cad_fk_database_table_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_tablewebhook_table_id_bded0cad_fk_database_table_id" for relation "database_tablewebhook" already exists
    Command was: ALTER TABLE ONLY public.database_tablewebhook
    ADD CONSTRAINT database_tablewebhook_table_id_bded0cad_fk_database_table_id FOREIGN KEY (table_id) REFERENCES public.database_table(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19076; 2606 16671 FK CONSTRAINT database_textfield database_textfield_field_ptr_id_f87f8cd8_fk_database_field_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_textfield_field_ptr_id_f87f8cd8_fk_database_field_id" for relation "database_textfield" already exists
    Command was: ALTER TABLE ONLY public.database_textfield
    ADD CONSTRAINT database_textfield_field_ptr_id_f87f8cd8_fk_database_field_id FOREIGN KEY (field_ptr_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19091; 2606 16825 FK CONSTRAINT database_token database_token_user_id_09848757_fk_auth_user_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_token_user_id_09848757_fk_auth_user_id" for relation "database_token" already exists
    Command was: ALTER TABLE ONLY public.database_token
    ADD CONSTRAINT database_token_user_id_09848757_fk_auth_user_id FOREIGN KEY (user_id) REFERENCES public.auth_user(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19092; 2606 18230 FK CONSTRAINT database_token database_token_workspace_id_8319ebdf_fk_core_workspace_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_token_workspace_id_8319ebdf_fk_core_workspace_id" for relation "database_token" already exists
    Command was: ALTER TABLE ONLY public.database_token
    ADD CONSTRAINT database_token_workspace_id_8319ebdf_fk_core_workspace_id FOREIGN KEY (workspace_id) REFERENCES public.core_workspace(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19093; 2606 16833 FK CONSTRAINT database_tokenpermission database_tokenpermis_database_id_fcb9f74f_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_tokenpermis_database_id_fcb9f74f_fk_database_" for relation "database_tokenpermission" already exists
    Command was: ALTER TABLE ONLY public.database_tokenpermission
    ADD CONSTRAINT database_tokenpermis_database_id_fcb9f74f_fk_database_ FOREIGN KEY (database_id) REFERENCES public.database_database(application_ptr_id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19094; 2606 16838 FK CONSTRAINT database_tokenpermission database_tokenpermission_table_id_52facb2e_fk_database_table_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_tokenpermission_table_id_52facb2e_fk_database_table_id" for relation "database_tokenpermission" already exists
    Command was: ALTER TABLE ONLY public.database_tokenpermission
    ADD CONSTRAINT database_tokenpermission_table_id_52facb2e_fk_database_table_id FOREIGN KEY (table_id) REFERENCES public.database_table(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19095; 2606 16843 FK CONSTRAINT database_tokenpermission database_tokenpermission_token_id_d06a00dc_fk_database_token_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_tokenpermission_token_id_d06a00dc_fk_database_token_id" for relation "database_tokenpermission" already exists
    Command was: ALTER TABLE ONLY public.database_tokenpermission
    ADD CONSTRAINT database_tokenpermission_token_id_d06a00dc_fk_database_token_id FOREIGN KEY (token_id) REFERENCES public.database_token(id) DEFERRABLE INITIALLY DEFERRED;

... / ...

pg_restore: [archiver (db)] Error from TOC entry 19200; 2606 18345 FK CONSTRAINT database_viewgroupby database_viewgroupby_field_id_a79fc066_fk_database_field_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_viewgroupby_field_id_a79fc066_fk_database_field_id" for relation "database_viewgroupby" already exists
    Command was: ALTER TABLE ONLY public.database_viewgroupby
    ADD CONSTRAINT database_viewgroupby_field_id_a79fc066_fk_database_field_id FOREIGN KEY (field_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;


pg_restore: [archiver (db)] Error from TOC entry 19201; 2606 18350 FK CONSTRAINT database_viewgroupby database_viewgroupby_view_id_24658e5d_fk_database_view_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_viewgroupby_view_id_24658e5d_fk_database_view_id" for relation "database_viewgroupby" already exists
    Command was: ALTER TABLE ONLY public.database_viewgroupby
    ADD CONSTRAINT database_viewgroupby_view_id_24658e5d_fk_database_view_id FOREIGN KEY (view_id) REFERENCES public.database_view(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19088; 2606 16780 FK CONSTRAINT database_viewsort database_viewsort_field_id_e51d1ca2_fk_database_field_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_viewsort_field_id_e51d1ca2_fk_database_field_id" for relation "database_viewsort" already exists
    Command was: ALTER TABLE ONLY public.database_viewsort
    ADD CONSTRAINT database_viewsort_field_id_e51d1ca2_fk_database_field_id FOREIGN KEY (field_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19089; 2606 16785 FK CONSTRAINT database_viewsort database_viewsort_view_id_2e9d197f_fk_database_view_id baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "database_viewsort_view_id_2e9d197f_fk_database_view_id" for relation "database_viewsort" already exists
    Command was: ALTER TABLE ONLY public.database_viewsort
    ADD CONSTRAINT database_viewsort_view_id_2e9d197f_fk_database_view_id FOREIGN KEY (view_id) REFERENCES public.database_view(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19301; 2606 1557846 FK CONSTRAINT integrations_localbaserowintegration integrations_localba_authorized_user_id_8dd67891_fk_auth_user baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_authorized_user_id_8dd67891_fk_auth_user" for relation "integrations_localbaserowintegration" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowintegration
    ADD CONSTRAINT integrations_localba_authorized_user_id_8dd67891_fk_auth_user FOREIGN KEY (authorized_user_id) REFERENCES public.auth_user(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19313; 2606 1557910 FK CONSTRAINT integrations_localbaserowtableservicefieldmapping integrations_localba_field_id_1f7d5505_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_field_id_1f7d5505_fk_database_" for relation "integrations_localbaserowtableservicefieldmapping" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowtableservicefieldmapping
    ADD CONSTRAINT integrations_localba_field_id_1f7d5505_fk_database_ FOREIGN KEY (field_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19309; 2606 1557886 FK CONSTRAINT integrations_localbaserowtableservicefilter integrations_localba_field_id_53e82940_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_field_id_53e82940_fk_database_" for relation "integrations_localbaserowtableservicefilter" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowtableservicefilter
    ADD CONSTRAINT integrations_localba_field_id_53e82940_fk_database_ FOREIGN KEY (field_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19311; 2606 1557898 FK CONSTRAINT integrations_localbaserowtableservicesort integrations_localba_field_id_f9dff360_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_field_id_f9dff360_fk_database_" for relation "integrations_localbaserowtableservicesort" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowtableservicesort
    ADD CONSTRAINT integrations_localba_field_id_f9dff360_fk_database_ FOREIGN KEY (field_id) REFERENCES public.database_field(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19302; 2606 1557841 FK CONSTRAINT integrations_localbaserowintegration integrations_localba_integration_ptr_id_c4976106_fk_core_inte baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_integration_ptr_id_c4976106_fk_core_inte" for relation "integrations_localbaserowintegration" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowintegration
    ADD CONSTRAINT integrations_localba_integration_ptr_id_c4976106_fk_core_inte FOREIGN KEY (integration_ptr_id) REFERENCES public.core_integration(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19310; 2606 1557891 FK CONSTRAINT integrations_localbaserowtableservicefilter integrations_localba_service_id_51c2e8fa_fk_core_serv baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_service_id_51c2e8fa_fk_core_serv" for relation "integrations_localbaserowtableservicefilter" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowtableservicefilter
    ADD CONSTRAINT integrations_localba_service_id_51c2e8fa_fk_core_serv FOREIGN KEY (service_id) REFERENCES public.core_service(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19312; 2606 1557903 FK CONSTRAINT integrations_localbaserowtableservicesort integrations_localba_service_id_d63d5b4c_fk_core_serv baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_service_id_d63d5b4c_fk_core_serv" for relation "integrations_localbaserowtableservicesort" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowtableservicesort
    ADD CONSTRAINT integrations_localba_service_id_d63d5b4c_fk_core_serv FOREIGN KEY (service_id) REFERENCES public.core_service(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19314; 2606 1557915 FK CONSTRAINT integrations_localbaserowtableservicefieldmapping integrations_localba_service_id_df624c41_fk_core_serv baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_service_id_df624c41_fk_core_serv" for relation "integrations_localbaserowtableservicefieldmapping" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowtableservicefieldmapping
    ADD CONSTRAINT integrations_localba_service_id_df624c41_fk_core_serv FOREIGN KEY (service_id) REFERENCES public.core_service(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19303; 2606 1557852 FK CONSTRAINT integrations_localbaserowgetrow integrations_localba_service_ptr_id_7e459844_fk_core_serv baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_service_ptr_id_7e459844_fk_core_serv" for relation "integrations_localbaserowgetrow" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowgetrow
    ADD CONSTRAINT integrations_localba_service_ptr_id_7e459844_fk_core_serv FOREIGN KEY (service_ptr_id) REFERENCES public.core_service(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19315; 2606 1557922 FK CONSTRAINT integrations_localbaserowupsertrow integrations_localba_service_ptr_id_9fd69568_fk_core_serv baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_service_ptr_id_9fd69568_fk_core_serv" for relation "integrations_localbaserowupsertrow" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowupsertrow
    ADD CONSTRAINT integrations_localba_service_ptr_id_9fd69568_fk_core_serv FOREIGN KEY (service_ptr_id) REFERENCES public.core_service(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19306; 2606 1557869 FK CONSTRAINT integrations_localbaserowlistrows integrations_localba_service_ptr_id_f2c14271_fk_core_serv baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_service_ptr_id_f2c14271_fk_core_serv" for relation "integrations_localbaserowlistrows" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowlistrows
    ADD CONSTRAINT integrations_localba_service_ptr_id_f2c14271_fk_core_serv FOREIGN KEY (service_ptr_id) REFERENCES public.core_service(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19307; 2606 1557879 FK CONSTRAINT integrations_localbaserowlistrows integrations_localba_table_id_21a3492c_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_table_id_21a3492c_fk_database_" for relation "integrations_localbaserowlistrows" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowlistrows
    ADD CONSTRAINT integrations_localba_table_id_21a3492c_fk_database_ FOREIGN KEY (table_id) REFERENCES public.database_table(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19316; 2606 1557927 FK CONSTRAINT integrations_localbaserowupsertrow integrations_localba_table_id_c9f8d243_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_table_id_c9f8d243_fk_database_" for relation "integrations_localbaserowupsertrow" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowupsertrow
    ADD CONSTRAINT integrations_localba_table_id_c9f8d243_fk_database_ FOREIGN KEY (table_id) REFERENCES public.database_table(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19304; 2606 1557862 FK CONSTRAINT integrations_localbaserowgetrow integrations_localba_table_id_f0a221b2_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_table_id_f0a221b2_fk_database_" for relation "integrations_localbaserowgetrow" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowgetrow
    ADD CONSTRAINT integrations_localba_table_id_f0a221b2_fk_database_ FOREIGN KEY (table_id) REFERENCES public.database_table(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19308; 2606 1557874 FK CONSTRAINT integrations_localbaserowlistrows integrations_localba_view_id_439c037e_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_view_id_439c037e_fk_database_" for relation "integrations_localbaserowlistrows" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowlistrows
    ADD CONSTRAINT integrations_localba_view_id_439c037e_fk_database_ FOREIGN KEY (view_id) REFERENCES public.database_view(id) DEFERRABLE INITIALLY DEFERRED;



pg_restore: [archiver (db)] Error from TOC entry 19305; 2606 1557857 FK CONSTRAINT integrations_localbaserowgetrow integrations_localba_view_id_bb09abe6_fk_database_ baserow
pg_restore: [archiver (db)] could not execute query: ERROR:  constraint "integrations_localba_view_id_bb09abe6_fk_database_" for relation "integrations_localbaserowgetrow" already exists
    Command was: ALTER TABLE ONLY public.integrations_localbaserowgetrow
    ADD CONSTRAINT integrations_localba_view_id_bb09abe6_fk_database_ FOREIGN KEY (view_id) REFERENCES public.database_view(id) DEFERRABLE INITIALLY DEFERRED;

****_redis_1 logs :

17 May 2024 16:27:18.286 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
17 May 2024 16:27:18.286 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started
17 May 2024 16:27:18.286 # Configuration loaded
17 May 2024 16:27:18.287 * monotonic clock: POSIX clock_gettime
17 May 2024 16:27:18.287 * Running mode=standalone, port=6379.
17 May 2024 16:27:18.287 # Server initialized

bram · May 23, 2024, 12:43pm

Hi @Sterfield, based on the output logs, it looks a like the tables already exist. Could it be that you’re trying to restore to a PostgreSQL database that is not empty? Maybe you’re trying to restore into the same database that you made your backup from.

Sterfield · May 27, 2024, 8:40am

Hi @bram, thank you for your response.
I took care to delete all the baserow volumes but I still get an error:
here is the content of logs:

WARNING: errors ignored on restore: 3
CommandError: The back-up failed because of the failure of the following sub-command, please read the output of the failed command above to see what went wrong. 
The sub-command which failed was:
pg_restore --host=db --dbname=baserow --port=5432 --username=baserow -Fd --jobs=1 -w /tmp/tmpn93vz4b6/20240524-1-baserow_backup.tar.gz/user_tables_batch_19
It failed with a return code of: 1
ERROR: 1

Here is the content of logs:

Attaching to guerineau_redis_1, guerineau_db_1
redis_1    1:C 27 May 2024 08:12:56.883 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1    1:C 27 May 2024 08:12:56.883 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1    1:C 27 May 2024 08:12:56.883 # Configuration loaded
redis_1    1:M 27 May 2024 08:12:56.884 * monotonic clock: POSIX clock_gettime
redis_1    1:M 27 May 2024 08:12:56.884 * Running mode=standalone, port=6379.
redis_1    1:M 27 May 2024 08:12:56.884 # Server initialized
redis_1    1:M 27 May 2024 08:12:56.884 * Ready to accept connections
db_1       The files belonging to this database system will be owned by user "postgres".
db_1       This user must also own the server process.
db_1       
db_1       The database cluster will be initialized with locale "en_US.utf8".
db_1       The default database encoding has accordingly been set to "UTF8".
db_1       The default text search configuration will be set to "english".
db_1       
db_1       Data page checksums are disabled.
db_1       
db_1       fixing permissions on existing directory /var/lib/postgresql/data ... ok
db_1       creating subdirectories ... ok
db_1       selecting default max_connections ... 100
db_1       selecting default shared_buffers ... 128MB
db_1       selecting default timezone ... Etc/UTC
db_1       selecting dynamic shared memory implementation ... posix
db_1       creating configuration files ... ok
db_1       running bootstrap script ... ok
db_1       performing post-bootstrap initialization ... ok
db_1       syncing data to disk ... ok
db_1       
db_1       Success. You can now start the database server using:
db_1       
db_1           pg_ctl -D /var/lib/postgresql/data -l logfile start
db_1       
db_1       
db_1       WARNING: enabling "trust" authentication for local connections
db_1       You can change this by editing pg_hba.conf or using the option -A, or
db_1       --auth-local and --auth-host, the next time you run initdb.
db_1       waiting for server to start....2024-05-27 08:12:59.738 UTC [48] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1       2024-05-27 08:12:59.768 UTC [49] LOG:  database system was shut down at 2024-05-27 08:12:57 UTC
db_1       2024-05-27 08:12:59.778 UTC [48] LOG:  database system is ready to accept connections
db_1        done
db_1       server started
db_1       CREATE DATABASE
db_1       
db_1       
db_1       /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
db_1       
db_1       waiting for server to shut down....2024-05-27 08:13:00.728 UTC [48] LOG:  received fast shutdown request
db_1       2024-05-27 08:13:00.735 UTC [48] LOG:  aborting any active transactions
db_1       2024-05-27 08:13:00.736 UTC [48] LOG:  background worker "logical replication launcher" (PID 55) exited with exit code 1
db_1       2024-05-27 08:13:00.736 UTC [50] LOG:  shutting down
db_1       2024-05-27 08:13:00.873 UTC [48] LOG:  database system is shut down
db_1        done
db_1       server stopped
db_1       
db_1       PostgreSQL init process complete; ready for start up.
db_1       
db_1       2024-05-27 08:13:00.947 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1       2024-05-27 08:13:00.947 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1       2024-05-27 08:13:00.963 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1       2024-05-27 08:13:00.989 UTC [76] LOG:  database system was shut down at 2024-05-27 08:13:00 UTC
db_1       2024-05-27 08:13:00.999 UTC [1] LOG:  database system is ready to accept connections
db_1       2024-05-27 08:18:15.350 UTC [386] ERROR:  relation "field_25311_seq" already exists
db_1       2024-05-27 08:18:15.350 UTC [386] STATEMENT:  CREATE SEQUENCE public.field_25311_seq
db_1       	    START WITH 1
db_1       	    INCREMENT BY 1
db_1       	    NO MINVALUE
db_1       	    NO MAXVALUE
db_1       	    CACHE 1;
db_1       	
db_1       	
db_1       	
db_1       2024-05-27 08:18:15.363 UTC [386] ERROR:  relation "field_25317_seq" already exists
db_1       2024-05-27 08:18:15.363 UTC [386] STATEMENT:  CREATE SEQUENCE public.field_25317_seq
db_1       	    START WITH 1
db_1       	    INCREMENT BY 1
db_1       	    NO MINVALUE
db_1       	    NO MAXVALUE
db_1       	    CACHE 1;
db_1       	
db_1       	
db_1       	
db_1       2024-05-27 08:18:15.377 UTC [386] ERROR:  relation "field_25333_seq" already exists
db_1       2024-05-27 08:18:15.377 UTC [386] STATEMENT:  CREATE SEQUENCE public.field_25333_seq
db_1       	    START WITH 1
db_1       	    INCREMENT BY 1
db_1       	    NO MINVALUE
db_1       	    NO MAXVALUE
db_1       	    CACHE 1;
db_1       	
db_1       	
db_1

bram · May 28, 2024, 8:49am

Hi @Sterfield, while trying to reproduce your problem with the import/export, I noticed that we’re running an outdated version of pg_dump and pg_restore. This will be fixed in the release of 1.25 tomorrow. After we’ve done the release, would you then mind trying again and see if the problem persists? If so, then I’ll take a deeper dive into it.

Sterfield · May 28, 2024, 10:01am

Hi @bram, I will wait for the new release and test it as soon as it is published, and let you know if the problem is resolved or not. I was unable to migrate to postgresql 15 (maybe for the same reason) ; I will therefore test version 1.25 with postgresql 11, if possible.

Sterfield · May 30, 2024, 8:23pm

Hi @bram, I performed various tests with baserow 1.25.0 and 1.25.1 but nothing that allowed me to restore the database.

Maybe I misunderstood how to proceed ?

Restores a Baserow database back_up file generated by the Baserow management command backup_baserow. Only restoring into existing blank databases
is supported, to restore over an existing database you must DROP and CREATE it yourself first.

Maybe I need to adapt this command to my configuration?

docker-compose run -v ~/baserow_backups:/baserow/backups backend restore -f /baserow/backups/baserow_backup.tar.gz

For the moment I back up my data by cloning the _media and pgdata folders into the docker volume folder.

I also cannot upgrade the PosgreSQL 11 database to PosgreSQL 15… perhaps this two problems are linked ? (I will create a separate post for this specific issue).

bram · June 3, 2024, 8:10am

Hi @Sterfield,

Maybe I misunderstood how to proceed ?
Maybe I need to adapt this command to my configuration?

It seems like the backup was successfully created. However, the problem is when restoring it that (some) tables already exist in PostgreSQL. The PostgreSQL database must be empty before attempting to restore. Can you verify that before you attempt to restore, the database is actually empty by running this database query: SELECT * FROM pg_catalog.pg_tables;?

I also cannot upgrade the PosgreSQL 11 database to PosgreSQL 15… perhaps this two problems are linked ? (I will create a separate post for this specific issue).

It looks like you’re using the multi serve docker compose approach. In this case, the PostgreSQL database is not embedded into one single image. If you would change the image to postgres:15, then it would fail to start because the data folder is for PG 11 and is incompatible. If you would like to upgrade to PG 15, then I recommend to stop your docker compose environment, change your database part to:

  db:
    image: pgautoupgrade/pgautoupgrade:15-alpine3.8

And start the environment again. This will automatically upgrade your data folder to PG 15.

I hope this helps