Once you’ve made the migrations, you need to actually execute them:
docker-compose -f docker-compose.dev.yml run --rm baserow-custom-number-field-plugin backend-cmd-with-db manage migrate
I’d suggest doing a quick tutorial on Django first perhaps as this will help you with writing a Baserow backend plugin as it’s just a Django app wrapped up and installed into Baserow.
@nigel Thanks for the answer.
I’ve tried some more times to create migrations but here is the problem with makemigrations command since migrations folder wasn’t created and the output for this command is “No changes detected”. It’s expected to see an error message that baserow_custom_number_field_plugin_integerfield table does not exist and running migrate command doesn’t make sense but anyway I’ve run it and nothing happed.
There are some plugins according to documentation but they were created 8 month ago.
Do you have any fresh examples?
In addition I didn’t do any changes/custom logic to code from Baserow documentation to brake something.
I don’t want to bother you with all this stuff but it doesn’t work and it’s important for me to create and use plugins for Baserow. Thanks in advance.
FYI: I’ve created migrations folder with “init.py” inside it manually and it helped me to have my first migration created so makemigrations command works fine but now there is an issue with migrate command. Please take a look on logs below.
docker-compose -f docker-compose.dev.yml run --rm baserow-custom-number-field-plugin backend-cmd-with-db manage migrate
WARNING: The PLUGIN_BUILD_UID variable is not set. Defaulting to a blank string.
WARNING: The PLUGIN_BUILD_GID variable is not set. Defaulting to a blank string.
Creating baserow-custom-number-field-plugin_baserow-custom-number-field-plugin_run ... done
[STARTUP][2023-03-02 10:38:10] No DATABASE_HOST or DATABASE_URL provided, using embedded postgres.
[STARTUP][2023-03-02 10:38:10] Using embedded baserow redis as no REDIS_HOST or REDIS_URL provided.
[STARTUP][2023-03-02 10:38:10] Importing REDIS_PASSWORD secret from /baserow/data/.redispass
[STARTUP][2023-03-02 10:38:10] Importing SECRET_KEY secret from /baserow/data/.secret
[STARTUP][2023-03-02 10:38:10] Importing BASEROW_JWT_SIGNING_KEY secret from /baserow/data/.jwt_signing_key
[STARTUP][2023-03-02 10:38:10] Importing DATABASE_PASSWORD secret from /baserow/data/.pgpass
[STARTUP][2023-03-02 10:38:10] Didn't find an existing postgres + redis running, starting them up now.
=========================================================================================
██████╗ █████╗ ███████╗███████╗██████╗ ██████╗ ██╗ ██╗
██╔══██╗██╔══██╗██╔════╝██╔════╝██╔══██╗██╔═══██╗██║ ██║
██████╔╝███████║███████╗█████╗ ██████╔╝██║ ██║██║ █╗ ██║
██╔══██╗██╔══██║╚════██║██╔══╝ ██╔══██╗██║ ██║██║███╗██║
██████╔╝██║ ██║███████║███████╗██║ ██║╚██████╔╝╚███╔███╔╝
╚═════╝ ╚═╝ ╚═╝╚══════╝╚══════╝╚═╝ ╚═╝ ╚═════╝ ╚══╝╚══╝
Version 1.14.0
=========================================================================================
Welcome to Baserow. See https://baserow.io/installation/install-with-docker/ for detailed instructions on
how to use this Docker image.
[STARTUP][2023-03-02 10:38:10] Running setup of embedded baserow database.
[POSTGRES_INIT][2023-03-02 10:38:10] Becoming postgres superuser to run setup SQL commands:
[POSTGRES_INIT][2023-03-02 10:38:10]
[POSTGRES_INIT][2023-03-02 10:38:10] PostgreSQL Database directory appears to contain a database; Skipping initialization
[POSTGRES_INIT][2023-03-02 10:38:10]
Error: Failed to connect to the postgresql database at localhost
Please see the error below for more details:
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (::1), port 5432 failed: Cannot assign requested address
Is the server running on that host and accepting TCP/IP connections?
[STARTUP][2023-03-02 10:38:10] No BASEROW_PUBLIC_URL environment variable provided. Starting baserow locally at http://localhost without automatic https.
Waiting for PostgreSQL to become available attempt 0/5 ...
[PLUGIN][SETUP] Found a plugin in /baserow/data/plugins/baserow_custom_number_field_plugin/, ensuring it is installed...
[PLUGIN][baserow_custom_number_field_plugin] Found a backend app for baserow_custom_number_field_plugin.
[PLUGIN][baserow_custom_number_field_plugin] Skipping install of baserow_custom_number_field_plugin's backend app as it is already installed.
[PLUGIN][baserow_custom_number_field_plugin] Running baserow_custom_number_field_plugin's custom runtime_setup.sh script
[PLUGIN][baserow_custom_number_field_plugin] Found a web-frontend module for baserow_custom_number_field_plugin.
[PLUGIN][baserow_custom_number_field_plugin] Skipping build of baserow_custom_number_field_plugin web-frontend module as it has already been built.
[PLUGIN][baserow_custom_number_field_plugin] Skipping runtime setup of baserow_custom_number_field_plugin's web-frontend module.
[PLUGIN][baserow_custom_number_field_plugin] Fixing ownership of plugins from 0 to baserow_docker_user in /baserow/data/plugins
[PLUGIN][baserow_custom_number_field_plugin] Finished setting up baserow_custom_number_field_plugin successfully.
[STARTUP][2023-03-02 10:38:12] Starting all Baserow processes:
2023-03-02 10:38:12,547 INFO Included extra file "/baserow/supervisor/includes/enabled/embedded-postgres.conf" during parsing
2023-03-02 10:38:12,547 INFO Included extra file "/baserow/supervisor/includes/enabled/embedded-postgres.conf" during parsing
2023-03-02 10:38:12,548 INFO Included extra file "/baserow/supervisor/includes/enabled/embedded-redis.conf" during parsing
2023-03-02 10:38:12,548 INFO Included extra file "/baserow/supervisor/includes/enabled/embedded-redis.conf" during parsing
2023-03-02 10:38:12,548 INFO Set uid to user 0 succeeded
2023-03-02 10:38:12,548 INFO Set uid to user 0 succeeded
2023-03-02 10:38:12,551 INFO supervisord started with pid 100
2023-03-02 10:38:12,551 INFO supervisord started with pid 100
Error: Failed to connect to the postgresql database at localhost
Please see the error below for more details:
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (::1), port 5432 failed: Cannot assign requested address
Is the server running on that host and accepting TCP/IP connections?
Waiting for PostgreSQL to become available attempt 1/5 ...
2023-03-02 10:38:13,555 INFO spawned: 'postgresql' with pid 227
2023-03-02 10:38:13,555 INFO spawned: 'postgresql' with pid 227
2023-03-02 10:38:13,557 INFO spawned: 'redis' with pid 228
2023-03-02 10:38:13,557 INFO spawned: 'redis' with pid 228
[REDIS][2023-03-02 10:38:13] 228:C 02 Mar 2023 10:38:13.603 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
[REDIS][2023-03-02 10:38:13] 228:C 02 Mar 2023 10:38:13.603 # Redis version=6.0.16, bits=64, commit=00000000, modified=0, pid=228, just started
[REDIS][2023-03-02 10:38:13] 228:C 02 Mar 2023 10:38:13.603 # Configuration loaded
[REDIS][2023-03-02 10:38:13] 228:M 02 Mar 2023 10:38:13.604 * Running mode=standalone, port=6379.
[REDIS][2023-03-02 10:38:13] 228:M 02 Mar 2023 10:38:13.604 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
[REDIS][2023-03-02 10:38:13] 228:M 02 Mar 2023 10:38:13.604 # Server initialized
[REDIS][2023-03-02 10:38:13] 228:M 02 Mar 2023 10:38:13.604 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
[REDIS][2023-03-02 10:38:13] 228:M 02 Mar 2023 10:38:13.604 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo madvise > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled (set to 'madvise' or 'never').
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.629 UTC [227] LOG: listening on IPv4 address "127.0.0.1", port 5432
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.629 UTC [227] LOG: could not bind IPv6 address "::1": Cannot assign requested address
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.629 UTC [227] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.634 UTC [227] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.654 UTC [253] LOG: database system was shut down at 2023-03-02 09:53:01 UTC
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.654 UTC [253] LOG: invalid resource manager ID 83 at 0/6931A48
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.654 UTC [253] LOG: invalid primary checkpoint record
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.654 UTC [253] PANIC: could not locate a valid checkpoint record
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.654 UTC [227] LOG: startup process (PID 253) was terminated by signal 6: Aborted
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.654 UTC [227] LOG: aborting startup due to startup process failure
[POSTGRES][2023-03-02 10:38:13] 2023-03-02 10:38:13.655 UTC [227] LOG: database system is shut down
2023-03-02 10:38:13,657 INFO exited: postgresql (exit status 1; not expected)
2023-03-02 10:38:13,657 INFO exited: postgresql (exit status 1; not expected)
2023-03-02 10:38:14,660 INFO spawned: 'postgresql' with pid 254
2023-03-02 10:38:14,660 INFO spawned: 'postgresql' with pid 254
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.727 UTC [254] LOG: listening on IPv4 address "127.0.0.1", port 5432
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.727 UTC [254] LOG: could not bind IPv6 address "::1": Cannot assign requested address
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.727 UTC [254] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.734 UTC [254] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] LOG: database system was shut down at 2023-03-02 09:53:01 UTC
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] LOG: invalid resource manager ID 83 at 0/6931A48
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] LOG: invalid primary checkpoint record
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] PANIC: could not locate a valid checkpoint record
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [254] LOG: startup process (PID 265) was terminated by signal 6: Aborted
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [254] LOG: aborting startup due to startup process failure
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.765 UTC [254] LOG: database system is shut down
2023-03-02 10:38:14,768 INFO exited: postgresql (exit status 1; not expected)
2023-03-02 10:38:14,768 INFO exited: postgresql (exit status 1; not expected)
Error: Failed to connect to the postgresql database at localhost
Please see the error below for more details:
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (::1), port 5432 failed: Cannot assign requested address
Is the server running on that host and accepting TCP/IP connections?
Waiting for PostgreSQL to become available attempt 2/5 ...
2023-03-02 10:38:16,772 INFO spawned: 'postgresql' with pid 268
2023-03-02 10:38:16,772 INFO spawned: 'postgresql' with pid 268
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.822 UTC [268] LOG: listening on IPv4 address "127.0.0.1", port 5432
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.822 UTC [268] LOG: could not bind IPv6 address "::1": Cannot assign requested address
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.822 UTC [268] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.827 UTC [268] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.848 UTC [279] LOG: database system was shut down at 2023-03-02 09:53:01 UTC
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.848 UTC [279] LOG: invalid resource manager ID 83 at 0/6931A48
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.848 UTC [279] LOG: invalid primary checkpoint record
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.848 UTC [279] PANIC: could not locate a valid checkpoint record
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.848 UTC [268] LOG: startup process (PID 279) was terminated by signal 6: Aborted
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.848 UTC [268] LOG: aborting startup due to startup process failure
[POSTGRES][2023-03-02 10:38:16] 2023-03-02 10:38:16.850 UTC [268] LOG: database system is shut down
2023-03-02 10:38:16,852 INFO exited: postgresql (exit status 1; not expected)
2023-03-02 10:38:16,852 INFO exited: postgresql (exit status 1; not expected)
Error: Failed to connect to the postgresql database at localhost
Please see the error below for more details:
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (::1), port 5432 failed: Cannot assign requested address
Is the server running on that host and accepting TCP/IP connections?
Waiting for PostgreSQL to become available attempt 3/5 ...
Error: Failed to connect to the postgresql database at localhost
Please see the error below for more details:
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (::1), port 5432 failed: Cannot assign requested address
Is the server running on that host and accepting TCP/IP connections?
Waiting for PostgreSQL to become available attempt 4/5 ...
2023-03-02 10:38:19,859 INFO spawned: 'postgresql' with pid 284
2023-03-02 10:38:19,859 INFO spawned: 'postgresql' with pid 284
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.918 UTC [284] LOG: listening on IPv4 address "127.0.0.1", port 5432
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.918 UTC [284] LOG: could not bind IPv6 address "::1": Cannot assign requested address
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.918 UTC [284] HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.926 UTC [284] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.951 UTC [295] LOG: database system was shut down at 2023-03-02 09:53:01 UTC
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.951 UTC [295] LOG: invalid resource manager ID 83 at 0/6931A48
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.951 UTC [295] LOG: invalid primary checkpoint record
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.951 UTC [295] PANIC: could not locate a valid checkpoint record
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.951 UTC [284] LOG: startup process (PID 295) was terminated by signal 6: Aborted
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.951 UTC [284] LOG: aborting startup due to startup process failure
[POSTGRES][2023-03-02 10:38:19] 2023-03-02 10:38:19.953 UTC [284] LOG: database system is shut down
2023-03-02 10:38:19,955 INFO exited: postgresql (exit status 1; not expected)
2023-03-02 10:38:19,955 INFO exited: postgresql (exit status 1; not expected)
2023-03-02 10:38:19,955 INFO gave up: postgresql entered FATAL state, too many start retries too quickly
2023-03-02 10:38:19,955 INFO gave up: postgresql entered FATAL state, too many start retries too quickly
Error: Failed to connect to the postgresql database at localhost
Please see the error below for more details:
connection to server at "localhost" (127.0.0.1), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
connection to server at "localhost" (::1), port 5432 failed: Cannot assign requested address
Is the server running on that host and accepting TCP/IP connections?
Waiting for PostgreSQL to become available attempt 5/5 ...
PostgreSQL did not become available in time...
[STARTUP][2023-03-02 10:38:23] ======== Cleaning up after Command ==========
2023-03-02 10:38:24,961 WARN received SIGTERM indicating exit request
2023-03-02 10:38:24,961 WARN received SIGTERM indicating exit request
2023-03-02 10:38:24,962 INFO waiting for redis to die
2023-03-02 10:38:24,962 INFO waiting for redis to die
[REDIS][2023-03-02 10:38:24] 228:M 02 Mar 2023 10:38:13.605 * Ready to accept connections
[REDIS][2023-03-02 10:38:25] 228:signal-handler (1677753504) Received SIGTERM scheduling shutdown...
[REDIS][2023-03-02 10:38:25] 228:M 02 Mar 2023 10:38:25.050 # User requested shutdown...
[REDIS][2023-03-02 10:38:25] 228:M 02 Mar 2023 10:38:25.051 # Redis is now ready to exit, bye bye...
2023-03-02 10:38:25,052 INFO stopped: redis (exit status 0)
2023-03-02 10:38:25,052 INFO stopped: redis (exit status 0)
ERROR: 1
Looks like your postgres database is corrupted:
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] LOG: database system was shut down at 2023-03-02 09:53:01 UTC
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] LOG: invalid resource manager ID 83 at 0/6931A48
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] LOG: invalid primary checkpoint record
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [265] PANIC: could not locate a valid checkpoint record
[POSTGRES][2023-03-02 10:38:14] 2023-03-02 10:38:14.762 UTC [254] LOG: startup process (PID 265) was terminated by signal 6: Aborted
You can completely delete this database and try re-create it by running if you are ok with loosing all of your baserow data in this dev environment:
docker-compose -f docker-compose.dev.yml down -v
After that one alternate and potentially easier method of running these django commands is by execing into your running Baserow backend django container by doing something like (I haven’t had a chance to manually test these work):
PLUGIN_BUILD_UID=$(id -u) PLUGIN_BUILD_GID=$(id -g) docker-compose -f docker-compose.dev.yml up -d --build
# wait for everything to launch
docker-compose -f docker-compose.dev.yml exec my-baserow-plugin /baserow.sh backend-cmd bash -c bash
cd /baserow/data/plugins/my_baserow_plugin/backend/
# now you can run all of the backend commands in this shell whilst the server is running
./baserow makemigrations
./baserow migrate
./baserow shell
And sorry for all the back and forth, we’ll take all of these pain points you’ve hit and fix up the boilerplate so its easier for the next person. Thanks for having the patience to go through them all with me.
And FYI here is the issue tracking the postgres database corruption, it happens when you run multiple Baserow containers at the same time pointing at the exact same postgres data volume, which is very easy to accidentally do using the backend-cmd-with-db command. Investigate, document and prevent or fix postgres database corruption when two all-in-one images run at the same time with same volume (#1524) · Issues · Bram Wiepjes / baserow · GitLab .
There is a command that recovers this database if this happens discussed on that issue.
@nigel It works. Thanks a lot for the help.
I had the same issue with my plugin (which I created on baserow 1.11 I believe, then subsequently upgraded to 1.12, 1.13.2, and now 1.15.2). The fix worked for me.