I was able to get Baserow to successfully start by removing these lines from Caddyfile:
interval 2m
burst 5
My data appears to be missing, but that might be down to me accidentally deleting Docker volumes in the process of downgrading and/or reinstalling Docker.