# Api with large files - how to make it faster?

**URL:** <https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267>\
**Category:** Technical Help\
**Created:** [February 21, 2022, 4:12pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267 "2022-02-21T16:12:03Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 21, 2022, 4:12pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/1 "2022-02-21T16:12:03Z")

</div>

Hello,

I am using this machine:  
 ![image](https://community.baserow.io/uploads/default/original/1X/e72b0eaece4bd4ba79d7c6da8a1115073eb16fda.png)

And it is very slow to create rows with api (we are talking about 9000 rows). It creates one row every 3-4 seconds.

I try to do the same with your online website and it is much more faster.

Do you what I can do to improve the website?

I follow your install on ubuntu guide.

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 21, 2022, 5:16pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/2 "2022-02-21T17:16:09Z")

</div>

Hey @ozzz ,

How many fields are in the table you are creating rows? Are there lots of the more complicated field types like `lookup` , `formula` and `link row` fields?

Additionally could you install a process monitoring tool like `htop` on your machine and see how much utilization is occurring whilst you are inserting rows / provide a screenshot perhaps privately via PM?

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 21, 2022, 5:22pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/3 "2022-02-21T17:22:08Z")

</div>

Just saw you mentioned Large files in the title, are you uploading lots of files with each row? Also it might be useful to know the specific endpoint you are using to create rows with. Is it also slow when you create rows manually using the GUI?

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 21, 2022, 5:32pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/4 "2022-02-21T17:32:22Z")

</div>

Hello,

thanks for your answer.

Sorry or not beeing more precise.

I am uploading a 9000 rows into a table. They have 26 texts fields + 1 link row with only one relation in it.

I will install the htop and keep you in touch

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 21, 2022, 5:55pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/6 "2022-02-21T17:55:08Z")

</div>

Go ahead i’ve downloaded the image!

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 21, 2022, 5:57pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/7 "2022-02-21T17:57:04Z")

</div>

And so when you say you are uploading 9000 rows, are you doing that via the `Import table` modal:

 ![image](https://community.baserow.io/uploads/default/original/1X/bf07c32a2fa0695ad8fdb8ef2bf2cfd265115245.png)

Or are you using the API programmatically somehow?

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 21, 2022, 6:00pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/8 "2022-02-21T18:00:43Z")

</div>

I am using the api.

to be honest I am not importing a file, I am migrating data from a database to baserow.

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 21, 2022, 6:09pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/10 "2022-02-21T18:09:15Z")

</div>

Your data doesn’t look unreasonable at all, feel free to remove the link. We recently merged onto our develop branch a profiling tool that can be enabled in dev mode called `django-silk`. This might be useful to figure out what is causing the slowdown. Are you comfortable with temporally switching your Baserow onto the develop branch + in dev mode to debug this? If so I’ll post some instructions on how to do this tomorrow morning!

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 21, 2022, 6:14pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/11 "2022-02-21T18:14:31Z")

</div>

I am ok to join the beta program.

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 10:17am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/12 "2022-02-22T10:17:03Z")

</div>

What are the minimal technical requirements you use on your live version ?

Other question: how to upload csv, json or other in a way to create a link row?

thank you

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 22, 2022, 11:38am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/13 "2022-02-22T11:38:06Z")

</div>

Hey ozzz,

We don’t yet have official technical requirements.

Here is a quick and dirty guide on how to enable the profiling for your Baserow environment. I’ve not tested this thoroughly and so whilst it should be safe, please only do this if you are OK with potentially breaking your Baserow environment and/or data loss.

```bash
$ sudo -i
$ cd /baserow
$ source env/bin/activate
$ pip3 install django-silk==4.2.0

# Ensure you copy and paste and run this entire command including the final EOT all in one go.
$ cat <<EOT >> /baserow/baserow/backend/src/baserow/config/settings/base.py 
# Delete these following lines to disable the silk profiling program once done
INSTALLED_APPS += ["silk"] # noqa: F405

MIDDLEWARE += [ # noqa: F405
    "silk.middleware.SilkyMiddleware",
]

SILKY_ANALYZE_QUERIES = True
DEBUG = True
EOT

$ cat <<EOT >> /baserow/baserow/backend/src/baserow/config/urls.py 
from django.contrib.staticfiles.urls import staticfiles_urlpatterns

urlpatterns += [re_path("^silk/", include("silk.urls", namespace="silk"))]
urlpatterns += staticfiles_urlpatterns()
EOT

$ export DJANGO_SETTINGS_MODULE='baserow.config.settings.base'
$ export DATABASE_PASSWORD='yourpassword'
$ export DATABASE_HOST='localhost'
$ export REDIS_HOST='localhost'
$ baserow migrate

$ supervisorctl restart all

```

Now you should be able to visit [http://localhost:8000/silk/](http://localhost:8000/silk/) and see a dashboard of per request performance. If you are accessing your Baserow server remotely then you’ll need to change `http://localhost:8000` to whatever you’ve set the PUBLIC\_BACKEND\_URL to be.

Once you can see the /silk/ page. Try make the requests you find to be slow and then visit that page and you should see the slowest requests. Could you then send screenshots of the slowest requests and importantly the SQL sub page showing each query and how long it took?

Heres a gif showing this:  
 ![silky](https://community.baserow.io/uploads/default/original/1X/51bd302a91af10f9b24883ebf0695675d6fad7e0.gif)

Once we are done profiling your Baserow later you can revert the changes above by running

```auto
$ sudo -i
$ cd /baserow/baserow
$ git checkout -- . 
$ source /baserow/env/bin/activate
$ pip uninstall -f django-silk
$ supervisorctl restart all

```

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 22, 2022, 11:41am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/14 "2022-02-22T11:41:07Z")

</div>

To answer your question about uploading csvs to create link rows this is not yet possible.

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 11:49am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/15 "2022-02-22T11:49:48Z")

</div>

> [@nigel](#):
>
> `$ baserow migrate`

Hello got an error here. it is sad ‘path’ is not defined.

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 11:50am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/16 "2022-02-22T11:50:30Z")

</div>

```auto
File "/baserow/baserow/backend/src/baserow/config/urls.py", line 24, in <module>
    urlpatterns += [path("silk/", include("silk.urls", namespace="silk"))]
NameError: name 'path' is not defined

```

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 11:54am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/17 "2022-02-22T11:54:05Z")

</div>

is it re\_path instead of path?

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 22, 2022, 11:54am UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/18 "2022-02-22T11:54:06Z")

</div>

Ah my bad, could you reset by doing

```auto
cd /baserow/baserow
git checkout -- . 
cd /baserow

```

And try run the commands above again? I’ve updated them to fix the error

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 12:02pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/19 "2022-02-22T12:02:38Z")

</div>

here it is

 ![screencapture-api-data-dirparcours-av42-silk-request-1d602837-4a10-479e-849d-6c172980c54a-2022-02-22-12_02_06](https://community.baserow.io/uploads/default/original/1X/3fbdbdcd16efa73e8d636f9b581c3c83c8f00ab9.png)

The problem is that I have a lot of request like this to do.

---

<div class="post-metadata">

**Author:** ![nigel](https://community.baserow.io/user_avatar/community.baserow.io/nigel/32/51_2.png) [@nigel](https://community.baserow.io/u/nigel)\
**Post date:** [February 22, 2022, 12:09pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/20 "2022-02-22T12:09:52Z")

</div>

Could you show me the SQL tab on that page?

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 12:09pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/21 "2022-02-22T12:09:56Z")

</div>

![screencapture-api-data-dirparcours-av42-silk-request-91567de3-8f47-4a86-8458-3a6455ea9412-sql-2022-02-22-12_09_38](https://community.baserow.io/uploads/default/original/1X/a3e2caae46b7b76aaca226a349a377805cb97fdc.jpeg)

The right asked picture.

---

<div class="post-metadata">

**Author:** ![ozzz](https://community.baserow.io/letter_avatar_proxy/v4/letter/o/ecd19e/32.png) [@ozzz](https://community.baserow.io/u/ozzz)\
**Post date:** [February 22, 2022, 12:10pm UTC](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267/22 "2022-02-22T12:10:10Z")

</div>

I got 13 pages like that.

[Next page](https://community.baserow.io/t/api-with-large-files-how-to-make-it-faster/267.md?page=2)