Apify

Apify and Crawlee Official Forum

b
F
A
J
A
Join

How to increase max memory?

JJeno
I am running a script that needs concurrency. I have 64 GB of RAM available and I want to use it to the max. I am running my script on a server so there is not much else running. The problem is, at around 15GB I always get memory overloaded error.

I have tried:
Plain Text
config.set('memoryMbytes', 50_000)
config.set('availableMemoryRatio', 0.95)

Nothing seems to change this behavior. Anything else I can try?
P
1 comment
PPepa J
Hello Jeno, based on https://sdk.apify.com/api/apify/interface/ConfigurationOptions#memoryMbytes it seems to me, that in your case, there is still the default memory limit being used.

Have you tried to run the script with CRAWLEE_MEMORY_MBYTES environmental variable being set?
Add a reply
Sign up and join the conversation on Discord
Join