Apify and Crawlee Official Forum
b
F
A
J
A
1,148
Apify Platform Forum
Crawlee JavaScript Forum
Crawlee for Python Forum
Sign up for Apify Platform here
Star Crawlee on GitHub
Star Crawlee for Python on GitHub
Powered by
Hall
Crawlee JavaScript Forum
0
Join
on Discord
How to increase max memory?
Crawlee JavaScript Forum
How to increase max memory?
0
Join
on Discord
J
Jeno
2 years ago
I am running a script that needs concurrency. I have 64 GB of RAM available and I want to use it to the max. I am running my script on a server so there is not much else running. The problem is, at around 15GB I always get memory overloaded error.
I have tried:
Plain Text
Copy
config.set('memoryMbytes', 50_000) config.set('availableMemoryRatio', 0.95)
Nothing seems to change this behavior. Anything else I can try?
P
1 comment
Share
P
Pepa J
2 years ago
Hello Jeno, based on
https://sdk.apify.com/api/apify/interface/ConfigurationOptions#memoryMbytes
it seems to me, that in your case, there is still the default memory limit being used.
Have you tried to run the script with
CRAWLEE_MEMORY_MBYTES
environmental variable being set?
Add a reply
Sign up and join the conversation on Discord
Join
on Discord