Apify and Crawlee Official Forum

Updated 5 months ago

Efficiency / Cost management of multiple smaller actors vs. one large actor

At a glance
I'm using the clockworks tiktok scraper and had dilemna on whether to use 8 actors with 4gb RAM each, spreading out pages to scrape evenly accross the actors vs. 1 large 32gb actor for all the pages.

It seems like from trying out the scrape its far cheaper (not much faster though), to spread out the jobs accross 8 actors (we're looking at $10-12 vs. $3-4). I was adviced from the team to try one large actor since it should scale okay, but seems like my findings are different?

Am I doing something wrong here, or is it just that this scrape is better with me batching the jobs.
Attachment
image.png
L
1 comment
IT really depends on the actual actor and input so your tests make sense
Add a reply
Sign up and join the conversation on Discord