Apify Discord Mirror

Updated 2 months ago

Multiple instance - PlaywrightCrawler, is it possible?

At a glance
The post asks if there is any state being shared between instances of PlaywrightCrawler when creating a new instance using const crawler = new PlaywrightCrawler({}). The community members provide the following responses:

No, there is no state being shared between instances of PlaywrightCrawler when you create a new instance using const crawler = new PlaywrightCrawler({}). Each instance of PlaywrightCrawler operates independently and maintains its own internal state.

However, one community member noticed that if you add maxRequestsPerCrawl, this does impact multiple crawlers, suggesting it sets a global state.

There is no explicitly marked answer.
If I am calling const crawler = new PlaywrightCrawler({}) is there any state being shared between the instances?
E
J
2 comments
No, there is no state being shared between instances of PlaywrightCrawler when you create a new instance using const crawler = new PlaywrightCrawler({}). Each instance of PlaywrightCrawler operates independently and maintains its own internal state.
I noticed if you add maxRequestsPerCrawl this does impact multiple crawlers i.e. it seems to set a global state
Add a reply
Sign up and join the conversation on Discord