Apify

Apify and Crawlee Official Forum

b
F
A
J
A
Join

Download Delay

BBanul;
does crawlee support download delay? Like in Scrapy? Because I want to crawl a website but this website has delay before to load its content, so my current crawlee project didn't get the content of the website.
1
R
B
N
5 comments
RRado Ch.
hey, how about waiting for a selector?
BBanul;
I will scrape many and different website's so that why I'm not depending on a selector
NNeoNomade
maxRequestsPerMinute
OOleg V.
Use waitUntil() (https://crawlee.dev/api/playwright-crawler/namespace/playwrightUtils#waitUntil) Just find proper event to wait for.

Or You can use sleep() (https://crawlee.dev/api/utils/function/sleep) (not the best approach).
BBanul;
thanks for this, I tried both of these and set a usecase for each. Also I try the networkIdle
Add a reply
Sign up and join the conversation on Discord
Join