Apify and Crawlee Official Forum

Home
Members
tomasj97
t
tomasj97
Offline, last seen 4 weeks ago
Joined October 27, 2024
Hello, I would like to use some proxies with crawlee and I am curious how strong is free proxy from apify. Is it what is used in backend by default right now? When I s witch to standard plan I will feel the difference, and how I can implement it in python? Thanks!
1 comment
O
Hello, do you have some nice tips how to get rid out of 429? I am not exactly how parrarellism working here, but I am afraid, even I am putting in the sleep the process, the other parallel request as considered as request and they can lead to 429, is there any nice tip/best practice how can I defend against it ? 😄
1 comment
M
Hello, I would like to use PlayWrightCrawler for scraping , but it is not clear from the documentation how can I set up correctly concurrency, memory, cpu's, etc. Can someone help me out? What is the best practice to set up this Crawler to make scraping parallel? Thanks in advance!
1 comment
M