Does crawlee (python) support custom proxy rotation logic?

Question

Does crawlee support rotating proxies with custom logic, for better cost optimization, we would like to create/add new proxy urls only when the seeded urls failed. At the same time we don't interrupt the crawling due to proxy issues.If the above case is not supported, what's the best way to continue crawling from the break point/checkpoint, with out crawling all previously processed urls?

Exp · Answer

Hi, crawlee support rotating proxies

fyi7388 · Answer

With custom logic?

fyi7388 · Answer

My q was not about rotating already seeded proxy urls, but rather create new proxy urls on the fly and supply it when already existing urls fails.

Exp · Answer

of course, you can try to it

Apify and Crawlee Official Forum

Does crawlee (python) support custom proxy rotation logic?