Apify and Crawlee Official Forum

Updated 3 months ago

Consultant services - Cloudflare Blocked

Does anyone know if Apify provide development consultant services?

We have a custom Indeed scraper, it's been running great for years, but now they have some advanced cloudflare protection in place. Our dev team have rewritten the Actor to use all the latest packages with best practices and recommendations as per the docs.

It runs successfully locally (using proxies) but when deployed to the platform we cannot bypass the protection.

We have also tried to use their recommended hosted indeed scraper and are also blocked.

An example URL would be: https://www.indeed.com/jobs?q=%22advanced+manufacturing%22+or+engineering+or+assembly+or+mechanical+or+automotive&l=New+Orleans%2C+LA&radius=50&jt=fulltime&sort=date
a
d
O
3 comments
I could help you out with it
How could you help?

Did you try to use Playwright + firefox?
It should help to bypass protection.

Also, You may find some inspiration here:
https://docs.apify.com/academy/anti-scraping
Add a reply
Sign up and join the conversation on Discord