I immediately get captcha on every URL. Accessing it in a normal GUI browser typing site homepage URL: captcha. Searching this site in google, clicking on the link in google results: browser shows site address and...: captcha.
(by the way, they changed it, few months ago this site was not that restrictive)
Well... what is our best solution for sites always showing cloudflare captchas ?
Truly often they always check, even for regular users. However, you can tweak your scraper's configuration to try bypassing it. Generally, using Playwright + firefox with a residential proxy tends to work best, although it’s also the most expensive option.
