The community member is experimenting to bypass Cloudflare security using Playwright, but is getting detected. Their IP is whitelisted, so they are asking for guidance on creating a fingerprint to bypass Cloudflare. The comments suggest using proxies, trying full (non-headless) mode or Playwright with Firefox, and adjusting Crawlee Playwright crawler configurations. The community members discuss that Crawlee might be closing the browser before Cloudflare can complete the bot detection, and provide a solution to add sessionPoolOptions: { blockedStatusCodes: [] } to the crawler options. The community members provide helpful feedback and suggestions to solve the issue.
Dear all, I am currently experimenting to bypass cloudflare Security. I am using playwright but getting dectected. My ip is whitelisted. Can any one of you please guide me how can I create a fingerprint which let me bypass cloudflare ?
But if my ip is already whitelisted then what difference a proxy could make ? In prod I will for sure use proxies but for testing am using my residential internet ip.
Are you using just playwright or crawlee playwright crawler? Crawlers already have fingerprints enabled by default. But may also want to try full (non-headless mode) or playwright + firefox combination
Thanks for the links. I think know the problem. Crawlee doesnt let the cloudflare complete the bot detection. It launches the browser, and before even cloudflare could check it, crawlee just close the browser.
oh yesss, I completely forgot about it. it should be addressed in one of the next releases - there apparently was some change in how cloudflare works. For now the fastest way to solve it would be to add sessionPoolOptions: { blockedStatusCodes: [] } to your crawler options
