Apify Discord Mirror

Updated 2 months ago

Proxy configuration am I doing it wrong?

At a glance

The community member is using the PlaywrightCrawler from the crawlee library and is encountering an error related to the global proxy configuration. A community member suggests that the issue may be related to the version of crawlee being used, and provides an alternative solution of passing the proxy parameter directly to playwright.launch(). The community member using version 0.4.3 of crawlee confirms that the issue was resolved by creating a fresh project.

Useful resources

SShine

Plain Text

async def main() -> None:
    """The crawler entry point."""
    proxy_configuration = ProxyConfiguration(

        proxy_urls=[
            'https://xxx:xx/',
            'https://xxx:xx/',
        ]
    )
    crawler = PlaywrightCrawler(
        headless=False,
        request_handler=router,
        max_requests_per_crawl=50,
        proxy_configuration=proxy_configuration,

    )

I am using playwrightcrawler and when I use this code its showing error ->

Plain Text

 playwright._impl._api_types.Error: Browser needs to be launched with the global proxy. If all contexts override the proxy, global proxy will be never used and can be any string, for example "launch({ proxy: { server: 'http://per-context' } })"

4 comments

AApifyBot

@Shine just advanced to level 1! Thanks for your contributions! 🎉

MMantisus

what version of crawlee are you using? In 0.4.2 I have no problem with this.

Alternative. You can try passing the proxy parameter { server: 'http://per-context' } to playwright.launch as per this example - https://github.com/apify/crawlee-python/discussions/715

SShine

I am using 0.4.3 version

SShine

it worked after i created a fresh project

Add a reply