The error is caused by the fact that
proxyConfiguration.newUrl()
returns a promise, so you must await it, like this:
const html = await cloudscraper.get({
uri: request.url,
proxy: proxyConfiguration ? await proxyConfiguration.newUrl() : undefined,
});
Moreover, when you configure the PlaywrightCrawler, there is no need to put your proxy into
launchContext.launchOptions
: you can just pass the
proxyConfiguration
object, like this:
const crawler = new PlaywrightCrawler({
requestQueue,
proxyConfiguration,
...
Hope this helps! π