Apify and Crawlee Official Forum

J
John
Offline, last seen 4 months ago
Joined August 30, 2024
Hello, I'm in trouble with Apify Proxy (on a free plan) I have a connexion refused. With and witout cloudscraper.

Any ideas please?


let proxyConfiguration;
if (useApifyProxy) {
proxyConfiguration = await Actor.createProxyConfiguration({ useApifyProxy: true });
} else if (proxyUrls.length > 0) {
proxyConfiguration = await Actor.createProxyConfiguration({ proxyUrls });
}

console.log(proxyConfiguration, proxyConfiguration.newUrl());

const crawler = new PlaywrightCrawler({
requestQueue,
launchContext: {
launchOptions: {
proxy: proxyConfiguration ? proxyConfiguration.newUrl() : undefined,
},
},
requestHandler: async ({ page, request }) => {

try {
// Use cloudscraper to fetch the page content
const html = await cloudscraper.get({
uri: request.url,
proxy: proxyConfiguration ? proxyConfiguration.newUrl() : undefined,
// Add other cloudscraper options as needed
});

// Process the HTML content as needed
const title = await page.title();
console.log("Page Title: " + title);

(...)
8 comments
M
A
J