Apify and Crawlee Official Forum

Updated 9 months ago

Long waiting time with proxy

Hello,
I'm using selenium + apify proxies.
general code idea:

Plain Text

                driver = await create_driver()
                Actor.log.info("Driver created")
                driver.get(url)
                Actor.log.info(f"Opened: {driver.current_url}")

In logs I can see

Plain Text

2024-02-14T10:42:49.648Z INFO  Launching Chrome WebDriver...
2024-02-14T10:42:50.709Z INFO  Driver created
2024-02-14T10:44:48.834Z INFO  Opened: <url>

So it looks like it took almost 2 minutes to open this page.

Why could it be?

4 comments

MMantisus

How long does it take without a proxy? Have you checked proxies from other countries?

Because one possible reason is that the site is waiting for some time to get the js scripts before it starts building the DOM tree. And if those scripts are blocked, then until that time expires, the DOM won't be built, which means the site won't return a response. This may well take a few minutes.

MMantisus

Here is an example of this behavior when running locally without a proxy

Attachment

DDanek

How long does it take without a proxy?

<1s

Have you checked proxies from other countries?

I only have access to free one so far

DDanek

it's facebook so I guess that's not geo problem

Add a reply