Apify and Crawlee Official Forum

Updated 3 months ago

Selenium WebDriverWait() not available?

Hello, I'm running a python Actor on a page that has some dynamic content that loads slowly. I'm using WebDriverWait() in my code that works outside of apify.

But when I run this code in apify I see this error message:

Plain Text
2024-05-05T19:36:27.317Z ERROR Cannot extract data from https://slow.example.com/.
2024-05-05T19:36:27.320Z       Traceback (most recent call last):
2024-05-05T19:36:27.322Z         File "/usr/src/app/src/main.py", line 74, in main
2024-05-05T19:36:27.324Z           ad_elements = WebDriverWait(driver, 35).until(
2024-05-05T19:36:27.326Z                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-05T19:36:27.328Z         File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/support/wait.py", line 95, in until
2024-05-05T19:36:27.329Z           raise TimeoutException(message, screen, stacktrace)
2024-05-05T19:36:27.331Z       selenium.common.exceptions.TimeoutException: Message:
2024-05-05T19:36:27.504Z INFO  Exiting actor ({"exit_code": 0})


I don't see anything in the docs for Python, but I do see references to context.waitFor() which looks like JS.

Is this type of waiting only supported in JS?
N
1 comment
Just to follow up, I think this is a problem in selenium. Locally it doesn't seem to work in headless mode. I would up rewriting my function in js using puppeteer and that worked. 🤷
Add a reply
Sign up and join the conversation on Discord