Apify

Apify and Crawlee Official Forum

Updated 2 weeks ago

Reclaiming failed request back to the list or queue. Navigation timed out after 60 seconds.

SSir LuBin
·
Hi guys, I just create a simple crawler from ts + crawlee template. The code is working on local but keep timeout on Apify.
The code has been modify to save to mongo-db.
https://github.com/blackmouse572/vnexpress-crawler
Attachment
image.png
S
O
2 comments
SSir LuBin
A bit of update, I have create new template, copy the route contents, and remove node-html-makrdown it run okay, but remove on my source code didn't work
Attachments
image.png
image.png
OOleg V.
Try saving a screenshot (https://crawlee.dev/docs/next/examples/capture-screenshot) in the preNavigationHook (https://crawlee.dev/api/playwright-crawler/interface/PlaywrightCrawlerOptions#preNavigationHooks) to see what’s happening on the page and why it’s struggling to navigate. Also, check if your scraper is running with low memory on the platform. If that’s the case, consider increasing the navigationTimeout (https://crawlee.dev/api/next/playwright-crawler/interface/PlaywrightCrawlerOptions#navigationTimeoutSecs) in your configuration.
Add a reply
Sign up and join the conversation on Discord