Apify and Crawlee Official Forum

Updated 5 months ago

Scraping Threads Timeline. But Not able to get entire data.

At a glance
I am making a scrapper to scrap threads.net using playwright and python and willing to host on Apify.

But It I am not able to scrap entire timeline. It is only giving some data and when scrolling down, I am not able to scrap the new data.

Any help or suggestion is highly appreciated.
A
L
S
3 comments
I recommend storing screenshots often, you will see exactly where you got stuck and why. More here https://docs.apify.com/academy/node-js/analyzing-pages-and-fixing-errors
Thank you guys. It helped.
Add a reply
Sign up and join the conversation on Discord