How to implement persistent login with crawlee-js/playwright?

Question

I need to scrape content on multiple pages in one social network (x.com) that requires auth. Where to implement the login mechanism in order to it happened before following urls and persisted to use it until it is valid?

! Spooky · Answer

await page.context().storageState({ path: authFilePath })Look up storageState() on the playwright docs

Crafty · Answer

that gets the cookie and localstorage state but how do you load it into a new session. im facing the same dilema

Oleg V. · Answer

You can store you cookies in named KV store and then modify Your session with function:
https://crawlee.dev/api/next/core/interface/SessionPoolOptions#createSessionFunction

or You can do the same (update your requests with those cookies) in preNavigationHooks (https://crawlee.dev/api/next/browser-crawler/interface/BrowserCrawlerOptions#preNavigationHooks)

Crafty · Answer

what about local storage? im supprised there seems to be no easy way to seed a session with local data

Crafty · Answer

and also, setCookie wants a raw cookie string and a url rather than the format the getState() gives you

Apify and Crawlee Official Forum

How to implement persistent login with crawlee-js/playwright?