Apify Discord Mirror

l
levav
Offline, last seen 5 months ago
Joined August 30, 2024
Hey everyone. I'm trying to run multiple actor calls simultaneously, and that works fine, unless I go over the allotted memory (32GB). I would like to use a requestQueue, which I assume would run calls in order whenever there is available memory? But I have a hard time understanding how to add actors to the queue with the API.
e.g. client.requestQueue("...").addRequest() accepts some unclear parameters that don't seem to contain an actor or the data to pass to the actor. This very much might be because I'm a javascript noob that has a bit of a hard time understanding the documentation XD

If I completely misunderstood the idea of a requestQueue, I would like to create my own. For that, I would like to be able to try/catch a normal client.actor(...).call() but specify the error, except I can't seem to find a way to specify the specific error you would get when requesting the use of too much memory (e.g. some error object to import from apify and use).

sorry for the noob questions XD
2 comments
l
P
I want to do this so that I can more easily choose the best way for me to run an actor (compare runs with different memory sizes for example) or even compare different actors that solve the same issues.
I've been looking at the apify js client documentation & at the apify api documentation and didn't find anything, but maybe I just missed it?
3 comments
L
A
Hi everyone, this doc: https://docs.apify.com/academy/puppeteer-playwright/browser-contexts
shows how to use persistent context when working with pure playwright. But how can I combine this with crawlee? is there a configuration for this while calling PlayWrightCrawler(...)? or a way to get similar behaviour?
4 comments
l
P
I'm trying to install crawlee through npx crawlee create my-crawler but it seems like npx can't find npm. (no such file or directory, lstat 'C:\users\ethan\AppData\Roaming\npm')

npm -v works fine, v18.17.1
I'm running on a windows 11 machine, from visual studio code with powershell.
1 comment
l
essentially - I want to make sure that I'm logged in, in any session that I run. Even better - That I log in with one user per session. How can I make sure that a new session won't open without running a log-in?
I have some code that looks something like this:
Plain Text
router.addHandler('log-in', async ({...}) => {...})

I want to reuse this handler in multiple crawlers. I could of course simply create the function log_in in some other file, but then I would use code hints for page/request etc. is there a better way to go about reusing the handler?
11 comments
l
T
A
P
I'm trying to pass a username and password to the async function in the default handler, since I'm using the default handler to log in to the website. I've seen different guides use all kinds of input parameters - request, page, enqueueLinks, log, pushData - but these seem to all be specific prebuilt parameters of the module? I'm not sure. so, how could I pass my own data through?
3 comments
A
l
L