Apify and Crawlee Official Forum

Updated 3 months ago

I want the first urls that I enqueue to run before the next forefront:true.

I have two places that do forefront: true, the very first URLs that I start the crawler with.

Then each page will also have a set of urls that go on the front of the queue, but, I want the very first urls that start the crawl to have priority, how would I accomplish this?

I was thinking of two request queues but had a lot of problems with that as well.
v
1 comment
You could use both 2 request queues and 2 crawler instances (each for one of those queues), running first the priority one and then the other one. See also my comment on your other post.
Add a reply
Sign up and join the conversation on Discord