Hello, i'm creating a supermarket data scraper. The supermarket i'm scraping has a sitemap where are the urls for every product are listed. Currently i'm loading those in like this:
const { urls } = await Sitemap.load('https://.../entities/products/detail.xml');And the passing them to my crawler:
However this writes all of them again to the default request queue. Writing +23.000 items to the requests queue every run costs me minimally $0.50 every time. Is there any way I can write the the request queue (or another place) once, and then read from there the next runs?
