main.ts
import { PlaywrightCrawler } from 'crawlee';
import { router } from './routes.js';
const startUrls = [
11 links
...
];
const crawler = new PlaywrightCrawler({
requestHandler: router,
maxRequestsPerCrawl: 1,
maxRequestRetries: 0,
maxRequestsPerMinute: 16,
maxConcurrency: 1,
headless: false,
});
await crawler.run(startUrls);I've updated route.ts and after
npm start everything works fine for the last 10 links but for the first link it is running old deleted code for some reason(confirmed by testing multiple times, editing different things). I tried deleting request queue but same thing happened. I'm at a loss for what is happening.
