Apify and Crawlee Official Forum
b
F
A
J
A
preNavigationHooks array in CheerioCrawlerOptions [1]const crawlerOptions = {
... preNavigationHooks: [],
});
const jsFunction = "async ({ page, request }) => { log.info(`preNavigationHook ${request.url}`); }";
crawlerOptions.preNavigationHooks.push( ??? WHAT ???)
const myCrawler = new CheerioCrawler(crawlerOptions);crawlerOptions.preNavigationHooks.push(jsFunction);, when I run crawler, I got error:WARN CheerioCrawler: Reclaiming failed request back to the list or queue. TypeError: hook is not a function
at CheerioCrawler._executeHooks (D:\Developpement\NodeJS\Nowis_Scraper\node_modules@crawlee\basic\internals\basic-crawler.js:834:23)
at CheerioCrawler._handleNavigation (D:\Developpement\NodeJS\Nowis_Scraper\node_modules@crawlee\http\internals\http-crawler.js:326:20)
at CheerioCrawler._runRequestHandler (D:\Developpement\NodeJS\Nowis_Scraper\node_modules@crawlee\http\internals\http-crawler.js:286:24)
eval can be harmful._runHookWithEnhancedContext [1] to make my own following JS function. _runHookWithEnhancedContext(hooks) {
return hooks.map((hook) => function enhancedContext(ctx) {
const { customData } = this.input;
hook({ ...ctx, customData });
}.bind(this));
}const preNavigationHooks = "[ async (context) => { context.log.info(context.customData);\n context.log.info(`preNavigationHook ${context.request.url}`); }, ]"
// Transforms a pre/post navigation hooks string into array of Functions.
evaledPreNavigationHooks = evalArrayJSFunction(preNavigationHooks, 'preNavigationHooks');
preNavigationHooks.push(..._runHookWithEnhancedContext(evaledPreNavigationHooks));