Apify Discord Mirror

Updated last year

Code refactoring - reusing a common handler in multiple crawlers while keeping code hints

At a glance

The community member has a code snippet with a router.addHandler function that they want to reuse across multiple crawlers. They are unsure if there is a better way to do this without losing code hints and linting features. The comments suggest that splitting the routes into separate files may be a solution, as mentioned in the Crawlee documentation. However, some community members express frustration that this is not a straightforward solution and can lead to "hacky" code. There is a discussion around using TypeScript to define a "handler function" type, but it is unclear if this would fully address the issue. One community member suggests importing the label and handler function from a separate file, but others note that this can still cause problems with linting and dev tool functionality. There is no explicitly marked answer in the comments.

Useful resources
I have some code that looks something like this:
Plain Text
router.addHandler('log-in', async ({...}) => {...})

I want to reuse this handler in multiple crawlers. I could of course simply create the function log_in in some other file, but then I would use code hints for page/request etc. is there a better way to go about reusing the handler?
1
T
l
P
11 comments
Why tf has this not been answered πŸ₯² The documentation even says
Plain Text
We would most likely go even further and split even the routes into separate files.
and yet this is very unclear without losing linting and useful dev features. I'm upset.
Yea... I ended up adding and removing a dummy router while developing to deal with this. Hacky solution, but it works.
When I asked this question, I was super new to the lovely thing called javascript. From what I've seen in the past two months - this isn't as much an issue with Crawlee, as it is an issue with Javascript in general... just a badly designed language.
I haven't tried this myself, but I think moving to typescript might solve this issue. (I can't use it since I'm limited by my team ;p)
I am using typescript and have XP across both. I think your idea is a bit of a red herring πŸ˜„
router.addHandler takes label and handler function as a parameter. You may import both from different file.
welp, was just a silly idea XD hoped you could use typescript to define a "handler function" type that you could reference in the new file. but again, I know nothing about typescript beyond it being a typed version of javascript ;p I'm just importing ideas from python/c++
is handler function a type? that you can import? thats the issue. that a generic function won't allow for dev tools to work properly.
you lose linting doing this as the types are unknown. causes hacky looking code in editor, I didn't say its not possible... just not great dev friendliness
just advanced to level 1! Thanks for your contributions! πŸŽ‰
you'd be manually redefining the function signature? so any change to the handler you'd have to account for in your cloned type. dodging the problem imo πŸ₯²
yupppp, I agree, its far from a perfect solution. but idk at this point.
Add a reply
Sign up and join the conversation on Discord