run logs on the platform

I have run cheerio crawler on the platform and it logs line like this:

2022-09-30T08:02:44.883Z WARN  CheerioCrawler: Reclaiming failed request back to the list or queue. Cannot read properties of null (reading 'match') {"id":"PxlxlTCgnI7zPOi","url":"https://www........","retryCount":1}

I have two questions:

why is there WARN instead of ERROR? I would prefer if it is ERROR and in red color, I believe it was always like this, was it changed?
why I can't see file and line where the error ocurred?

What should I change to solve this?

2 comments

LLukas Krivka

Yeah, it was. We will see if more people will want error. The idea is that retries are normal and you should not care much, only the last retry is a hard error.
The same thing, to not spam the log. You can print the whole error in errorHandler

HHonzaS

Ok, from my view as somebody who is trying to debug the program on the platform >

Waiting for the last retry is pretty time and CU consuming with large queue as that hard error will happen after a few hours. I want to see error as early as possible so I can change the code and try again.
That makes the platform less user friendly for debugging as I need to write more code to see the error. So it is encouraged to develop locally and only push to platform after it is bug free, right? So far I have been writing the small scripts directly on the platform as it was faster.

Add a reply

Join on Discord

Apify and Crawlee Official Forum

run logs on the platform