Apify Discord Mirror

Updated 5 months ago

Purging storage using npx crawlee run does not work

At a glance

The community member is trying to develop a crawlee scraper locally and needs to easily purge all data from the default and named datasets as well as request queues to test their changes. However, it does not purge storage. They intend to use the crawlee code in an Apify Actor and are seeking suggestions on what might be the issue.

In the comments, another community member suggests using the drop method, and notes that each Apify Actor run has its own default storage. The original poster then asks if it is also possible to do this from the CLI when starting the crawlee scraper.

Useful resources
I am trying to develop a crawlee scraper locally and it that regard I need to easily purge all data from the default and named datasets as well as request queues to test my changes.
However, it does not purge storage.
I intend to use the crawlee code in an Apify Actor.
Do you have any suggestions of what might be the issue?
H
C
2 comments
You can use the drop method:
https://crawlee.dev/api/core/class/Dataset#drop

Each Apify Actor run has its own default storage
thanks, is it also possible to do it from the cli when starting the crawlee scraper? πŸ™‚
Add a reply
Sign up and join the conversation on Discord