Apify and Crawlee Official Forum

Updated 2 days ago

Google Maps crawler - Increasing place limit after initial run

Hi everyone,
I recently ran a Google Maps scraper (https://apify.com/compass/crawler-google-places) to collect place data, and I've discovered that there are many more places available than what was initially collected in my first run.
Current Situation:
  • Successfully completed an initial scrape
  • Have collected data for X places
  • Discovered there are significantly more places available
  • Already have a dataset from the first run
Questions:
Is it possible to increase the place limit on my existing run configuration?
If I need to create a new run, what's the best way to:
  • Import/merge my existing scraped data
  • Avoid duplicating places already collected
  • Continue from where the previous run stopped
Any guidance on the most efficient approach would be greatly appreciated.
Thanks in advance!
P
C
S
4 comments
Hi @Cuivrax What you describe really depend on the Actor implementation.

Generally there is no way to extends already scraped data or to start a new Run that would be skipping data based on previous Runs.
@Pepa J That's unfortunate because you have to run your actor again and pay again for data that's already been scraped. It should have a history feature to allow extending the scraping process. For now, I'll stick with my Python script.
I have a googlemap crawler that I made with python script. And this script can solve the problem you are having.
Add a reply
Sign up and join the conversation on Discord