Apify and Crawlee Official Forum

Updated 5 months ago

Hello Friends,

At a glance
I am a new user and loving what apify can do. Have run some actors already and results are ok. Had a question:

a) If I want to write a custom scrapper or host an existing code here then does apify limit scrapping to only public websites or can it scrap those with username and password (which we will provide of course in our scrapper)?
b) How do I better control the proxy settings as I want to get country specific IP and see results on the website for those.
A
p
2 comments
a) No limits, just be aware that its your responsibility how to deal with website ToS and anti-bot protection in each given case. In tech terms there is even tutorial https://docs.apify.com/academy/puppeteer-playwright/common-use-cases/logging-into-a-website
b) https://docs.apify.com/academy/expert-scraping-with-apify/solutions/rotating-proxies#limiting-proxy-location
Add a reply
Sign up and join the conversation on Discord