Apify and Crawlee Official Forum

Updated 4 months ago

Dealing with images?

I need to display profile pictures (from scraped social media profiles), to my users on a website. After scraping profiles, I get a returned URL relating to a profile picture.

I'm assuming I can't just set my website up to fetch the image from the URL to display because I need to do this potentially millions of times which would result in IP banning and black listing, no?

So how do I deal with this? Is there a way I can download images using Apify (ideally with settings to change resolution / size, etc) that I can then store in my own database?

Thanks :)
O
1 comment
You can download image.
Here is an article:
https://docs.apify.com/academy/puppeteer-playwright/common-use-cases/downloading-files

Also, "Dataset Image Downloader & Uploader" Public Actor can be useful in your case:
https://apify.com/lukaskrivka/images-download-upload
Add a reply
Sign up and join the conversation on Discord