The community member needs to display profile pictures from scraped social media profiles on their website. They are concerned that directly fetching the images from the returned URLs could result in IP banning and blacklisting due to the potentially large number of requests. The community member asks if there is a way to download the images using Apify, with the ability to adjust the resolution and size. In the comments, another community member suggests using the "Dataset Image Downloader & Uploader" Public Actor on Apify, and provides a link to an article on downloading files with Puppeteer and Playwright.
I need to display profile pictures (from scraped social media profiles), to my users on a website. After scraping profiles, I get a returned URL relating to a profile picture.
I'm assuming I can't just set my website up to fetch the image from the URL to display because I need to do this potentially millions of times which would result in IP banning and black listing, no?
So how do I deal with this? Is there a way I can download images using Apify (ideally with settings to change resolution / size, etc) that I can then store in my own database?
