I'm reaching out to the community for some assistance with an issue I'm encountering while using Playwright and Apify. I've built a scraper that incorporates proxy rotation with sessions, but I'm facing a problem where the proxy isn't changing when the session changes. Additionally, I'd like to ensure the same IP address is used for both the proxy and the captcha solver.
Problem Description:
Playwright script utilizes Apify's proxy configuration with session management. Proxy rotation is set to occur every 10 downloads. Session IDs are dynamically generated as my_session{index}. Despite session switching messages, the proxy server remains the same.
Switching to proxy session: my_session1 with {'server': 'http://10.0.94.249:8011', 'username': 'groups-auto,session-my_session1', '}
Switching to proxy session: my_session2 with {'server': 'http://10.0.94.249:8011', 'username': 'groups-auto,session-my_session2', '}
Switching to proxy session: my_session3 with {'server': 'http://10.0.94.249:8011', 'username': 'groups-auto,session-my_session3', }