Apify Discord Mirror

Updated last month

Apify Proxy not working with https urls

At a glance

The community member is trying to test the Apify proxy with their Python code, but they are encountering issues with HTTPS URLs. They provide a code snippet that works with HTTP URLs but not with HTTPS URLs, resulting in a proxy error and a timeout. Another community member suggests that the issue could be a timeout error that can be solved by creating and using another proxy URL or increasing the timeout. However, the original poster confirms that the issue persists even with increased timeout. The issue was eventually resolved, with a community member stating that "the issue was the user's connection".

Useful resources
I want to test the apify proxy and how it works to integrate it with my py code.
Running a very simple check I found it's not working with https urls. here's a snippet:
Plain Text
import asyncio, httpx
from apify import Actor
import dotenv

async def main():
    async with Actor:
        proxy_configuration = await Actor.create_proxy_configuration(
            password=dotenv.get_key('.env', 'APIFY_PROXY_PASSWORD'),
        )
        proxy_url = await proxy_configuration.new_url()
        proxies = {
            'http': proxy_url,
            'https': proxy_url,
        }
        async with httpx.AsyncClient(proxy=proxy_url) as client:
            for _ in range(3):
                response = await client.get('https://httpbin.org/ip')
                if response.status_code == 200:
                    print(response.json())
                elif response:
                    print(response.text)

if __name__ == '__main__':
    asyncio.run(main())

giveing me a proxy error:
Plain Text
          raise mapped_exc(message) from exc
      httpx.ReadTimeout
[apify] INFO  Exiting Actor ({"exit_code": 91})

If i just only change the protocol to http://httpbin.org/ip it works.
Apify proxy should support https as stated on the site. Thanks in advance.
Marked as solution
I'll write you a DM

edit: the issue was the user's connection
View full solution
R
3 comments
@🟢mido🟢 hey, I'm not able to reproduce the issue. Does this happen repeatedly? It seems to be just a timeout error that could be solved by creating and using another proxy url or increasing the timeout (https://www.python-httpx.org/exceptions/, https://www.python-httpx.org/advanced/timeouts/)
@Rado Ch. Hi, Actually yes it happens with any https url. I tried increasing timeout=60 but still getting same error. And http urls even work with timeout=5
I'll write you a DM

edit: the issue was the user's connection
Add a reply
Sign up and join the conversation on Discord