Apify

Apify and Crawlee Official Forum

b
F
A
J
A
Members
Nyanmaru
N
Nyanmaru
Offline, last seen 4 weeks ago
Joined August 28, 2024
Has anybody tried downloading the HTML file of the URL using Crawlee? Was wondering if Crawlee has a capacity of downloading the HTML file of the URL since I've just been using Crawlee and really loving the experience.
6 comments
M
E
N
C
s
Hi and good day. I'm creating a POST API that access the following JSON body:
{
"url": "https://crawlee.dev/python/",
"targets": ["html", "pdf"]
}

Inside the list of targets, is the extension which my code downloads if it discovers.

I'm already at my wit's end since I don't get the error I'm getting which is:
[crawlee.memory_storage_client._request_queue_client] WARN Error adding request to the queue: Request ID does not match its unique_key.

Does anyone encountered this problem?

The following is my whole code:
9 comments
A
N
O
H
M
Crawlee Code works when I put the URL directly inside the crawlee.run([url]), but when I put the code inside an endpoint and execute the URL on postman error NotImplementedError shows.

How to use crawlee w/ FastAPI?
Has anybody tried downloading the HTML file of the URL using Crawlee? Was wondering if Crawlee has a capacity of downloading the HTML file of the URL since I've just been using Crawlee and really loving the experience.
4 comments
M
N
E