Apify and Crawlee Official Forum

Updated 4 months ago

Problem with apify platform integration of custom crawler built offline with plaint JS

Hello guys i have created a scraper for a certain website, is written in javascript and does not use any scraping framework like puppeteer cheerio crawlee etc.. It only uses plain js and requests and it works just fine for me. In the same script i have inserted code for a mongodb connection which will insert the scraped data directly into my cluster on mongodb atlas. My problem is that i want to run this scraper on the apify platform but i can't seem to figure out the right way to do it. First i tried by connecting the github repo with the apify platform and it worked the scraper was built but when i tried to run it there was no response only an issue. After that i tried with the javascript node template from apify but i ran that locally and after i inserted my code into the template i pushed it to apify platform from the console and it worked, the scraper was again build, this time i have no error when running it but nothing happens. I must say that the urls i want to scrape are hardcoded in my script already and when i try to delete the input from that json on apify it does not let me and the default apify input is this : {
"startUrls": [
{
"url": "https://apify.com/"
}
]
} Can anyone help me ?
M
1 comment
To change the input of an actor on the platform, you need to edit the input schema. Generally, it is .actor/input_schema.json: https://docs.apify.com/platform/actors/development/actor-definition/input-schema/specification/v1
Add a reply
Sign up and join the conversation on Discord