Apify

Apify and Crawlee Official Forum

b
F
A
J
A
Members
eneiromatos
e
eneiromatos
Offline, last seen last month
Joined August 30, 2024
eeneiromatos
·

Running nested crawlers

How can I run a crawler inside a running crawler? I have a cheerio crawler running and want to run a new crawler for example a JSDOMCrawler per each page the Cheerio Crawler visits. I know that I can run them in parallel but what I want is to run them nested.
2 comments
s
eeneiromatos
·

How to create a new cheerio instance $?

I need to instatiate a new cheerio object, i'm doing a search in a set o elements and need to select just one element for further processing, my actual code is:
Plain Text
function getOrigin($: typeof cheerioModule) {
    let origin = ""
    const specElements = $('#product_specs table tr').toArray()
    for (const spec of specElements) {
        const _$ = cheerioModule.load(spec)
        const specTitle = _$(".attrib").text().trim()
        if(specTitle.includes("제조국")){
            origin = _$(".attrib-val").text().trim()
            break
        }
    }
    return origin
}

but i'm not sure if that is the right way to instantiate a new cheeio object in crawlee.
2 comments
t
e
eeneiromatos
·

How to create a new cheerio instance $?

I need to instatiate a new cheerio object, i'm doing a search in a set o elements and need to select just one element for further processing, my actual code is:
Plain Text
function getOrigin($: typeof cheerioModule) {
    let origin = ""
    const specElements = $('#product_specs table tr').toArray()
    for (const spec of specElements) {
        const _$ = cheerioModule.load(spec)
        const specTitle = _$(".attrib").text().trim()
        if(specTitle.includes("제조국")){
            origin = _$(".attrib-val").text().trim()
            break
        }
    }
    return origin
}

but i'm not sure if that is the right way to instantiate a new cheeio object in crawlee.
2 comments
t
e