Apify and Crawlee Official Forum

Updated last year

dynamic rendering with button clicking

in a playwright crawler, how could i keep clicking and waiting a second and clicking a button that renders in more data until that button is no longer present and end the loop

and while the clicking is going on, rendered product urls are scraped

1 comment

LLukas Krivka

I think you can do a simple for loop with a sleep. something like:

Plain Text

let noMoreButtonTimes = 0;
for (;;) {
    const button = await page.$('button')
    if (button) {
        await button.click()
    } else {
       noMoreButtonTimes++;
       if (noMoreButtonTimes > 3) break 
    }
    await sleep(2000)
}

Add a reply