Apify and Crawlee Official Forum

T
Tobey
Offline, last seen 2 weeks ago
Joined August 30, 2024
Hello, how to make the images load in store in the actor readme? Currently, they are not showing up despite being present in the repo.
https://apify.com/cyberfly/listingmonitor
8 comments
T
A
a
https://console.apify.com/view/runs/4rGZolVyfYVQesyvO
ArgumentError: Expected property maxRequestsPerCrawl to be of type number but received type string in object PlaywrightCrawlerOptions
Plain Text
maxRequestsPerCrawl: process.env.ACTOR_MAX_PAID_DATASET_ITEMS || input.limit,

Plain Text
typeof crawler.options.maxRequestsPerCrawl
'number'

???
2 comments
T
H
T
Tobey
·

ForceCloud

I find the docs around the forceCloud behavior confusing. Is this feature still available? None of these statements from the SDK docs seem to hold true currently, with the SDK:

Note that you can force usage of the cloud storage also by passing the forceCloud option to Dataset.open function, even if the APIFY_LOCAL_STORAGE_DIR variable is set.

Dataset stores its data either on local disk or in the Apify cloud, depending on whether the APIFY_LOCAL_STORAGE_DIR or APIFY_TOKEN environment variables are set.

Also, what is StorageManagerOptions as referenced in the docs?

https://docs.apify.com/sdk/js/reference/class/Dataset#open
6 comments
T
P
A
v
How to make a crawler use non-random input based session ID while still making it possible to rotate/randomize and persist the session when needed (e.g. bad/blocked proxy)? E.g. first use session ID email and if the session is retired and request retried then use session ID email-randomNumber etc.
3 comments
T
How can a storageState be injected into crawler's page before the page is created? Basically, how to reproduce this but using storageState from the session instead:
Plain Text
        prePageCreateHooks: [
            async (pageId, browserController, pageOptions) => {
                const storageState = await KeyValueStore.getValue('storageState');
                if (pageOptions && storageState)
                    pageOptions.storageState = storageState;
            },
        ],
8 comments
T
A
s