Apify

Apify and Crawlee Official Forum

Home
Members
microworlds
m
microworlds
Offline, last seen 3 months ago
Joined August 30, 2024
mmicroworlds
·

Get user payment status from actor

Hi,

I have a PPR actor and want to access the user's payment status so I can prioritize their queries and set some limitations for non-paying users as well. My current actor implementation is in Node js. I understand that there is an endpoint /users/me but how do I get this info via my actor. Below is a skeletal implemention of the actor:

Plain Text
import { Actor } from 'apify';
import { PuppeteerCrawler } from 'crawlee';
import { router } from './routes.js';

await Actor.init();

const input = await Actor.getInput();
const startUrls = input?.startUrls || [{ url: 'https://apify.com' }];

const proxyConfiguration = await Actor.createProxyConfiguration();

const crawler = new PuppeteerCrawler({
    proxyConfiguration,
    requestHandler: router,
});

await crawler.run(startUrls);

await Actor.exit();


Thanks! 🙏
10 comments
1
g
H
D
m
J
mmicroworlds
·

Keyvaluestore file extensions

Hi,

How do I configure Keyvalue store to have the .mhtml file extension? Using the code below seems to always set it to .bin extension

Plain Text
await KeyValueStore.setValue("some-name", data, {
  contentType: `application/x-mimearchive` \\ multipart/related doesn't work either,
});

// data above is either text or buffer


The goal is to have a file written to KV similar to some-name.mhtml

Thanks!
14 comments
m
A
A
mmicroworlds
·

Override browser permission on PuppeteerCrawler

Hi, a quick question: how do I override a certain permission on a page when using PuppeteerCrawler? Something like so:

Plain Text
...,
preNavigationHooks: [
  async (crawlingContext, gotoOptions) => {
    const { page, browserController: { browser } } = crawlingContext;

    const context = await browser.defaultBrowserContext();
    context.clearPermissionOverrides();
    context.overridePermissions('some-domain', ['some-feature']);
  }
],
...


When I run it, I get a warning message:
WARN PuppeteerCrawler: Reclaiming failed request back to the list or queue. Cannot read private member from an object whose class did not declare it

I understand that I'm accessing private prop but how/where do I override such permissions on PuppeteerCrawler? 🙂

Thanks! 🙏
11 comments
v
m