Apify and Crawlee Official Forum

Updated 3 months ago

Cannot get a ad-hoc webhook attached to a run of a js client actor

hi apifyers,
i can't get ad-hoc webhooks connected to the "website-content-crawler" actor via js apify client.

i am building a array of objects like this and base64 encode it as a string and give it via webhooks param to the actor input config.

i also see the webooks value as a string in the input in the apify backend when the client is build and run but no integration was set up.

any clue what i do wrong. although there's documentation i am missing a more detailed example for the apify client js .

Plain Text
`
// List all possible actor run event types:
  const actorRunEventTypes = [
    "ACTOR.RUN.CREATED",
    "ACTOR.RUN.SUCCEEDED",
    "ACTOR.RUN.FAILED",
    "ACTOR.RUN.ABORTED",
    "ACTOR.RUN.TIMED_OUT",
    "ACTOR.RUN.RESURRECTED",
  ];

  // Create the webhooks for the actor run events
  const webhookUrls = actorRunEventTypes.map((eventType) => ({
    eventTypes: [eventType],
    requestUrl: `${webhookUrl.origin}{webhookUrl.pathname}`,
    payloadTemplate: {
      userId: "{{userId}}",
      createdAt: "{{createdAt}}",
      eventType: "{{eventType}}",
      eventData: "{{eventData}}",
      resource: "{{resource}}",
      resourceId: "{{resource.id}}",
    },
  }));

  console.log({ webhookUrls });

  // Base64 encode webhookUrls to send along with the request
  const encodedWebhookUrls = Buffer.from(JSON.stringify(webhookUrls)).toString(
    "base64"
  );

  // Prepare Actor input
  const input = {
    startUrls: [
      {
        url: targetCrawlUrl,
      },
    ],
    useSitemaps: false,
    crawlerType: "playwright:firefox",
    includeUrlGlobs: [],
    excludeUrlGlobs: [],
    ignoreCanonicalUrl: false,
    maxCrawlDepth: 1,
    maxCrawlPages: 2,
    initialConcurrency: 0,
    maxConcurrency: 10,
    initialCookies: [],
    proxyConfiguration: {
      useApifyProxy: true,
    },
    maxSessionRotations: 8,
    maxRequestRetries: 5,
    requestTimeoutSecs: 60,
    dynamicContentWaitSecs: 10,
    maxScrollHeightPixels: 5000,
    removeElementsCssSelector:
      'nav, footer, script, style, noscript, svg,\n[role="alert"],\n[role="banner"],\n[role="dialog"],\n[role="alertdialog"],\n[role="region"][aria-label*="skip" i],\n[aria-modal="true"]',
    removeCookieWarnings: true,
    clickElementsCssSelector: '[aria-expanded="false"]',
    htmlTransformer: "readableText",
    readableTextCharThreshold: 100,
    aggressivePrune: false,
    debugMode: false,
    debugLog: false,
    saveHtml: true,
    saveMarkdown: true,
    saveFiles: true,
    saveScreenshots: true,
    maxResults: 50,
    clientSideMinChangePercentage: 5,
    renderingTypeDetectionPercentage: 10,
    webhooks: encodedWebhookUrls,
  };

  // Run the Actor and wait for it to finish
  const apifyActorRunObject = await client
    .actor("apify/website-content-crawler")
    .start(input);
Add a reply
Sign up and join the conversation on Discord