Writing extractors

An agent can register TypeScript through define_extractor that turns a PageSnapshot into structured data. This page covers the contract, the code skeleton, common patterns and the boundaries.

Contract

An extractor is an ES module whose default export has an extract(snapshot) returning any JsonValue:

import type { PageSnapshot, JsonValue } from "@exray/exray-api";

export default {
  async extract(snapshot: PageSnapshot): Promise<JsonValue> {
    // your logic
  },
};

Restrictions (enforced statically and by the sandbox):

Only import type from @exray/exray-api — no runtime modules at all
No eval / new Function
No top-level side effects (only imports, exports, type declarations and the default export)
Bundle size ≤ 1 MB
The child isolate has no network access (globalThis.fetch throws outbound_blocked)
No direct access to the Browser binding — an extractor receives a PageSnapshot and never touches the browser

Violations are rejected at registration (extractor_invalid); runtime violations throw extractor_runtime_error.

The full signature is extract(snapshot, ctx?). When the project has the kv binding enabled, ctx.kv gives the extractor key-value storage shared with fetchers in the same project — useful for dedupe markers and caching:

export default {
  async extract(snapshot, ctx) {
    const seen = await ctx.kv.get(snapshot.final_url);
    if (seen) return { skipped: true };
    await ctx.kv.put(snapshot.final_url, { at: Date.now() });
    return { title: snapshot.metadata.title };
  },
};

Two things to know:

Extractors get kv only — not db, not storage. Those are the fetcher's domain.
This makes an extractor stateful. Any assumption that "the same snapshot always yields the same output" stops holding, which matters for replay and caching.

ctx is optional, so existing extract(snapshot) code keeps compiling and running unchanged. When the binding isn't enabled, calling ctx.kv throws a structured binding_not_enabled.

PageSnapshot fields

interface PageSnapshot {
  url: string;
  final_url: string;        // after redirects
  status: number;
  fetched_at: string;       // ISO datetime
  dom: SerializedDOM;       // serialized DOM tree
  html: string;
  markdown: string;
  text: string;
  screenshot?: string;      // base64 PNG, only when requested
  network: NetworkEntry[];  // [{ url, status, type, size }]
  console: ConsoleEntry[];  // [{ level, text }]
  metadata: {
    title: string;
    description?: string;
    openGraph?: Record<string, string>;
    canonical?: string;
  };
}

Exact types live in @exray/exray-api (an internal monorepo package, not yet on public npm).

Recommended patterns

Because runtime imports are rejected, work from snapshot.html / text / markdown with string handling, or do the DOM work in the fetcher via page.evaluate() and let the extractor only reshape the result.

1. Simple fields — read markdown or text

import type { PageSnapshot } from "@exray/exray-api";

export default {
  async extract(snapshot: PageSnapshot) {
    const titleMatch = snapshot.text.match(/^# (.+)$/m);
    return { title: titleMatch?.[1] ?? null };
  },
};

2. Network log — find the API JSON the page fetched

import type { PageSnapshot } from "@exray/exray-api";

export default {
  async extract(snapshot: PageSnapshot) {
    const api = snapshot.network.find((e) => e.url.includes("/api/items"));
    return { api_url: api?.url ?? null, api_status: api?.status ?? null };
  },
};

3. Metadata and canonical

import type { PageSnapshot } from "@exray/exray-api";

export default {
  async extract(snapshot: PageSnapshot) {
    return {
      title: snapshot.metadata.title,
      canonical: snapshot.metadata.canonical ?? snapshot.final_url,
      og_image: snapshot.metadata.openGraph?.image ?? null,
    };
  },
};

4. Isomorphic chaining — return next URLs to drive the next round

Pairs with crawl's follow: "result:next":

import type { PageSnapshot } from "@exray/exray-api";

export default {
  async extract(snapshot: PageSnapshot) {
    const next = [...snapshot.html.matchAll(/href="(\/posts\/[^"]+)"/g)]
      .map((m) => new URL(m[1], snapshot.final_url).toString());
    return {
      title: snapshot.metadata.title,
      next, // crawl(follow: "result:next") queues these for the next round
    };
  },
};

Registering

{
  "name": "define_extractor",
  "arguments": {
    "name": "product-card",
    "source": "<TS source as a string>",
    "api_version": "1",
    "input_schema": { "type": "object" },
    "output_schema": { "type": "object", "properties": { "title": { "type": "string" } } }
  }
}

Returns { id, name, code_hash, broadcast }. Registration leaves the asset in draft — publish it before it can run:

{ "name": "publish_definition", "arguments": { "kind": "extractor", "name": "product-card" } }

After publishing it's callable as a named tool:

{ "name": "extractor_product-card", "arguments": { "snapshot": {} } }

From the command line the same two steps are:

exray define extractor product-card --file ./extractor.ts
exray publish extractor product-card

Errors and diagnostics

Code	Trigger
`extractor_invalid`	Static validation failed (eval / external import / top-level side effect / oversize / wrong entry point)
`extractor_invalid_output`	Return value isn't a valid JsonValue
`extractor_runtime_error`	Threw at runtime (the stack includes child-source line numbers)
`definition_incompatible_api`	`api_version` doesn't match the runtime `@exray/exray-api` major
`bundle_missing`	The registered bundle.js isn't in R2

Full stacks for failed calls are in content[0].text of the MCP tool response, or via GET /api/jobs/<id>/output for the complete outcome.

Performance and quotas

Wall-clock per extractor call ≤ 10 s
Output per call ≤ 10 MB
Monthly loader_calls quota (set on the token; exceeding it gives budget_exceeded)
Registered versions are immutable: same name with different source produces a new code_hash, and the old version stays in R2

Connect an MCP client
Built-in tools
CLI reference
Type contract: @exray/exray-api (internal monorepo package, not yet on public npm)