catalogue / tools / Fetch & extract / Structured Data Extract API

Structured Data Extract API

Fetches a web page and returns its embedded machine-readable data -- JSON-LD, OpenGraph, microdata and meta tags -- plus a normalised merge across all four.

Endpoint
POST /v1/structured-data-extract
Price
$0.002 per call
Family
Fetch & extract

01When to use it

  • It needs the network and a real DOM, and the extraction is the code an agent keeps re-implementing wrong: skipping a broken JSON-LD script instead of throwing, telling og: from name= apart, and walking microdata with its parent/child nesting rule. One call replaces four brittle scrapers.

When not to use it

  • The input is over 5 MB (5,242,880 bytes): structured-data-extract answers too_large (413).
  • The destination needs more than 8 s to answer: the call ends with upstream_timeout (424).
  • The destination is on a private or local network: the network guard refuses it with blocked_target (403).

02Parameters

FieldTypeRequiredDescription
urlstringyesAbsolute http(s) URL of the HTML or XHTML page to read, up to 2048 characters; up to 5 redirects are followed and relative URLs resolve against the final URL or its <base href>.
typesarraynoWhich of jsonld, opengraph, microdata and meta to extract, at least one; omitted means all four, and a type not requested comes back null and is left out of the normalized merge.

03Limits

LimitValue
Size cap5 MB (5,242,880 bytes)
Timeout8 s

04Example

The published example of structured-data-extract, verbatim: the request body and the response it returns.

Request
POST /v1/structured-data-extract
content-type: application/json
payment-signature: <base64 x402 payload>

{
  "url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"
}
Response
200 OK
payment-response: <base64 settlement receipt>

{
  "url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html",
  "source_url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html",
  "final_url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html",
  "http_status": 200,
  "jsonld": [
    {
      "@context": "https://schema.org",
      "@type": "Article",
      "headline": "The first frost of the season",
      "description": "A light ground frost on the lower beds, measured at dawn.",
      "image": "../icon-32.png",
      "datePublished": "2026-08-24T06:30:00Z",
      "author": {
        "@type": "Organization",
        "name": "Field Notes Editors"
      }
    }
  ],
  "jsonld_total": 1,
  "jsonld_truncated": false,
  "jsonld_parse_errors": 0,
  "opengraph": {
    "og:title": "The first frost of the season",
    "og:type": "article",
    "og:url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html",
    "og:image": "https://grist.tools/samples/web/fieldnotes/icon-32.png"
  },
  "microdata": [
    {
      "type": "https://schema.org/Place",
      "properties": {
        "name": "Lower beds",
        "description": "The lowest corner of the garden, where cold air settles first."
      }
    }
  ],
  "microdata_total": 1,
  "microdata_truncated": false,
  "meta": {
    "description": "A light ground frost on the lower beds, measured at dawn.",
    "author": "Field Notes Editors",
    "article:published_time": "2026-08-24T06:30:00Z"
  },
  "normalized": {
    "title": "The first frost of the season",
    "description": "A light ground frost on the lower beds, measured at dawn.",
    "image": "https://grist.tools/samples/web/fieldnotes/icon-32.png",
    "type": "Article",
    "author": "Field Notes Editors",
    "published": "2026-08-24T06:30:00Z"
  },
  "fetched_at": "2026-09-01T12:00:00.000Z"
}

Sample file: web/fieldnotes/notes/first-frost.html. Field Notes article page carrying an Article JSON-LD script, OpenGraph and author meta tags, and one schema.org Place microdata item.

Response fieldTypeIn the example
urlstring"https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"
source_urlstring"https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"
final_urlstring"https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"
http_statusnumber200
jsonldarray1 item, each with @context, @type, headline, description, image, datePublished and author
jsonld_totalnumber1
jsonld_truncatedbooleanfalse
jsonld_parse_errorsnumber0
opengraphobjectan object with og:title, og:type, og:url and og:image
microdataarray1 item, each with type and properties
microdata_totalnumber1
microdata_truncatedbooleanfalse
metaobjectan object with description, author and article:published_time
normalizedobjectan object with title, description, image, type, author and published
fetched_atstring"2026-09-01T12:00:00.000Z"

05Call it from code

JavaScript
// structured-data-extract: $0.002 per call, paid in USD Coin on eip155:8453 via x402
// Install: npm install @x402/fetch@2 @x402/evm@2 viem@2
// Save as client.mjs (ES module, Node 18+), export EVM_PRIVATE_KEY with the paying wallet's key in your shell, then run: node client.mjs
// The wrapper reads the 402 (payment-required), signs and retries with payment-signature.
import { wrapFetchWithPayment, x402Client, decodePaymentResponseHeader } from '@x402/fetch'
import { ExactEvmScheme } from '@x402/evm/exact/client'
import { privateKeyToAccount } from 'viem/accounts'

const account = privateKeyToAccount(process.env.EVM_PRIVATE_KEY)
const client = new x402Client().register('eip155:8453', new ExactEvmScheme(account))
const fetchWithPayment = wrapFetchWithPayment(fetch, client)

const body = {
  "url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"
}

const res = await fetchWithPayment('https://grist.tools/v1/structured-data-extract', {
  method: 'POST',
  headers: { 'content-type': 'application/json' },
  body: JSON.stringify(body),
})
console.log(res.status, await res.json())
const settle = res.headers.get('payment-response')
if (settle) console.log(decodePaymentResponseHeader(settle).transaction)
Python
# structured-data-extract: $0.002 per call, paid in USD Coin on eip155:8453 via x402
# Install: pip install "x402[requests,evm]>=2.24,<3"
# The session reads the 402 (payment-required), signs and retries with payment-signature.
import os
from eth_account import Account
from x402 import x402ClientSync
from x402.http import decode_payment_response_header
from x402.http.clients.requests import x402_requests
from x402.mechanisms.evm.exact import register_exact_evm_client
from x402.mechanisms.evm.signers import EthAccountSigner

account = Account.from_key(os.environ["EVM_PRIVATE_KEY"])
client = x402ClientSync()
register_exact_evm_client(client, EthAccountSigner(account), networks="eip155:8453")
session = x402_requests(client)

payload = {
    "url": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"
}

res = session.post("https://grist.tools/v1/structured-data-extract", json=payload)
print(res.status_code, res.json())
settle = res.headers.get("payment-response")
if settle:
    print(decode_payment_response_header(settle).transaction)
curl
# structured-data-extract: $0.002 per call, paid in USD Coin on eip155:8453 via x402
# 1. Unpaid call: HTTP 402, the requirements in the payment-required header (base64 JSON) and in the body.
curl -i -X POST 'https://grist.tools/v1/structured-data-extract' -H 'content-type: application/json' -d '{"url":"https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"}'

# 2. Same call with the signed payment (an EIP-3009 authorization, EIP-712 signed: it cannot be
#    typed by hand). x-payment is accepted as the v1 alternative. HTTP 200 carries payment-response.
curl -i -X POST 'https://grist.tools/v1/structured-data-extract' -H 'content-type: application/json' -H 'payment-signature: <base64 x402 payload>' -d '{"url":"https://grist.tools/samples/web/fieldnotes/notes/first-frost.html"}'

06Errors

CodeHTTPFor this service
invalid_input400The body does not match the schema; its fields are url and types.
blocked_target403
unreachable_target424
upstream_status424
upstream_timeout424
unsupported_content_type415
too_large413
unprocessable422
internal500

Input that is rejected before the tool runs (malformed JSON, schema mismatch, body over the size limit) is never charged.

When each code is raised, and what it means for payment: /docs/structured-data-extract.

07Questions

How much does a structured-data-extract call cost?
$0.002 per call, paid in USDC on Base via x402, with no account and no API key.
What does structured-data-extract return?
A JSON object with 15 top-level fields: url, source_url, final_url, http_status, jsonld, jsonld_total, jsonld_truncated, jsonld_parse_errors, opengraph, microdata, microdata_total, microdata_truncated, meta, normalized and fetched_at. The example on this page is a real call over web/fieldnotes/notes/first-frost.html.
What happens when a structured-data-extract call fails?
The tool answers with a typed JSON error from the errors table. Payment is settled only after the tool has produced its result; a call that fails inside the tool is never settled. A 503 upstream_unavailable can follow a settled call; the x402 payments guide covers it, 429, 402 and an uncertain 500.

08Related