catalogue / tools / Fetch & extract / Link Extract API

Link Extract API

Fetches a web page and returns every link on it, resolved to absolute, each tagged with its rel, a nofollow flag and whether it stays on the same site.

Endpoint
POST /v1/link-extract
Price
$0.002 per call
Family
Fetch & extract

01When to use it

  • It needs the network and a real DOM: resolving relative and protocol-relative hrefs against a possible <base>, classifying internal versus external, and reading rel/nofollow correctly is fiddly to get right, and feeding raw HTML to a model to do it costs far more in tokens than this call costs.

When not to use it

  • The input is over 5 MB (5,242,880 bytes): link-extract answers too_large (413).
  • The destination needs more than 8 s to answer: the call ends with upstream_timeout (424).
  • The destination is on a private or local network: the network guard refuses it with blocked_target (403).

02Parameters

FieldTypeRequiredDescription
urlstringyesAbsolute http(s) URL of the HTML or XHTML page to read, up to 2048 characters; up to 5 redirects are followed and links resolve against the final URL or its <base href>.
internal_onlybooleannoWhen true, return only links whose host equals the host of the fetched page (the final redirect hop); the totals then count only those links. Default: false.

03Limits

LimitValue
Size cap5 MB (5,242,880 bytes)
Timeout8 s

04Example

The published example of link-extract, verbatim: the request body and the response it returns.

Request
POST /v1/link-extract
content-type: application/json
payment-signature: <base64 x402 payload>

{
  "url": "https://grist.tools/samples/web/fieldnotes/index.html",
  "internal_only": false
}
Response
200 OK
payment-response: <base64 settlement receipt>

{
  "url": "https://grist.tools/samples/web/fieldnotes/index.html",
  "source_url": "https://grist.tools/samples/web/fieldnotes/index.html",
  "final_url": "https://grist.tools/samples/web/fieldnotes/index.html",
  "http_status": 200,
  "base_url": "https://grist.tools/samples/web/fieldnotes/index.html",
  "links": [
    {
      "href": "https://grist.tools/samples/web/fieldnotes/index.html",
      "text": "Home",
      "rel": null,
      "nofollow": false,
      "internal": true
    },
    {
      "href": "https://grist.tools/samples/web/fieldnotes/about.html",
      "text": "About",
      "rel": null,
      "nofollow": false,
      "internal": true
    },
    {
      "href": "https://grist.tools/samples/web/fieldnotes/feed.xml",
      "text": "Subscribe",
      "rel": "alternate",
      "nofollow": false,
      "internal": true
    },
    {
      "href": "https://grist.tools/samples/web/fieldnotes/notes/first-frost.html",
      "text": "The first frost of the season",
      "rel": null,
      "nofollow": false,
      "internal": true
    },
    {
      "href": "https://grist.tools/samples/web/fieldnotes/notes/rain-gauge.html",
      "text": "Reading a rain gauge by hand",
      "rel": null,
      "nofollow": false,
      "internal": true
    },
    {
      "href": "https://example.org/glossary",
      "text": "an outside glossary",
      "rel": "nofollow noopener",
      "nofollow": true,
      "internal": false
    },
    {
      "href": "https://grist.tools/samples/web/fieldnotes/sitemap-index.xml",
      "text": "Sitemap",
      "rel": null,
      "nofollow": false,
      "internal": true
    }
  ],
  "links_total": 7,
  "links_returned": 7,
  "links_truncated": false,
  "fetched_at": "2026-09-01T12:00:00.000Z"
}

Sample file: web/fieldnotes/index.html. Home page of the Field Notes sample site: relative nav and article links, one nofollow outside link, a mailto and a fragment link, plus RSS, Atom and icon declarations in its head.

Response fieldTypeIn the example
urlstring"https://grist.tools/samples/web/fieldnotes/index.html"
source_urlstring"https://grist.tools/samples/web/fieldnotes/index.html"
final_urlstring"https://grist.tools/samples/web/fieldnotes/index.html"
http_statusnumber200
base_urlstring"https://grist.tools/samples/web/fieldnotes/index.html"
linksarray7 items, each with href, text, rel, nofollow and internal
links_totalnumber7
links_returnednumber7
links_truncatedbooleanfalse
fetched_atstring"2026-09-01T12:00:00.000Z"

05Call it from code

JavaScript
// link-extract: $0.002 per call, paid in USD Coin on eip155:8453 via x402
// Install: npm install @x402/fetch@2 @x402/evm@2 viem@2
// Save as client.mjs (ES module, Node 18+), export EVM_PRIVATE_KEY with the paying wallet's key in your shell, then run: node client.mjs
// The wrapper reads the 402 (payment-required), signs and retries with payment-signature.
import { wrapFetchWithPayment, x402Client, decodePaymentResponseHeader } from '@x402/fetch'
import { ExactEvmScheme } from '@x402/evm/exact/client'
import { privateKeyToAccount } from 'viem/accounts'

const account = privateKeyToAccount(process.env.EVM_PRIVATE_KEY)
const client = new x402Client().register('eip155:8453', new ExactEvmScheme(account))
const fetchWithPayment = wrapFetchWithPayment(fetch, client)

const body = {
  "url": "https://grist.tools/samples/web/fieldnotes/index.html",
  "internal_only": false
}

const res = await fetchWithPayment('https://grist.tools/v1/link-extract', {
  method: 'POST',
  headers: { 'content-type': 'application/json' },
  body: JSON.stringify(body),
})
console.log(res.status, await res.json())
const settle = res.headers.get('payment-response')
if (settle) console.log(decodePaymentResponseHeader(settle).transaction)
Python
# link-extract: $0.002 per call, paid in USD Coin on eip155:8453 via x402
# Install: pip install "x402[requests,evm]>=2.24,<3"
# The session reads the 402 (payment-required), signs and retries with payment-signature.
import os
from eth_account import Account
from x402 import x402ClientSync
from x402.http import decode_payment_response_header
from x402.http.clients.requests import x402_requests
from x402.mechanisms.evm.exact import register_exact_evm_client
from x402.mechanisms.evm.signers import EthAccountSigner

account = Account.from_key(os.environ["EVM_PRIVATE_KEY"])
client = x402ClientSync()
register_exact_evm_client(client, EthAccountSigner(account), networks="eip155:8453")
session = x402_requests(client)

payload = {
    "url": "https://grist.tools/samples/web/fieldnotes/index.html",
    "internal_only": False
}

res = session.post("https://grist.tools/v1/link-extract", json=payload)
print(res.status_code, res.json())
settle = res.headers.get("payment-response")
if settle:
    print(decode_payment_response_header(settle).transaction)
curl
# link-extract: $0.002 per call, paid in USD Coin on eip155:8453 via x402
# 1. Unpaid call: HTTP 402, the requirements in the payment-required header (base64 JSON) and in the body.
curl -i -X POST 'https://grist.tools/v1/link-extract' -H 'content-type: application/json' -d '{"url":"https://grist.tools/samples/web/fieldnotes/index.html","internal_only":false}'

# 2. Same call with the signed payment (an EIP-3009 authorization, EIP-712 signed: it cannot be
#    typed by hand). x-payment is accepted as the v1 alternative. HTTP 200 carries payment-response.
curl -i -X POST 'https://grist.tools/v1/link-extract' -H 'content-type: application/json' -H 'payment-signature: <base64 x402 payload>' -d '{"url":"https://grist.tools/samples/web/fieldnotes/index.html","internal_only":false}'

06Errors

CodeHTTPFor this service
invalid_input400The body does not match the schema; its fields are url and internal_only.
blocked_target403
unreachable_target424
upstream_status424
upstream_timeout424
unsupported_content_type415
too_large413
unprocessable422
internal500

Input that is rejected before the tool runs (malformed JSON, schema mismatch, body over the size limit) is never charged.

When each code is raised, and what it means for payment: /docs/link-extract.

07Questions

How much does a link-extract call cost?
$0.002 per call, paid in USDC on Base via x402, with no account and no API key.
What does link-extract return?
A JSON object with 10 top-level fields: url, source_url, final_url, http_status, base_url, links, links_total, links_returned, links_truncated and fetched_at. The example on this page is a real call over web/fieldnotes/index.html.
What happens when a link-extract call fails?
The tool answers with a typed JSON error from the errors table. Payment is settled only after the tool has produced its result; a call that fails inside the tool is never settled. A 503 upstream_unavailable can follow a settled call; the x402 payments guide covers it, 429, 402 and an uncertain 500.

08Related